Hadoop Developer Resume
Newark, DE
SUMMARY:
- Overall 15+ years of experience in all phases of development life cycle (SDLC) and more than 5 years of experience as a Hadoop Developer .
- Experienced on working with Big Data and Hadoop Distributed File System (HDFS).
- Hands on Experience in working with ecosystems like HDFS, Big Data, RDBMS, Hive, Pig, Sqoop, Flume, OoZie, SPARK, Scala, IMAPAL.
- Strong Knowledge on Hadoop and Hive's analytical functions.
- Capturing data from existing databases that provide SQL interfaces using Sqoop.
- Efficient in building hive, pig and map Reduce scripts.
- Implemented Proofs of Concept on Hadoop stack and different big data analytic tools, migration from different databases (i.e Teradata, Oracle) to Hadoop.
- Loaded the dataset into Hive for ETL Operation.
- Good knowledge on Hadoop Cluster architecture and monitoring the cluster.
- Hands on experience in IDE tools like Eclipse, Visual Studio.
- Experience in database design using Stored Procedure, Functions, Triggers and strong experience in writing complex queries for DB2, SQL Server.
- Excellent problem - solving skills, high analytical skills, good communication and interpersonal skills.
- Good Experience on L3/L2 PRODUCTION SUPPORT, also have good experience giving JOBS instruction to L2 Team.
- Participate in all walk-through meetings, project status & release calls.
- Executed various projects in Airlines, Banking, Healthcare and Telecoms Domains.
- Experienced in using agile methodology for developing the projects.
- Worked on MAINFRAMES as a developer like COBOL, JCL, DB2, VSAM, CICS, IMS-DB.
- Experienced in using XML related standards and technologies like XSD, DTD, JAXB, Xpath and XSLT.
- Experience with database like Oracle, MySQL and DB2.
- Hands on experience for Defect tracking & defect management from the discovery of defects, submitting the defects with elaboration so that Testers are able to recreate & provide the fix faster, retesting of the fixed defects & tracking the defect status in defect tracking tools till it is closed.
TECHNICAL SKILLS:
Operating Systems: WINDOWS 95/98/2000/XP, UNIX, Linux
Teradata Viewpoint, TOAD, SOAPUI, JIRA, Version: one, Clear Quest, Tomcat 6.0.20, Jenkins, MAVEN, Test NG, JUnit 4.3.1 ANT 1.6.5, Selenium 2.40.1, Java 1.5(JDK 5), ALM
Hadoop Ecosystem: Map Reduce, Sqoop, Hive, Pig, Hbase, Cassandra, HDFS, Zookeeper, Spark, Kafka, Scala, IMPALA, CLOURERA MANAGER. AWS
RDBMS/NoSQL: SQL server, HBase, Cassandra, ORACAL, TERADATA
COBOL, JCL, DB2, VSAM, CICS, IMS: DB, QMF, SPUFI, FILE-AID, CA7, Endeavor
Methodologies: Waterfall, Software Development Life Cycle (SDLC), Agile, Scrum, SDET
S RECEIVED: Spotlights, Customer Focus
PROFESSIONAL EXPERIENCE:
HADOOP DEVELOPER
Confidential, NEWARK, DE
Responsibilities:
- Analyze and define client’s business strategy and determine system architecture requirements to achieve business goals.
- Lead the team to compare critical and measurable milestone of project.
- Experienced in the performance tuning of spark, Applications for setting right batch interval time, correct level of Parallelism and memory tuning.
- Load the data into spark RDD and do in memory data computation to generate the output response.
- Optimizing of existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames and Pair RDD’s.
Environment: Hadoop, HDFS, Hive, IMPALA, SPARK, SCALA, Sqoop, TERADATA, Oracle, HBASE, Cloudera Manager, SOAPUI, TERADATA Viewpoint, GIT, Eclipse, JIRA, FILE ZILLA, JAVA, Shell Script
HADOOP DEVELOPER
Confidential
Responsibilities:
- Analyzing and Understand the software Requirement Specifications and development of USE CASES
- Involved in HADOOP Cluster design meetings and participated all client calls
- Replaced default Derby metadata storage system for Hive with MySQL system
- Executed queries using Hive and developed Map-Reduce jobs to analyze data
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS
- Developed the Pig UDF's to preprocess the data for analysis
- Developed Hive queries for the analysts
- Attending the defect triage meetings and resolving the issues
- Involved in data Ingestion by using flume, KAFKA and integrating with different API
- Involved in loading data from RDBMS system to HDFS by using SQOOP
- Involved in Spark Streaming for online analytics and Spark SQL for Hive tables.
Environment: Hive, PIG, HDFS, IMPALA, SPARK, SCALA, Sqoop, TERADATA, Oracle, HBASE, AWS, SOAPUI, GIT, Eclipse, JIRA, FILE ZILLA, JAVA
Big data Analyst
Confidential, SeaTac, WA
Responsibilities:
- Replaced default metadata storage system for Hive with MySQL system.
- Executed queries using Hive and developed Map-Reduce jobs to analyze data.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Testing load test by adding more throughputs and comparing the velocity of the application.
- Developed Hive queries for the analysts.
- Utilized Apache Hadoop environment by Hortonworks.
- Involved in loading data from LINUX and UNIX file system to HDFS.
- Supported in setting up QA environment and updating configurations for implementing scripts with HIVE.
Environment: Apache Hadoop, HDFS, Map Reducer, Hive, ALM, My Sql, UNIX, Sqoop, PIG. SPARK, SCAALA
Confidential, Denver, CO
Big Data Analyst
Responsibilities:
- Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase NoSQL database and Sqoop.
- Importing and exporting data in HDFS and Hive using Sqoop.
- Extracted files from MongoDB through Sqoop and placed in HDFS and processed.
- Experience with NoSQL databases.
- Written Hive HDFS to extract data from staging tables.
- Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
- Familiarized with job scheduling using Fair Scheduler so that CPU time is well distributed amongst all the jobs.
- Involved in the regular Hadoop Cluster maintenance such as patching security holes and updating system packages.
- Managed Hadoop log files.
- Analyzed the web log data using the HiveQL.
Environment: Hadoop, Hive, Hbase, MangoDB, Linux, Map Reduce, HDFS, Shell Scripting, Mysql, ALM
Confidential, Sacramento, CA
HADOOP Developer
Responsibilities:
- Involved in the complete testing lifecycle spanning pre-testing phase, acceptance-testing phase and also in user acceptance testing.
- Involved in entering requirements, developing test plan tree, designing the test steps, writing test cases and running test scripts using Test Management Tool ALM
- Documented test cases and expected results
- Ensured the appropriate parties review and sign-off on test cases prior to test execution
- Exported test cases from Excel and requirements from MS Word to ALM
- Involved in smoke testing and System testing
- Tested manually the various navigation steps and basic functionality of the application.
- Created positive and negative testing scenarios and tested them manually as well as using automated scripts.
- Conducted Functionality testing during various phases of the application.
- Used Check Points, Output Values, Recovery Scenarios, Regular Expressions, Object Repository Administration extensively in QTP
- Imported and exported data from and to Excel to and from QTP.
- Performed database testing using SQL, executed various DDL and DML statements
- Created Database Checkpoints in for database testing
- Regularly interacted with Developers to discuss various problems and issues
Environment: Selenium Web driver 2.24, Eclips 3.5, JIRA, SoapUI, Maven 3.0, Windows, Linux, Oracle10g,Log4J,ANT, ALM, Jenkins, IMPALA
Confidential, Orlando, FL
Software EngineerResponsibilities:
- Automated and customized the scripts for the entire application using Selenium.
- Reviewed the Business Requirement specification Document and the technical specification of the application.
- Reviewing System Requirements to define testing scope for various levels of testing (Functional, System, Regression, and UAT).
- Experience testing Transaction processing system for End Users during UAT and ensuring that payments made and received were calculated based on all the Business rules.
- Developed Test Objectives and Test Plans.
- Created manual and automated Test Cases and Test Scripts.
- Created Standard, Image, Table, Page, Text/Text Area Checkpoints.
- Parameterized the test(s) and modified steps affected by parameterization.
- Worked with developers to communicate regarding issues in the application.
- Coordinated the testing effort towards End-to-End testing to verify the proper functioning.
- Created several compiled modules.
- Performed Regression Testing.
- Wrote SQL queries to test the application for data integrity.
- Tested User Interface inconsistency and application functionality.
Environment: COBOL, DB2, CICS, VASAM
Confidential
Team Member
Technology: COBOL, JCL, VSAM, SAS
Responsibilities:
- Requirements Analysis and testing of the Product.
- Answering questions related to data raised from other application teams.
- Acquiring knowledge and documenting the various functionality of the system.
- Reviewing programs for QA.
- Involved in the Test-case preparation (UTP’s and UTR’s).
Confidential
Software Engineer
Responsibilities:
- Requirements Analysis, Design, Development and testing of the Product.
- Answering questions related to data raised from other application teams.
- Performance Tuning, Monitoring for various Jobs and job Streams by using STROBE and OPC Tools.
- Acquiring knowledge and documenting the various functionality of the system.
- Reviewing programs for QA.
- Providing functional /presentations to the team members.
- Involved in the Test-case preparation (UTP’s and UTR’s).
Confidential
Technology: IMS DB/DC, COBOL, JCL, DB2
Software Engineer
Responsibilities:
- Understanding the requirements of Existing Application Systems
- Knowledge acquisition of the existing business processes and application design
- Analysis, Design, Coding and Testing of new business requirements
- Modification of the applications as per business requirements
- Interacting with the client on regular basis to solve the issues
- Ensuring that the delivered products/services conform to the customer requirements and agreed quality standards