SUMMARY

• Having 13 years of experience in application development, enhancement, maintenance, support and testing.
• Around 5 years of experience working with Big Data using Hadoop Eco - system components (HDFS, MapReduce (MR Version1/YARN), Pig, Hive, HBase, Sqoop, Flume, Oozie, Zookeeper).
• Hands on experience in using Hive and Pig scripts to perform data analysis, data cleaning and data transformation.
• Hands on experience in capturing data and importing using Sqoop from existing relational databases (Oracle, MySQL, DB2) with the help of connectors and fast loaders.
• Solid experience in Storage, Querying, Processing and Analysis of Big Data using Hadoop framework.
• Good Experience in managing and reviewing Hadoop log files.
• Involved in analysis, design, application development & testing using Hadoop Framework.
• Experience with Oozie Workflow Engine in running workflow jobs with actions that run Hadoop jobs.
• Experience in using different file formats like Parquet, AVRO, ORC and CSV using different compression Techniques.
• Excellent understanding of Hadoop Architecture and different daemons of Hadoop cluster, which includes Job Tracker, Task Tracker, Name Node, and Data Node and YARN daemons like Resource Manager and Node Manager and Job History Server daemon.
• Good understanding of Hadoop/Spark design principals, cluster connectivity and performance tuning.
• Having good knowledge of SparkSQL and Spark using Scala.
• Worked on Version control systems like SVN, SCCS and Librarian.
• Proficiency knowledge in parallel processing using MapReduce and Spark.
• Strong in preparation of Unix shell script and Python.
• Experience in mainframe technologies using COBOL, PL/1, JCL, VSAM and DB2.
• Quick learner and good team player. Possess good analytical and communication skills.
• Good understanding and experience with Software Development methodologies like Agile and Waterfall.

TECHNICAL SKILLS

Programming Languages: Java, Scala, Python, shell Scripting, Cobol, JCL, JavaScript.

Hadoop Technologies: HDFS, MapReduce, Hive, HBase, Pig, Spark, Oozie

Databases: DB2, MySQL, HBase, Cassandra, Oralce

Operating Systems: Linux, UNIX, Windows, IBM Mainframe Z/OS

IDE/Testing Tools: Eclipse, EGL, VAG

Tools: Maven, Scala Build Tool (SBT), SVN, Change, SCCS

Distribution: Apache Hadoop, MapR and Hortonworks framework

PROFESSIONAL EXPERIENCE

Confidential

Sr. Hadoop Developer

Responsibilities:

• Monitoring/Maintaining day-to-day batch jobs using Event Engine.
• Interacting with SOR team to fix production data issues.
• Performing data transformation using Pig and Hive.
• Performing Snapshot, full refresh and Load append data refresh.
• Providing support for Confidential raw data and ODL ingestion, EFS Big Data applications
• Providing weekly/monthly status reports to customer.
• Involving status report discussion with customer.
• Onsite Technical Lead - Delegate & review the tasks for offshore team.
• Involved in Knowledge transition activities between application owner and offshore team.
• Deploying the code into the various environments and monitored.
• Monitor workflows in production and RCA for failed workflows.
• Performing ETL process using Datameer and Talend.
• Performing code change management (Unix shell script, Pig and Hive).
• Involving in code changes in production environment.
• Performing data loading into Parquet file Hive tables.
• Analyzing code and fixing the production issues.
• Understand the application architecture and explain current Platform architecture to the stakeholders.

Environment: Pig, Hive, Apache Sqoop, Python, Oozie, MapR, Datameer, SVN, Event Engine, HDFSShell script and Talend.

Confidential

Hadoop Developer

Responsibilities:

• Understand the SRS and prepared technical specification document.
• Prepared the test data as per the requirement and load into Development/Test environment.
• Developed Data Ingestion and data clearing scripts using Shell script.
• Design and develop the Hive DDL according to Data source and DW Schema.
• Design and Developed Sqoop and file import jobs to HDFS.
• Developed Pig script to process Unstructured data and loaded into Hive tables using HCatalog.
• Performed Incremental and Full refresh tables.
• Data loaded RC, ORC, Parquet file types in Hive tables.
• Hive, Oozie metadata set up using Oracle database.
• Developed Hive UDF based on requirements.
• Design and Develop ETL workflow in Hive using Oozie workflow.
• Developed Pig UDFs using piggybank and implemented in Pig.
• According to business flow change modify Oozie workflow.
• Understand the application architecture and explain current Platform architecture to the stakeholders.
• Deploying the code into the various environments and monitored.

Environment: HDFS, Pig, Map Reduce, Hive, Apache Spark, Oozie, Sqoop, Cloudera Manager, Flume & Scala.

Confidential

Technical Lead

Responsibilities:

• Analyzing existing programs.
• Performed code changes to existing programs.
• Preparation of Quality deliverables.
• Providing status report to client and senior Management.
• Onsite - Offshore co-ordination.
• Monitoring & supporting Control-M batch processing.
• Provided technical training and system knowledge to team members.

Environment: JCL, OS/390,COBOL, DB2, Changeman, and Eztrieve.

Confidential

Technical Lead

Responsibilities:

• Involved in analyzing production issues.
• Preparation of technical understanding documents.
• Performing Unit & System integration testing.
• Providing knowledge transfer to offshore team members.
• Preparation of Quality deliverables.
• Providing status report to client and senior Management.
• Onsite - Offshore co-ordination.
• Monitoring & supporting Control-M batch processing.

Environment: COBOL, Oracle, Unix, My Service, Clear view, Shell script.

Confidential

IT Consultant

Responsibilities:

• Understanding Project specification documents.
• Preparing Component specifications.
• Performed coding using COBOL and PL/1 to develop new programs.
• Prepared Unit test cases.
• Performed Unit & System testing.
• Delivery of tasks in time.
• Conducting status meeting with customer.
• Discussions with customer about tasks..
• Support other team members in training, Problem fixing and coding.
• Performing system analysis.
• Fixing production problems.

Environment: OS/390,Z/OS, COBOL, DB2, PL/1, Servlets, EGL 7.1, Data Manager, Librarian, Test Director, CCS, Service Center.

Confidential

Mainframe Developer

Responsibilities:

• Understanding the Existing System.
• Prepared component specifications and low-level design documents.
• Developed programs to extract data from IMS DB into files.
• Developed programs to insert data into DB2 database.
• Prepared Unit Test Plan and Unit Test cases.
• Performed Unit Testing.

Environment: COBOL, JCL, VSAM, PL/1, DB2, IMS DB and Librarian.