CAREER OBJECTIVE:

To pursue a highly rewarding career, seeking for a job in challenging and healthy work environment where I can utilize my skills and knowledge efficiently for organizational growth.

EXPERIENCE SUMMARY:

• 14 Years of extensive IT experience in Analysis, Design, Development, Testing and Implementation of software applications which includes 4.5 plus years of experience in Hadoop Technologies.
• Good experience in working with different Hadoop ecosystem components such as HDFS, HiVE, Sqoop, Scala, Spark, Yarn, Kafka, Zookeeper, PIG, HIVE, MapReduce, HBase, Oozie, Scala and Flume.
• Very good knowledge on usage of various big data ingestion techniques using Sqoop, Flume, Kafka, Native HDFS java API.
• Hands on experience in installing, configuring and using ecosystem components like Hadoop MapReduce, HDFS, HBase, ZooKeeper, Oozie, Hive, Sqoop, Kafka, Pig, Flume, YARN, SPARK, Azkaban.
• Experienced in handling large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins Transformations using Scala.
• Design and Development of Scala code/Spark Jobs for Data Ingestion into Hadoop Telecom Datawarehouse
• Extensive experience in NoSQL and SQ development.
• Experience in transforming big data requirements into Hadoop centric technologies.
• Worked on Multi Clustered environment and setting up Cloudera Hadoop echo System.
• Experience in performance tuning the Hadoop cluster by gathering and analyzing the existing infrastructure.
• Experience in writing shell scripts to dump the Sharded data from MySQL servers to HDFS Oozie.
• Experience in supporting analysts by administering and configuring HIVE.
• Experience in writing the Hive UDF’s and API’s.
• Experience in providing support to data analyst in running PIG and Hive queries.
• Developed Map Reduce programs to perform analysis.
• Experience in working with Flume to load the log data from multiple sources directly into HDFS.
• Performed Importing and exporting data into HDFS and Hive using Sqoop and Kafka.
• Responsible for complete SDLC management using different methodologies like Agile Incremental, Waterfall, etc.
• Experience in Data migration from existing data stores and mainframe NDM (Network Data mover) to Hadoop.
• Experience in configuring the Zookeeper to coordinate the servers in clusters and to maintain the data consistency.
• Experience in designing both time driven and data driven automated workflows using Oozie.
• Extensive knowledge and experience in data modeling.
• Experience in creating the Conceptual, Logical and Physical data models.
• Excellent written and verbal communication skills in fast - paced, demanding and team-oriented settings, independent and self-motivating.
• Good knowledge in Software development life cycle. Provided end-to-end system solutions for various business functions in Insurance and Banking domain projects.
• Good knowledge in ETL technologies, relational and dimensional data structures.
• Good experience and understanding of Enterprise Data warehouse(EDW) architecture and possess End to End knowledge of EDW functioning
• Experience in writing Test Plans, Test Strategies and Test Cases for various Bigdata projects implementation.
• Evaluated and implemented emerging technologies, and providing training to team members.
• Provided Hadoop hands-on training to 100-member team.

TECHNICAL SKILLS:

Big Data Eco System: Hadoop, MapReduce, HDFS, HBase, Zookeeper, Hive, Pig, Sqoop, Oozie, Flume, Kafka, Linux, Ranger, Knox, Ambari, Hue, Spark, Scala, Azkaban.

Programming Languages: C, C++, Core JAVA.

Testing/Logging: MR Unit, SOAP UI, JIRA, Remedy.

Tools: / Special Software: ANT, TOAD, Rally, SPLUNK, Maven, SVN, Jenkins, Nexus. WinSCP, Putty.

Database: Oracle 11g/10g, DB2, MySQL, MS Access.

Application Server: Apache Tomcat, Web Sphere

Operating System: Windows XP/Vista/7, UNIX, Linux

Methodologies: Agile and Waterfall.

Languages /Tools / Special Software: PL/1, COBOL, JCL, CICS, SQL, AS, MARK IV, VBA, Easytrieve, DTCN, ENDEVOR, CHANGEMAN, QMF, XPEDITOR, XICS, FILE-AID, QUICK EDIT, Telon, CMR, MoveIT, MQ Series, INFORMATICA 7.0.

PROFESSIONAL EXPERIENCE:

Confidential

Hadoop Consultant/Architect

Environment: Spark, Scala, Linux, Sqoop, Java, Hive, Unix Shell Scripting, JIRA, AGILE Methodology, SCRUM, Eclipse IDE

Responsibilities:

• Designed and developed big data analytic models to read huge files from remote server and process using spark and Scala.
• Analyse existing Data Ingestion Design Patterns and provide inputs on Performance improvement to Application Teams
• Working with Data Scientists to understand data requirements and guide developers to work with data providers to create data pipelines for analytics
• Developed Scala scripts for transforming the data by using both Data frames and RDD in Spark 1.6 and 2.X.
• Worked on tuning the performance of spark SQL and hive queries.
• Prepared technical design documents, detailed design documents.
• Importing and exporting the data using Sqoop from HDFS to Relational Database systems and vice-versa.
• Automated the workflow using shell scripts
• Installed Oozie workflow engine to run multiple Hive and spark jobs.
• Co-ordinating with other programmers in the team to ensure that all the modules complement each other well.
• Played role in project estimation and management activities.

Confidential

Hadoop Consultant

Environment: Eclipse IDE, Linux, Hadoop Map Reduce, Pig Latin, Sqoop, Java, Hive, Hbase, Hive, Flume, Unix Shell Scripting, JIRA AGILE Methodology, SCRUM, SPLUNK, Eclipse IDE, Spark, Scala.

Responsibilities:

• Developed big data analytic models for customer fraud transaction pattern detection models using Hive from customer transaction data. It also involved transaction sequence analysis with gaps and no gaps, network analysis between common customers for the top fraud patterns.
• Developed customer transaction event path tree extraction model using Hive from customer transaction data.
• Developed the spark scripts by using spark-scale shell commands
• Developed Scala scripts, UDFFs using both Data frames/SQL and RDD/MapReduce in Spark 1.6 for Data Aggregation, queries and writing data back into OLTP system through Sqoop.
• Worked on migrating Map Reduce programs into Spark transformations using Spark and Scala.
• Worked on tuning the performance of spark SQL and hive queries.
• Prepared technical design documents, detailed design documents.
• Process large data sets utilizing our Hadoop cluster.
• Designing NoSQL schemas in Hbase.
• Extensive data validation using HIVE.
• Importing and exporting the data using Sqoop from HDFS to Relational Database systems and vice-versa.
• Automated the workflow using shell scripts
• Installed Oozie workflow engine to run multiple Hive and Pig jobs.
• Assigning the work to offshore team and ensuring the work is delivered on time.
• Co-ordinating with other programmers in the team to ensure that all the modules complement each other well.
• Played role in project estimation and management activities.

Confidential

Bigdata Test lead

Environment: Hue, Oozie, Linux, Pig Latin, Sqoop, HBase, Hive, JIRA, Podium, AGILE Methodology, SCRUM, WinScp, Putty, SVN.

Responsibilities:

• Taking part in requirement analysis with end users and Data Analyst.
• Played role in project estimation and management activities.
• Assisting Data Analyst during data validation and initial assessment.
• Developed and tested functionalities using Sqoop, Hive and Podium to load the data into Data Lake.
• Ingestion and extraction of data fromCreated Hive tables and loaded data from Oracle database using Sqoop.
• Took vital role in Continues Integration testing.
• Writing Test Plans, Test Strategies and reviewing the Test Cases.
• Managed the execution of the enhancement requests from initiation to completion, coordinated production releases with Project Manager and other dependent teams
• Assigning work to the offshore team and reviewing the work done by the offshore team and making sure the working is delivered on time.

Confidential

Hadoop Consultant

Environment: Ambari, Hue, Oozie, Ranger, Knox, Linux, Pig Latin, Sqoop, Hive, Sandbox, Unix Shell Scripting, Python, JIRA WinScp, Putty, SVN, Curl, Spark.

Responsibilities:

• Designed and created Oozie workflows and prototypes for the development Project.
• Developed big data analytic models for customer fraud transaction pattern detection models using Hive from customer transaction data. It also involved transaction sequence analysis with gaps and no gaps, network analysis between common customers for the top fraud patterns.
• Worked with Sqoop import and export functionalities to handle large data set transfer between Oracle database and HDFS
• Managing and giving the technical guidance to the offshore and on-site team and tracking work status on daily basis.
• Successfully completed the POC on Spark and presented to the customer.
• Designed and implemented custom writable, custom input formats, custom partitions and custom comparators in Mapreduce.
• Converted existing SQL queries into Hive QL queries
• Extracted data from oracle database transformed and loaded into hadoop cluster (ETL).

Confidential

Hadoop Consultant

Environment: Ambari, Hue, Oozie, Ranger, Knox, Linux, Pig Latin, Sqoop, Hive, Vagrant, Sandbox, Unix Shell Scripting, Python, JIRA, AGILE Methodology, SCRUM, confluence, connect, WinScp, Putty, SVN, Jenkins, Nexus, REST API, Curl, Azkaban.

Responsibilities:

• Designed and created Oozie workflows and prototypes for the development Project.
• Created Vagrant Sandbox environments for the development team and published in Confluence.
• Setting up the standards and Best practices for the Development UAT environments.
• Giving the directions to development team and working on adhoc issue raised by the he development team.
• Successfully completed the POC on Azkaban and presented to the customer.

Confidential

Senior Developer

Environment: Eclipse IDE, Linux, Hadoop Map Reduce, Pig Latin, Sqoop, Java, Hive, Hbase, Hive, Flume, Unix Shell Scripting, JIRA AGILE Methodology, SCRUM, SPLUNK, Eclipse IDE

Responsibilities:

• Extensive experience in Hadoop map reduces as Programmer Analyst in business requirement gathering, analysis, scoping, and documentation, designing, developing and creating Test Cases.
• Developed big data analytic models for customer fraud transaction pattern detection models using Hive from customer transaction data. It also involved transaction sequence analysis with gaps and no gaps, network analysis between common customers for the top fraud patterns.
• Developed customer transaction event path tree extraction model using Hive from customer transaction data.
• Design and implement Map/Reduce jobs to support distributed data processing.
• Process large data sets utilizing our Hadoop cluster.
• Designing NoSQL schemas in Hbase.
• Developing map-reduce ETL in Java/Pig.
• Extensive data validation using HIVE.
• Importing and exporting the data using Sqoop from HDFS to Relational Database systems and vice-versa.
• Developed and executed hive queries for de normalizing the data
• Installed and configured Hadoop Cluster for development and testing environment.
• Automated the workflow using shell scripts
• Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
• Installed Oozie workflow engine to run multiple Hive and Pig jobs.

Confidential

Developer

Environment: Eclipse IDE, Linux, Hadoop Map Reduce, Pig Latin, Sqoop, Java, Hive, Hbase, Hive, Flume, Unix Shell Scripting, JIRA AGILE Methodology, SCRUM, SPLUNK, Eclipse IDE

Responsibilities:

• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with PIG
• Implemented complex map reduce programs to perform map side joins using distributed cache
• Designed and implemented custom writable, custom input formats, custom partitions and custom comparators in Mapreduce.
• Converted existing SQL queries into Hive QL queries
• Extracted data from oracle database, transformed and loaded into hadoop cluster (ETL)
• Developed the UDF's in Pig, Hive using Java
• Responsible for creating Hive tables based on business requirements
• Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for efficient data access
• Experienced in defining workflows using oozie
• Developed Hive tables to transform, analyze the data in HDFS.
• Moving data from HDFS to RDBMS and vice-versa using SQOOP and Kafka.
• Co-coordinating with other programmers in the team to ensure that all the modules complement each other well.

Confidential

Application Lead / System Analyst

Technology and Tools: MS Access, VBA, core JAVA, Macros, Excel, SOAP UI, AGILE Methodology, SCRUM, Unix Shell Scripting, Eclipse 3.2, JIRA, Tomcat 5.5, SQL.

Responsibilities:

• Involved in various phases of Software Development such as modelling, system analysis and design, code generation and testing using AGILE Methodology
• Participated in daily Stand up meetings with Scrum Master.
• Designed, developed and deployed application using Eclipse and Tomcat application Server.
• Classes are designed by using Object oriented Design(OOD) concepts like encapsulation, inheritance etc
• Involved in unit integration, bug fixing, acceptance testing with test cases, code reviews.

Confidential

Developer

Responsibilities:

• Analyse and provide solutions to resolve Incidents and Problems with in Service Level Agreement (SLAs) defined and ensure System availability.
• Support 24/7 user and production support activities from offshore. Analyze the problem, approximate the turnaround time, involve the crisis management team (if required), and resolve the problem. Document the resolution for next day’s status meeting.
• Reviews, Code Walk-through, Release sign-offs and implementations.
• On site-Offshore co-ordination.

Confidential

Tech lead

Technology and Tools: COBOL, JCL, SQL, PL/1, IMSDB, DB2, VSAM, EXPEDITOR, SPUFI, FILE AID, MQ Series, Informatica.

Responsibilities:

• System analysis of new requirement in securities pricing function. Create technical specifications and develop COBOL, CICS & IMS DB, DB2 code for the assigned unit of work.
• Create test plans and develop test scripts. Perform unit testing, log, track and fix defects.
• Participate in project meetings, provide status and effort information for the assigned tasks and highlight issues/risks.
• Creating the Conceptual, Logical and Physical data models.
• Perform system integration test (SIT), document results, fix assigned SIT defects and provide status for tracking.
• Present test results to the onshore stakeholders - internal audit team, development managers, and testing managers.
• Review of design document.
• Prepared High-level analysis documents, effort estimates, system test plans and scripts.
• Developing ETL Mappings using Informatica Powercenter. prepared technical design/specifications for data Extraction, Transformation and Loading.

Confidential

Tech lead

Technology and Tools: COBOL, JCLSQL, IMSDB, DB2, VSAM, EXPEDITOR, SPUFI, FILE AID, CHANGEMAN, MQ Series.

Responsibilities:

• Responsible for managing team of 4 onshore members and successfully delivered 3 releases of JAPA product on time and within the allocated budget.
• Offshore Technical Lead and Owner of Technical Deliverables.
• Responsible to keep track of deliverable status and time lines.
• Responsible for Risk and Issue escalations in Delivery.
• Provided management with Daily Defect Status Report.
• Change request estimation, design and implementation.
• Coordinated work between various functional and non-functional teams.
• Triaged current and backlog defects, defect assignment, and defect fixing.
• Performed impact and gap analysis and responsible to identify change requests.
• Provided solution to application development team.
• Performed application deployments and defined deployment process.
• Analysing the requirements and providing the HLE documents.
• Prepared A&D documents, HLE documents, effort estimates, system test plans and scripts, involved in client meetings, quality process, reviews & code walk through.
• Performed program & Specification reviews.

Confidential

Developer

Technology and Tools: COBOL, JCL, SQL, IMSDB, DB2, VSAM, EXPEDITOR, SPUFI, FILE AID, DTCN, MQ Series.

Responsibilities:

• System analysis of new requirement in securities pricing function. Create technical specifications and develop COBOL, CICS & DB2 code for the assigned unit of work.
• Create test plans and develop test scripts. Perform unit testing, log, track and fix defects.
• Participate in project meetings, provide status and effort information for the assigned tasks and highlight issues/risks.
• Perform system integration test (SIT), document results, fix assigned SIT defects and provide status for tracking.
• Present test results to the onshore stakeholders - internal audit team, development managers, and testing managers.
• Review of design document
• Prepared High level analysis documents, effort estimates, system test plans and scripts.