PROFESSIONAL SUMMARY:

• Over 9 years of IT experience in which 5 years as a Hadoop Developer. Work experience includes Analysis, Design and Development of complex Hadoop eco systems.
• Worked extensively on Cloudera Distribution platform. Has knowledge on MapR as well.
• In depth knowledge of MapReduce, Sqoop, Hive, Impala, Oozie, Kudu, Pig and Spark/Scala.
• Developed Spark - SQL applications with Scala for faster processing of structured data.
• Imported and Exported data from RDBMS to Hadoop Vice-versa using Sqoop.
• Developed data transformation scripts using Pig Latins.
• Good exposure loading data into Hive External tables, Impala, Hive Serde, Impala and performing ACID transactions using Apache Kudu.
• Developed Hive CSV Serde, Hive JSON Serde, Hive Regex Serde for structured and semi structured record parsing.
• Experience in designing both time driven and data driven automated workflows using Oozie.
• Development experience on different code editors like Eclipse, Intellij.
• Excellent understanding of RDBMS like Oracle, SQL Server.
• Data Visualization with Apache Hue.
• Daily code check-ins to SVN or GitHub repositories.
• Prepared documentations like User requirement spec, Impact analysis, Technical Design, System test case document with ISO 9001 and CMM level quality standards.

TECHNICAL SKILLS:

Big Data Ecosystems: HDFS, MapReduce, Pig, Hive, Sqoop, Oozie, Kudu, Impala & Spark/Scala

Hadoop Distributions: Cloudera, MapR

Programming Languages: Core Java, Scala, SQL

Databases: Oracle, SQL Server, DB2

Web Interface: Apache Hue

Development IDEs: Eclipse, Intellij

Version Control Tools: SVN, Git Hub, Bit Bucket

Build Tools: Maven, SBT

Scripting Languages: Linux Shell scripting

Secondary Skill: Mainframes

PROFESSIONAL EXPERIENCE:

Confidential, Minneapolis, MN

Lead Developer

Responsibilities:

• As a Lead, Responsible for complete development life cycle of ICE v2.0 project.
• Worked on Cloudera Distribution Platform (CDH).
• Created Data Lake by extracting customer’s data from various data sources into Hadoop that includes data from RDBMS, CSV. Used Shell scripting, Sqoop for this.
• Created JDBC programs to hit information schema & generated a file of Sqoop commands for seamless execution.
• Convert existing stored procedures using Spark-SQL & process the data.
• Performance tuning of Hive, Impala queries & Spark jobs.
• Built Oozie workflows for executing Shell, Sqoop, Hive, Spark jobs.
• Performed Manual, Automated testing to compare Source and Target data.
• Calculated data samples using Kudu for each product category that manufactured by Confidential & reported data samples on weekly basis.

Technology Stack: Oracle, SQL Server, Core Java, Linux, Sqoop, Hive, Cloudera, Spark with Scala, Kudu, GitHub, Hue and Oozie.

Confidential, Tampa, FL

Lead Developer

Responsibilities:

• Long term business need is to provide best-in-class deduplication across different platforms (Television, Digital and Social).
• Responsible for driving technical design, code development, quality excellence, timely delivery of TCR.
• Created a modular architecture for an entire application.
• Worked on Cloudera Distribution Platform (CDH).
• Performance benchmarking of all Oozie jobs.
• Prepared Shell scripts for landing various files from SFTP to HDFS.
• Historical data importing using Sqoop from Netezza .
• Data aggregation, batch and incremental data processing using HiveQL, Impala, Spark .
• Appropriate use of Hive SerDe wherever necessary.
• Calculate total content ratings & unique audience for each platform.
• Develop Data quality checks in IMPALA & execute them using Oozie Shell action .
• Production Support on rotational basis.

Technology Stack: Netezza, Core Java, Linux, Python, Impala, Hive, Cloudera, Hue, Spark with Scala, SVN and Oozie.

Confidential

Associate Lead

Responsibilities:

• Defining Big Data solutions - understanding client use cases and workflows and translate them into engineering deliverables.
• Worked on MapR Distribution Platform.
• Assisting in the development of technical design documents.
• Build highly complex enhancements using MapReduce & Schedule those from Tidal & resolve complex bugs.
• Customized Java classes, HiveQL, IMPALA are used wherever fits in architecture.
• Build and execute unit test cases as per unit test plans.
• Identify architecture/design issues - risks. If necessary, escalate them in a timely manner.
• Communicate management and development teams via WebEx about project deliverables.

Technology Stack: Linux, Sqoop, Java, Hive, Vertica, Map Reduce, Impala, MapR.

Confidential

Senior Software Engineer

Responsibilities:

• Responsible for guiding the full lifecycle of a Hadoop solution including requirements gathering, platform selection, reviewing functional & technical architecture design, development, testing and deployment.
• Created shell scripts (SCP commands) to fetch logs in Landing Server & copy them to HDFS.
• Used Sqoop to get Mainframe Datasets to HDFS.
• Union HDFS files with same schema for generate large datasets.
• Data Transformation and File processing using Pig scripts.
• Staging processed output of Pig, existing Oracle data into Hive externals.
• Developing MapReduce jobs for reading & processing huge set of logs.
• Building a workflow, Job Scheduling using Oozie.
• Reporting and fixing complex problems on Daily basis.
• Interacting with the clients, Business Analysts everyday regarding the status of implementation of business rules and data migration.
• As an IQA (Internal Quality Analyst), following standards in maintaining documents, Preparing metrics in terms of schedule-efforts-defects etc.

Technology Stack: Linux, Sqoop, Pig, Hive, Map Reduce, Oozie, Cloudera.

Confidential

Software Engineer

Responsibilities:

• Analyzing the code changes required as per Business service Unit and preparing the Impact Analysis document.
• Creating Technical Specification Development based on the Functional Spec.
• Coding as per Technical specification document and build code using Core Java, PLSQL, Mainframes.
• Unit Testing.
• Fixing application bugs identified in Unit Testing.
• Documenting the resolution details for the problem reported.
• Maintaining Application Source code using the Standard Version Control norms.
• Post Implementation Support after an enhancement/development of the Project.

Technology Stack: Core Java, SQL, Mainframes