SUMMARY:

• Software Professional with 8+ years of experience in Hadoop and Web application development in Java/J2EE technologies and excellent management and communication skill.
• Knowledge of Large Data Clusters with ability to work in administration of large data clusters in big data environments using Hadoop technologies.
• Well acquainted with the complete software development life cycle: Design, Development, Testing, Deployment and Documentation.
• Excellent project implementation skills, a team - player with good communication, problem-solving and analytical skills and ability to quickly adapt new environments and technologies.
• Eight (8+) years of IT Experience in various domains such as Financial Services and Manufacturing.
• Experience in Hadoop Technologies and its components like Hortonworks, HUE, HDFS, Hive, Impala, HBase, Pig, Spark, Scala, Spark-Sql, Kafka, Oozie, Sqoop, Mapreduce.
• Design and build Data pipelines using tools and Frameworks in Hadoop ecosystem.
• Design and build Data ETL Piplelines to automate ingestion of structured and unstructured data.
• Design and build Data Pipelines to facilitate data analysis.
• Experience in developing Spark programs using Scala, RDD and SparkSQL for processing and analytics of data.
• Experience in AWS technologies - Amazon EC2,Amazon S3,Athena,IAM,Auto Scaling.
• Experience in Amazon EC2 infrastructure for computational tasks and Simple Storage Service (S3) as Storage mechanism.
• Design recommend best approach suited for data movement from different sources to HDFS using Apache/Confluent Kafka.
• Implementation of data ingestion large volumes of data to Kafka.
• Implementation and adding the performance tuning parameters for Kafka Producers,Broker,Rest Proxy and Consumers for optimum performance.
• Wrote Kafka producers to stream the data from external rest APIs to Kafka topics.
• Implementation in Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control center.
• Experience in creating producers, consumers and consumer groups for handling data.
• Experience working with setting up Confluent Connectors & Replicators to replicate data from one data center to another.
• Experience in Talend ETL Big data tool.
• Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.
• Experience in big data application phases like data ingestion,data analytics and data visualization.
• Experience in designing and developing data lakes using Real time data ingestion into AWS S3.
• Used Avro,Parquet and ORC data formats to store data in HDFS.
• Experience in usage of Hadoop distribution with Cloudera.
• Experience in creating external Hive tables,partitioning,bucketing, loading and aggregating data using HIVE.
• Expertise as a developer in Java J2EE technologies and languages that include Core Java, J2EE, JSP, Servlet, Struts framework, Spring, Hibernate, SQL, REST Web-Services, SOAP and Object Oriented methodology.
• Worked on Hadoop Framework and its ecosystem like MapReduce Programming, HDFS, Hive, Pig, HBase, Spark-Sql,Sqoop, Oozie.
• Hands on Experience in Maven, Git and Eclipse.
• Extensive experience in design, development and implementation of software applications in distributed environment like JSP, Servlet.
• Experience in Object-Relational mapper, Hibernate, for data access layer.
• Expertise with Deployment in UNIX Environment and UNIX Commands.
• Experience in handling Client Interactions and ability to quickly adapt to new environments and learn new technologies.
• Good understanding of Software Development Life Cycle (SDLC) and Project Life Cycle.
• Excellent communication, good leadership and problem solving skills.

TECHNICAL SKILLS SUMMARY:

Operating Systems: Windows 2000,Unix, Windows XP

Programming Languages: C, C++, Java, Scala,J2EE, SQL, HTML.

Technologies: Hadoop, Spark,Java Servlets, JSP, JDBC, Log4J.

Framework: Hortonworks,Struts, Hibernate, Spring, JUnit, Pig, Hive.

Databases: Oracle 9i SQL,Greenplum,WinSQL.

Web/Application Servers: Apache-Tomcat, Cygwin.

IDEs: Talend, Redwood, Perforce, JIRA, QuickBuild, Autosys, Eclipse, EPMO (Project management Tool).

Web Designing: HTML, CSS, XML.

PROFESSIONAL EXPERIENCE:

Confidential, Rosemead, CA

Technology Lead - Hadoop

Responsibilities:

• Developing applications using Hadoop Big Data technologies- Hue, Hive, Impala, Spark,Scala, Spark-Sql, Kafka, Map-Reduce, Scala, Hbase and Oozie.
• Developing Spark applications using Scala, RDD and SparkSql for processing of data.
• Importing data using Spark RDD and performing transformations, actions on RDD.
• Imported data from AWS S3 into Spark RDD and performed transformations and actions on RDD.
• Implementation and adding the performance tuning parameters for Kafka Producers,Broker and Consumers for optimum performance.
• Developing Kafka producers to stream the data from external rest APIs to Kafka Topics.
• Experience in Kafka Connect to fetch data from database into Kafka Topic through Kafka Connect Source and Sink Connectors and using Schema Registry to maintain the Schema level details for the Kafka.
• Development of KSQL scripts for querying the data recieved in Kafka.
• Integration of Spark along with Kafka to better movement of data.
• Experience in creating external Hive tables,partitioning,bucketing, loading and aggregating data.
• Worked with various file-formats like Avro, Parquet and ORC.
• Used Spark-Sql to load structured and JSON data and create schema RDD and loaded into Hive Tables using Spark-Sql.
• Imported data from database source using Sqoop into HDFS.
• Experience in using Redwood scheduler to schedule the jobs.
• Experience in designing and development of data lakes using Real time data ingestion into AWS S3.
• Creating HiveQL and Impala scripts for creating the tables and populating the data.
• Experience in using Hive Partitions, Bucketing in Hive and designed both Managed and External tables in Hive to optimize performance.
• Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.
• Designed data Ingestion framework to load business validated data of around 3 million customers into Hadoop Data Lake to store Historical and Incremental data.
• Designing applications based on identified architecture and support Implementation design by resolving complex technical issues.
• Experience in Talend ETL Tool for handling huge volume of data.
• Involved in Customer discussions during requirement analysis with knowledge of requirement elicitation and analysis tools and techniques like story boarding, Agile user stories to understand system requirements and non-functional requirements like performance,security, availability and scalability etc.
• Creating source system data snapshot in Non-production Hadoop environment.
• Design and develop data routing techniques in the data ingestion framework and provide support to write test cases to valid the logic.
• The Enterprise data lake is used to provide support for various uses cases including Analytics, processing, storing and Reporting of voluminous, rapidly changing, structured and unstructured data.
• Processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture.

Environment: Hadoop Big Data, Hue, Hive,Impala, Spark-Sql, Sqoop, AWS S3, AWS EC2,Hbase, Oozie, HiveQL, XML, Log 4j, Redwood, Autosys, JIRA, Perforce, Quick Build, Eclipse, UNIX

Confidential, Irvine, CA

Technology Lead - Hadoop.

Responsibilities:

• Developing applications using Hadoop Big Data technologies- Hue, Pig, Hive, Impala,Spark, Map-Reduce, Kafka, Hbase, Spark-sql and Oozie.
• Analyzing and Transformations of Data of Investments and Funds with Hive and Pig.
• Planning and Conducting collaboration sessions and requirements walkthrough with concerned stakeholders (Business SMEs, Architects, Tech Leads, Developers, QA).
• Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.
• Creating HiveQL scripts for creating the tables and populating the data.
• Developing Map reduce jobs using apache commons components.
• Involved in Requirement Analysis, Design, and Development.
• Export and Import data into HDFS, HBase and Hive using Sqoop.
• Worked closely with the business and analytics team in gathering the system requirements.
• Load and transformation of large sets of structured and semi structured data.

Environment: Hadoop- Pig, Hive, Sqoop, Hbase, Oozie, HiveQL, XML, Log 4j, Autosys, JIRA, Perforce, Quick Build, Eclipse, UNIX

Confidential, Madison, WI

Technology Lead - Hadoop.

Responsibilities:

• Analyzing and Transformations of Data of Auto Insurance and Life Insurance Policies with Hive and Pig.
• Experience in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake.
• The Enterprise data lake is used to provide support for various uses cases including Analytics, processing, storing and Reporting of voluminous, rapidly changing, structured and unstructured data.
• Processing of large datasets using Hortonworks Hue Platform.
• Developing applications involving Hadoop Big Data technologies- Hue,Pig,Spark, Hive, Impala, Map-Reduce,Scala, Hbase and Oozie.
• Experience in Talend Big data tool for handling huge volume of data.
• Creating HiveQL and Impala scripts for creating the tables and populating the data.
• Developing Map reduce jobs using apache commons components.
• Supporting code/design analysis, strategy development and project planning.

Environment: Hadoop - Hortonworks, Pig, Hive, Sqoop, Hbase, Oozie, HiveQL, XML,Log 4j,Autosys, Cygwin,Eclipse, UNIX

Confidential

Programmer Analyst- Hadoop.

Responsibilities:

• Worked on applications involving Hadoop Big Data technologies- Pig, Hive, Map-Reduce, Hbase, Spark and Oozie
• Worked on importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Analyzing/Transforming data with Hive and Pig.
• Developed Oozie workflow for scheduling the job.
• Developing Java Web Applications using JSP and Servlets, Struts 2.0,Hibernate,Spring, REST Web Services, SOAP and Oracle.
• Configured using Hibernate for the Database Connection and Hibernate Query Language (HQL) to add and retrieve the Information from the Database.
• Created and executed test cases in Junit for unit testing of application.

Environment: Hadoop - Hortonworks,Pig, Hive, Sqoop, Hbase, Oozie, HiveQL, XML,Log 4j,Autosys, Eclipse, UNIX.

Confidential

Hadoop Developer

Responsibilities:

• Worked on applications involving Hadoop Big Data technologies- Pig, Hive, Map-Reduce, Hbase, Oozie.
• Involved in understanding requirements, functional specifications, designing documentations and testing strategies.
• Developed Map reduce jobs in java using apache commons components.
• Developed UDFs in Java as and when necessary to use in PIG and HIVE queries.
• Provided support to the testing team and supporting UAT functional testing.
• Involved in development, build and testing.
• Coordinated with various stakeholders such as the End Client, DBA Teams, Testing Team and Business Analysts.
• Worked on importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Involved in Unit Testing using Junit.
• Involved in UI designing, Coding, Database Handling.
• Provided support to the testing team and supporting UAT functional testing.
• Analyzed and resolved issues critical production issues (Severity # 1 issues).

Environment: Hadoop - Pig, Hive, Sqoop, Hbase, Oozie, HiveQL Java, JSP, Spring, Hibernate 3.0, Struts framework, XML,Log 4j, Eclipse, UNIX.

Confidential

Senior Software Engineer

Responsibilities:

• Developing Java Web Applications using JSP and Servlets, Struts 2.0, Hibernate, Spring, Rest Web Services, SOAP.
• Worked over the entire Software Development Life Cycle (SDLC) as a part of a team as well as independently.
• Responsible for doing validations and cleansing the data.
• Finding the right joins logics and creates valuable data sets for further data analysis.
• Developed various modules for Invoice Processing, Payment and Refund.
• Deployment and support of Web sphere and JBOSS servers in Putty using UNIX commands.
• Developed Java code with minimal defects.
• Developed and modified SQL code.
• Writing SQL queries to query the database and providing data extracts to users as per request.
• Updated technical documents as per the functionality
• Provided support to the testing team and supporting UAT functional testing.
• Prepared UTP for testing the changes in the system and supporting UAT functional testing.
• Performed knowledge transfer and training to the Application Development Team.

Environment: Java, JSP, Servlet, Spring, Hibernate 3.0, Struts framework, HTML, XML, Log 4j, Eclipse, UNIX.