SUMMARY

• Having 8+ years of experience in all phases of Software Application requirement analysis, design, development and maintenance of Hadoop/Big Data application and web applications using java/J2EE technologies.
• Having 3+ years of hands on experience in Big Data applications using Hadoop Map Reduce framework, Spark, Pig, Hive, Sqoop, Flume, Oozie, Zookeeper in a range of industries such as Financing sector and Health care.
• In depth understanding of Hadoop Architecture including YARN and various components such as HDFS, Resource Manager, Node Manager, Name Node, Data Node and MR v1 & v2 concepts
• Extensive Experience in loading data using Hive and writing scripts for data transformations using Hive and Pig.
• Experience in Extraction, Transformation and Loading (ETL) of data from multiple sources.
• Experience in importing and exporting data using Sqoop from Relational Database Systems to HDFS and vice - versa
• Experience in writing workflows using Oozie..
• Knowledge on installation and administration of multi-node clusters using Cloudera Hadoop and Apache Hadoop.
• Experience in writing Impala scripts for fast access to data.
• Strong knowledge on No SQL solutions HBase.
• Well versed with different SDLC methodologies with Agile and waterfall models.
• Well versed with UNIX and Linux command line and shell script.
• Experience in logging and tracing tools like log4j on web applications.
• Hands on experience with Spark Core, Spark SQL and Data Frames/Data Sets/RDDs.
• Experience in developing Apache Spark programs using Java, scala.
• Strong knowledge in Spark Streaming and Kafka, flume.
• Replaced existing MR jobs and Hive scripts with Spark SQL & Spark data transformations for efficient data processing.
• Worked on building hadoop cluster inAWSCloud on multiple EC2 instances.
• Hands on experience on AmazonAWSconcepts like S3, EMR & EC2 web services which provides fast and efficient processing ofBigData.

TECHNICAL SKILLS

Big Data / Hadoop: Apache Hadoop, Map Reduce, HDFS, Spark, Hive, Pig, Sqoop, Flume, Impala, Zookeeper, Oozie, Kafka.

Spark: Spark Core, Spark Sql, Spark Streaming.

No SQL Solutions: HBase, Cassandra.

Languages: Java, Scala, Python and SQL

Cloud Amazon: EMR, EC2, EBS,S3, Lambda,Redshift

Build Tools: Maven2, Sbt.

Database: RDBMS Oracle9i, Oracle 10g, MySQL, Teradata.

IDE: Development Tools RAD, Eclipse

Operating Systems: UNIX, Windows, LINUX.

Conceptual Frameworks: AGILE

PROFESSIONAL EXPERIENCE

Confidential, Charlotte, NC

Hadoop Developer

Responsibilities:

• Design data storage, schema and partitioning system.
• Developed spark programs for business trasformations applying business logic by combining multiple data sets for faster execution.
• Writing spark programs creating data frames by dynamically applying schema to the flat source files by reading target hive table schema using hive context.
• Writing data to parquet tables both non partitioned and partitioned tables by adding dynamic data to partitioned tables using spark.
• Developed hive scripts to perform business transformations on the data.
• Involved in performance tuning of Hive from design, storage and query perspectives.
• Developing a common framework to import the data from Teradata to HDFS and to export to Teradata using sqoop.
• Creatingimpala views on top of Hive tables for faster access to analyze data through toad.
• Support architecture, design review, code review, and best practices to implement a Hadoop architecture
• Developed UDFs in java for hive.
• Preparing JIL scripts for automating the workflows using Autosys.

Environment: CDH5.8, HDFS, Apache Spark, Hive, Sqoop, Oozie, Impala, Teradata, Linux, Java, Eclipse, SQL Assistant, Autosys.

Confidential - Hartford, CT

Hadoop Developer

Responsibilities:

• Assess current and future ingestion requirements, review data sources, data format and recommend processes for loading data into Hadoop.
• Converted Netezza based processing queries by understanding the business to Hadoop processing.
• Developed Map Reduce programs by mapping multiple datasets and applying business logic for reducing multiple staging data read and write operations in first phase.
• Developing hive scripts for business transformations.
• Automated the jobs with Oozie and scheduled them with Autosys.
• Developed Sqoop import scripts for importing reference data from Netezza.
• Developing UDFs in java for hive.
• Preparing JIL scripts for automating the workflows using Autosys.
• Migrated our existing application intoAWSCloud, designed anAWSinfrastructure using VPC, EC2, S3, IAM.
• MonitoringAWSinfrastructure services Amazon Simple Storage Service (Amazon S3), Amazon Elastic Compute Cloud (Amazon EC2) and Elastic MapReduce (EMR).

Environment: CDH5.3, HDFS, Map Reduce, Hive, Pig, Sqoop, Oozie, Impala, Netezza, Teradata,Linux, Java, Eclipse, SQL Assistant.

Confidential

Java Developer

Responsibilities:

• Involved in High level and detailed design, testing, and implementation of the applications.
• Developed Struts based action classes and struts configuration files.
• Developed ORM mapping files and HQL queries using hibernate for data retrieval from the backend database.
• Developed spring configuration files for dependency injection using spring IOC and declarative transaction management with Spring AOP aspects Writing stored procedures and functions using PL/SQL
• Quality assurance. Working closely with testing teams and participate in walkthroughs of test Strategy and test plan and eventually working on test cases during unit testing, integration testing and user acceptance testing.
• Unit testing, debugging and set test conditions based on code specifications. Perform peer code reviews and performance enhancements.
• Development of Functional Specification Documents, Design Specification Documents, UserRequirement Analysis, and Test plans
• Developed UI components using JSP, JavaScript.

Environment: Java, J2EE, JSP, struts1.2, Oracle 9i, Web Logic Application Server, Spring 2.5, Hibernate, SVN and Quality center, Eclipse3.1, jasper reports, Active widgets.

Confidential

Java Developer

Responsibilities:

• Active co-ordinations with Business teams to gather different web content for different modules.
• Studying the current system and effort estimation for the new requests.
• Interact with testing team to finalize the test scripts.
• Documented the designs so as to facilitate the personnel to understand the process and in corporate the changes as and when necessary.
• Developed Struts action classes and configuration files for request handler mappings.
• Developed hibernate configuration files for mapping to the backend tables for data retrieval from the backend database.
• Regular deployments of new code on tomcat.
• Performing the Unit testing and basic functional testing with different set of data.

Environment: Java, J2EE, struts, JSP, Oracle 9i, Tomcat, Hibernate3.x, SQL, PL/SQL, SVN and Quality center, Eclipse3.1, TOAD.