PROFESSIONAL SUMMARY:

• An accomplished, results - driven IT professional wif around 6+ years of experience in the IT industry.
• 2+ years of experience as a Hadoop & Spark Developer in all phases of Hadoop ecosystem and Apache Spark.
• Experienced wif Hadoop distributions like Cloudera depth noledge on Hadoop architecture including Yarn, Pig, Hive, Sqoop, HDFS, Spark, Oozie, Zookeeper, kafka, Hbase and Impala.
• Experience in importing and exporting data from different RDBMS Servers like MySQL and Oracle into HDFS and Hive using Sqoop.
• Experience in developing Streaming Applications Kafka.
• Experienced in analyzing the data using PIG Latin scripts.
• Experience in designing table partitioning, bucketing and optimized Hive scripts using different performance utilities and techniques.
• Experience in designing tables and views for reporting using Impala.
• Experienced in Developing Spark application using Spark Core and Spark SQL.
• Experience in creating DStreams from sources like Kafka and performed different Spark transformations and actions on it.
• Rich experience in automating Sqoop and Hive queries using Oozie workflow.
• Experience in scheduling the jobs using Oozie Coordinator, Bundler and Crontab.
• Experienced in working wif different file formats - Avro, Parquet, JSON, CSV.
• Experience in different compression techniques like Snappy and Bzip2.
• Involved in development of the application using Spring boot MVC and other components of the Spring framework. Also implemented Dependency Injection using the Spring framework.
• Familiar wif developing environments like JIRA, Confluence, Jenkins, Agile/Scrum and Waterfall.
• Proficient in Manual (Blackbox), Database, functional and non-functional, negative, regression, system integration (SIT), acceptance (UAT), security, browser compatibility and system testing.
• Performed unit testing Junit testing framework and Log4j to monitor the error log.
• Strong Communication skills of written, oral, interpersonal and presentation.
• Proven ability to interface wif stakeholders to ensure solutions to meet customer requirements.
• Adept at managing multiple projects simultaneously. Strategic planning through operational and technology design for optimizing Software development, architectural design, creating various proof of concepts.
• Exceptional ability to learn new technologies and to deliver outputs in short deadlines.

TECHNICAL SKILLS:

Hadoop: Hadoop 2.2, HDFS, Hive, Sqoop, Zookeeper, Yarn, Spark, Kafka, Hbase, Impala and Hue.

Hadoop management & Security: Cloudera Manager, Hortonworks

Web Technologies: HTML, XML, CSS, SOAP

Database: Microsoft SQL Server, MySQL, SQL, RDBMS, Familiar wif NoSQL- Hbase

Programming Languages: Java, J2EE, Scala

Web Servers: Apache Tomcat 5.x

OS/Platforms: Mac OS X 10.9.5, Windows2008/Vista/2003/XP/2000/NT, Linux, Unix

Client side: CSS, HTML

Testing & Case Tools: Junit, Rational Clear Case.

Version control Tools: SVN, Git

IDEs: Intellij, Eclipse

Framework: Spring Boot MVC, CDAP

PROFESSIONAL EXPERIENCE:

Confidential, New Jersey

Hadoop & Spark Developer

Responsibilities:

• Requirement gathering from the Business Partners and Subject Matter Experts.
• Involved in complete Big Data flow of the application starting from data ingestion from upstream to HDFS, processing and analyzing the data in HDFS.
• Responsible for implementing various modules of the application using Spring Boot architecture.
• Developed Java application to integrate Collectors wif DMaaP.
• Designed CDAP application to prepare, transform and load the incoming virtual functions data into DCAE Controller.
• Used Maven as a build tool to manage dependencies in the java application.
• Responsible for deploying the image of the Java application to the Docker registry.
• Developed Kafka consumer’s and Producer’s API in Java for consuming data from Kafka topics.
• Collected the JSON data from the Kafka cluster and developed Spark APIs dat halps to do inserts and updates in Hive tables using Hive-QL.
• Using Avro file format compressed wif Snappy in intermediate tables for faster processing of data.
• UsedImpalafor faster querying the data in HDFS.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.
• Developed Spark core and Spark SQL scripts using Scala for cleansing the data and faster data processing.
• Involved in designing and developing tables in HBase and storing aggregated data from Hive Table.
• Worked on partitioning and used bucketing in HIVE tables and setting tuning parameters to improve the performance.
• Performed unit testing using Junit.
• Used Git for version control, JIRA for project tracking and Jenkins for continuous integration.

Environment: HDFS, Yarn, MapReduce, Hive, Sqoop, Oozie, HBase, Kafka, ImpalaSpark SQL, Eclipse, Oracle, Teradata, Cloudera.

Confidential, New Jersey

Java & Hadoop Developer

Responsibilities:

• Developed Sqoop jobs to import data in csv file format from Yahoo finance.
• Developed automated shell scripts to convert the csv to parquet file format and store it into HDFS.
• Designed and Configured end-to- end Oozie workflows to automate
• Developed PIG scripts for source data validation and transformation.
• Worked on analyzing/transforming the data wif Hive and Pig. DevelopedSparkscripts by using Scala shell commands as per the requirement. UsedSparkAPI over Cloudera Hadoop YARN to perform analytics on data in Hive.
• Extensively used HiveQL DDL statements for creating external tables for data analysis, used partioning and Bucketing is used for query optimization.
• Used Monte Carlo simulation to randomly generate the market features across a fixed number of trials.
• Designed, documented operational problems by following standards and procedures using a software reporting tool JIRA.

Environment: HDFS, Hive, Sqoop, Oozie, HBase, Spark SQL, Eclipse, Zookeeper, Spark core, Java, Git, Shell scripting, Cloudera.

Confidential

Java/J2ee/SQL Developer

Responsibilities:

• Captured requirements from Business and Systems Analysts and Business Users, documenting them as detailed statements and requirements.
• Worked in a Scrum Agile process wif two-week iterations delivering new features and working software at each iteration.
• Executed SQL queries to perform crud operations on customer records.
• Used various enterprise design patterns in the development of code for extensibility and maintainability (Singleton and MVC design pattern).
• Converted various SQL statements into stored procedures theirby reducing the number of Data accesses.
• Involved in development of Web Services using SOAP for sending and getting data from the external interface in the XML format.
• Developed Web services using Restful and JSON.
• Worked wif configuration management groups for providing various deployment environments set up including System Integration testing, Quality Control testing etc.
• Implemented REST Web Services wif Jersey API to deal wif customer requests.
• Developed JUnit Test Cases for Code unit test.

Environment: Java, spring core, JUNIT, SVN, XML, Maven, SQL, Jenkins.

Confidential

QA Tester

Responsibilities:

• Have coordinated wif PM, QA, Senior BAs and offshore for UAT
• Prepared test plan based on the requirement walk through given by Client BA
• Discussed issues wif the major stakeholders on calls and raised defects whenever required
• Attended defect triage calls wif developers and BAs
• Scrambled client files and sent for QA testing to test in lower testing environments
• Coordinated wif senior domain specialists and data lead to test the Pre-Production, Prod Fix Production files
• Presented test plan walk through to major stake holders and got the sign off
• Coordinated wif file transfer team in getting the file to the client on servers on weekly, biweekly or monthly basis in Production
• Participated in TCS process activities like preparing Internal WSR and Client WSR. Coordinated wif the team members for the same.
• Coordinate wif Production Support after the warranty on how to work wif client files
• Working wif business partners, analysts, developers and project managers to develop test plans, produce test scenarios and repeatable test cases through all parts of the development lifecycle, execute and sign-off.
• Created and managed defects in ALM.
• Publishing reports to onsite managers.
• Preparing estimation for the deliverables.
• Working change management coordinator to support the team during production release and security patch implementation.
• Preparing and reviewing SOP documents to assist support and halp desk team to raise the defect ticket on priority basis.

Environment: MySQL, SVN, SQL Server 2008, XML, Quality Assurance

Confidential

QA Tester

Responsibilities:

• Prepared test plans based on the requirement walk through given by the client.
• Discussed issues wif the major stakeholders on calls and raised defects whenever required.
• Designed and executed test cases in ALM.
• Prepared test execution summary report for the completed
• Prepared in-house defect tracking tool using Java, HTML, CSS,

Environment: MySQL, SVN, Quality Assurance.