PROFESSIONAL SUMMARY:

• IT professional with 8+ years of experience in Analysis, Design, Development, Integration, Testing and maintenance of various applications using JAVA /J2EE technologies along with 5+ years of Big D Confidential /Hadoop experience.
• Experienced in building highly scalable Big - d Confidential solutions using Hadoop and multiple distributions i.e. Cloudera, Horton works and NoSQL platforms (HBase& Cassandra).
• Expertise in big d Confidential architecture with Hadoop File system and its eco system tools MapReduce, HBase, Hive, Pig, Zookeeper, Oozie, Flume, Avro, Impala and Apachespark.
• Hands on experience on performing D Confidential Quality checks on petabytes of d Confidential
• Solid understanding of Hadoop MRV1 and Hadoop MRV2 (or) YARN Architecture.
• Good knowledge on Amazon AWS concepts like EMR & EC2 web services which provides fast and efficient processing of BigD Confidential .
• Developed, deployed and supported several Map Reduce applications in Java to handle semi and unstructured d Confidential .
• Experience in writing Map Reduce programs and using Apache Hadoop API for analyzing the d Confidential .
• Strong experience in developing, debugging and tuning Map Reduce jobs in Hadoop environment.
• Expertise in developing PIG and HIVE scripts for d Confidential analysis
• Hands on experience in d Confidential mining process, implementing complex business logic and optimizing the query using Hive QL and controlling the d Confidential distribution by partitioning and bucketing techniques to enhance performance.
• Expertise in using Apache Hcat with different big d Confidential processing tools.
• Experience working with Hive d Confidential, extending the Hive library using custom UDF's to query d Confidential in non- standard formats
• Experience in performance tuning of Map Reduce, Pig jobs and Hive queries
• Involved in the Ingestion of d Confidential from various D Confidential bases like TERAD Confidential ( Sales D Confidential Warehouse),AS400, DB2, SQL-SERVER using Sqoop
• Experience working with Flume to handle large volume of streaming d Confidential .
• Good working knowledge on Hadoop hue ecosystems.
• Extensive experience in migrating ETL operations into HDFS systems using PigScripts.
• Good knowledge in evaluating big d Confidential analytics libraries (MLlib) and use of Spark-SQL for d Confidential exploratory.
• Experienced in using Apache ignite for handling streaming d Confidential .
• Experience in implementing a distributed messaging queue to integrate with Cassandra using Apachekafka and zookeeper.
• Expert in creating and designing d Confidential ingest pipelines using technologies such as spring Integration, ApacheStorm-kafka
• ExperiencewithOozieWorkflowEngineinrunningworkflowjobswithactionsthatrunHadoopMapReduce and Pig jobs.
• WorkedwithdifferentFileFormatslikeTEXTFILE,AVROFILE,ORCforHIVEQueryingandProcessing
• Used Compression Techniques (snappy ) with file formats to leverage the storage in HDFS
• Experienced with build tools Maven, ANT and continuous integrations like Jenkins.
• Hands-on experience in using relational d Confidential bases like Oracle, MySQL, PostgreSQL and MS-SQL Server.
• Experience using IDEs tools Eclipse 3.0, My Eclipse, RAD and Net Beans
• Hands on development experience with RDBMS, including writing SQL queries, PLSQL, views, stored procedure, triggers, etc.
• Participated in all Business Intelligence activities related to d Confidential warehouse, ETL and report development methodology
• Expertise in Waterfall and Agile software development model & project planning using Microsoft Project Planner and JIRA.
• Highly motivated, dynamic, self-starter with keen interest in emerging technologies

TECHNICAL SKILLS:

Big D Confidential Technologies: HDFS, Map Reduce, Hive, Pig, Sqoop, Flume, Oozie, Avro, Hadoop Streaming, Zookeeper, Kafka, Impala, Apache Spark, hue, Ambari. Apache ignite.

Hadoop Distributions: Cloudera (CDH4/CDH5),Horton Works

Languages: Java, C, SQL, PYTHON,PL/SQL,PIG-Latin, HQL

IDE Tools: Eclipse, NetBeans, RAD

Framework: Hibernate, Spring, Struts, Junit

Web Technologies: HTML5, CSS3, JavaScript, JQuery, AJAX, Servlets, JSP,JSON, XML, XHTML, JSF, Angular JS

Web Services: SOAP,REST, WSDL, JAXB, and JAXP

Operating Systems: Windows (XP,7,8), UNIX, LINUX, Ubuntu, CentOS

Application Servers: JBoss, Tomcat, Web Logic, Web Sphere

Reporting Tools/ETL Tools: Tableau, Power view for Microsoft Excel, Informatica

D Confidential bases: Oracle, MySQL, DB2, Derby, PostgreSQL, No-SQL D Confidential base (HBase, Cassandra)

PROFESSIONAL EXPERIENCE:

Confidential

Sr. Hadoop Developer

Responsibilities:

• Experienced in migrating the huge volume of d Confidential from EDW to IDW Environment.
• Written Map Reduce code to process and parsing the d Confidential from various sources and storing parsed d Confidential into HBase and Hive using HBase-Hive Integration.
• Experienced in Migrating d Confidential of file sources and Mount sources from RDMS system to Hadoop by using Sqoop.
• Exporting the d Confidential using Sqoop to RDBMS servers and processed that d Confidential for ETL operations.
• Developed Pig Latin scripts to extract the d Confidential from the web server output files to load into HDFS.
• Used some of the Pig and written pig scripts to transform the d Confidential in structured format.
• Implemented Pig scripts and used Skewed, replicated and merge Joins for performance improvements.
• Designing ETL D Confidential Pipeline flow to ingest the d Confidential from RDBMS source to Hadoop using shell script, Sqoop, package and mysql.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the d Confidential onto HDFS.
• Developed workflow in Oozie to automate the tasks of loading the d Confidential into HDFS and pre-processing with Pig.
• Involved in developing Shell scripts to orchestrate execution of all other scripts (Pig, Hive, and MapReduce) and move the d Confidential files within and outside of HDFS.
• Handled importing of d Confidential from various d Confidential sources, performed transformations using Hive, MapReduce, loaded d Confidential into HDFS and Extracted the d Confidential from Oracle into HDFS using Sqoop.
• Worked in transforming d Confidential from HBase to Hive as bulk operations.
• Worked with Autosys scheduler to automate the jobs.
• Implemented POC to migrate map reduce jobs into Spark RDD transformations.
• Used spark for real-time batch processing.
• Active member for developing POC on streaming d Confidential using Apache Kafka and Spark Streaming.

Environment: Hadoop, Map reduce, Hive, Pig, HBase, Cassandra, Flume, Spark, Storm, Rabbit MQ, Active MQ, Sqoop, Accurev, Zookeeper, Oozie, Autosys, shell scripting.

Confidential

Sr. Hadoop Developer

Responsibilities:

• Crawling of D Confidential from 100 + sites based on ontology maintenance.
• Designed schema and modeling of d Confidential and written algorithm to store all validated d Confidential in Cassandra using Spring D Confidential Cassandra Rest.
• To standardize the Input Merchants D Confidential, Uploading images, index the given D Confidential sets into Search and persist the d Confidential on HBase tables.
• Setting up the Spark streaming and Kafka Cluster and developed a Spark Streaming Kafka App.
• Developed prototype Spark applications using Spark-Core, Spark SQL, D Confidential Frame API
• Involved in d Confidential analysis using python and handling the ad-hoc requests as per requirement.
• Developing python scripts for automating tasks.
• Exported the analyzed d Confidential to the relational d Confidential bases using Sqoop for visualization and to generate reports for the BI team.
• Identified trends, opportunities, and risks to current forecasts and the next period's plan.
• Importing and exporting d Confidential into HDFS and Hive using Sqoop.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.
• Used AWS services like EC2 and S3 for small d Confidential sets.
• Generate Stock Alerts, Price Alerts, Popular Product Alerts, New Arrivals for each user based on given likes, favorite, shares count information.

Environment: Cassandra, Hive, Spark (Core, SQL, MLLib, Streaming), Hadoop, MapReduce, Scala, Java, AWS, Zookeeper, Shell Scripting

Confidential

Hadoop Developer

Responsibilities:

• Developed a simulator to send / emit events based on NYC DOT d Confidential file.
• Built Kafka Producer to accept / send events to Kafka Producer which is on Storm Spout.
• Created, altered and deleted topics using Kafka Queues when required with varying
• Setting up and managing Kafka for stream processing and Broker and topic configuration and creation
• Load and transform large sets of unstructured d Confidential from UNIX system to HDFS.
• Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.
• Experienced in running Hadoop streaming jobs to process terabytes of CSV format.
• Supported Map Reduce Programs those are running on the cluster.
• Developing d Confidential pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral d Confidential and financial histories into HDFS for analysis.
• Written Storm topology to accept events from Kafka Producer and Process Events.
• Developed Storm Bolts to Emit d Confidential into HBase, HDFS, Rabbit-MQ Web Stomp.
• Hive Queries to Map Truck Events D Confidential, Weather D Confidential, Traffic D Confidential

Environment: Hadoop, HDFS, Hive, HBase, Kafka, Storm, Rabbit-MQ WebStorm, Google Maps, New York City Truck Routes from NYC DOT.

Confidential

Java/ Hadoop Developer

Responsibilities:

• Involving in sprint planning as part of monthly deliveries.
• Involving in daily scrum calls and standup meetings as part of agile methodology.
• Good hands on experience on Version One tool to update the work details and working hours for a task.
• Involving in the designing part of views.
• Involving in Writing Spring Configuration Files and Business Logic based on Requirement.
• Involved in code-review sessions.
• Implementing Junit test’s based on the business logic w.r.t to assigned backlog in sprint plan.
• Implementing the Fixtures to execute the FitNesse test tables.
• Good experience on creating the Jenkins CI jobs and Sonar jobs.

Environment: Core Java, spring, Maven, XMF Services, JMS, Oracle10g, PostgresSql, 9.2, Fitness, Eclipse, SVN

Confidential

Java Developer

Responsibilities:

• Responsible for all java related activities like analysis, design and development.
• Participated R & D for two months and provided a solution for making this application work for upgraded backend SAP system. Now this solution is highly used in organization for SAP upgrade related projects.
• Further developments in this project using J2EE and related technologies
• Interacting with clients for giving demos and taking requirements, suggestions.

Environment: Java, NWDI, NWDS, Spring IoC