SUMMARY

• 10+ years of IT experience in software design, analysis, and development on Core Java, Apache, MySQL, Confidential BPM technologies and Apache Hadoop ecosystem.
• 3 years hands on development experience on Apache Hadoop ecosystem
• Managed HDFS File system
• MapReduce programming using Java
• Developed partition tables and HiveQL for Hive
• Developed UDFs for PIG and HIVE
• Worked on Flume, Sqoop, Oozie
• Good knowledge on Apache Spark, HBase and Mongo DB nosql databases
• Good knowledge on Cloudera and Hortonworks distributions
• Strong hands on experience in writing Core Java enterprise class applications
• Strong hands on experience in designing RDBMS schemas for enterprise class applications and implementing MySQL Circular Master for HA
• Comfortable experience in Python programming
• Hands on experience in SOA development using Confidential Process suite
• Hands on experience in Apache HTTPD, Proxy, Custom Modules
• Good Exposure to XML related technologies such as XSL, XSLT, XPath, DOM, and SAX and Confidential NOM.
• Experience on SOAP and REST for developing Web Services

TECHNICAL SKILLS

Big Data Technologies: Hadoop, Yarn, HDFS, Map Reduce, Sqoop, Hive, Pig, Flume, Zookeeper, Oozie, HBase, Spark

Languages: Core Java, C, C++, Python

SOA and Integration Technologies: Confidential BPM Suite

OS: Windows, Linux, Redhat, CentOs

RDBMS: MySQL 5.x, Oracle and MS SQL Server

Database Scripting: SQL and PL/SQL

Application & Web Servers: Apache, IIS, Tomcat

PROFESSIONAL EXPERIENCE

Confidential

Hadoop Senior Developer

Environment: Hadoop, MapReduce, HDFS, Yarn, Sqoop, Oozie, Pig, Hive, Hbase, Spark, Java, Eclipse, UNIX shell scripting, python

Responsibilities:

• Worked on analyzing, writing Hadoop MapReduce jobs using JavaAPI, Pig and Hive.
• Responsible for building scalable distributed data solutions using Hadoop.
• Involved in loading data from edge node to HDFS using shell scripting.
• Created HBase tables to store variable data formats of PII data coming from different portfolios.
• Exported the analysed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Worked with using different kind of compression techniques to save data and optimize data transfer over network using LZO, Snappy, Bzip etc.
• Analyze large and critical datasets using Cloudera, HDFS, Hbase, MapReduce, Hive, Hive UDF, Pig, Sqoop, Zookeeper, & Spark.
• Developed custom aggregate functions using Spark SQL and performed interactive querying.
• Used Scoop to store the data into HBase and Hive.
• Worked on installing cluster, commissioning & decommissioning of DataNode, NameNode high availability, capacity planning, and slots configuration.
• Creating Hive tables, dynamic partitions, buckets for sampling, and working on them using HiveQL.
• Used Pig to parse the data and Store in Avro format.
• Stored the data in tabular formats using Hive tables and Hive SerDes.
• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Worked with NoSQL databases like Hbase for creating Hbase tables to load large sets of semi structured data coming from various sources.
• Implemented a script to transmit information from Oracle to Hbase using Sqoop.
• Implemented MapReduce programs to handle semi/unstructured data like XML, JSON, and sequence files for log files.
• Fine - tuned Pig queries for better performance.
• Involved in writing the shell scripts for exporting log files to Hadoop cluster through automated process.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.

Confidential, Irving, Tx

Hadoop Senior Developer

Environment: Hadoop, MapReduce, HDFS, Yarn, Sqoop, Oozie, Pig, Hive, Java, Eclipse, UNIX shell scripting, python

Responsibilities:

• Developed MapReduce jobs in java to process the input data and generate output files in HDFS.
• Worked with Business Analysts to gather all the requirements and documented all the use cases.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Developed HiveQL for Hive
• Designed workflow jobs in Oozie to combine a series of implementations.
• Implemented coordinator workflows in Oozie to schedule various jobs.
• Experienced in managing and reviewing Hadooplog files.
• Configured Sqoop to map SQL types to appropriate Java classes.
• Responsible to manage data coming from different sources.
• Implemented Pig Scripts for basic data transformation tasks.
• Developed UDFs in Pig and hive.
• Worked with support and BI teams to generate reports from the output files generated from MapReduce jobs.

Confidential,

Architect, Lead

Environment: Hadoop, HDFS, Sqoop, Zookeeper, Shell Scripting, Python, Java, MySQL, Ubuntu, Red Hat Linux.Customers: EO and Solcon ISPs, The Netherlands.

Responsibilities:

• Actively participated in the entire project life cycle
• Taken care of the entire design and the implementation part
• Importing and Exporting data in HDFS
• Developed MapReduce jobs in java to process the input data
• ShellScripts and Python scripts are written to execute Hadoop supporting tasks
• Implemented Custom Input format classes to read the zip files
• Designed RDBMS to handle customer requirements
• Reports are automated as per Business use case
• Integrated Fusion Charts in web display.
• Mentoring the other resources
• Prepared the Demos for acceptance testing

Confidential

Architect, Lead

Environment: RHEL 3.0, Apache HTTPD, MySQL, C, C++, Java, web-services, SOAP and Confidential ESB (Enterprise Service Bus).Customers: EO and Solcon ISPs, The Netherlands. Cyntellix, US, SiteBlocker, MobiGreen

Responsibilities:

• Implemented MySQL Circular Master
• Actively participated in the entire project life cycle such as analysis, design, planning and realization
• No Single point of failure by design
• Designed RDBMS system to meet the complex business use cases.
• Developed Plug and Play modules on Apache HTTPD
• Application Connectors are written over Confidential XQY layer
• Single Unified Logging mechanism implemented across the Parental Control Modules
• Mentored and handled other engineers to have them in the project realization
• Suggested and Implemented architectural changes in the deployment design of the Server Cluster.
• Providing support in case of urgent issues through WebEx and Teleconference.

Confidential

Architect, Lead Developer

Environment: RHEL 3.0, Apache HTTPD, MySQL, C, C++, Java, web-services, SOAP and Confidential ESB (Enterprise Service Bus).Customers: Confidential India and Confidential Netherlands.

Responsibilities:

• Explored the internals of Domain Controller to get the LogOn information
• Measures are taken to avoid Single Point of Failure
• Actively participated in the entire project life cycle such as analysis, design, planning and realization
• Handled entire realization, testing, bug fixing configuration management and documented the user and configuration guides for the project
• Completely designed and developed the Custom Application Connector for this.
• Application Connector is written over Confidential XQY layer for better speed.
• Developed all the Apache related modules for integration
• Involved in maintenance and support for this project on various enhancement requirement
• Mentored and handled other engineers to have them in the project realization
• Architectural changes in the deployment design of the server cluster.
• Maintenance of the servers and backtracking deployment issues.