SUMMARY

• Over 7+ years of IT experience with 4+ years of Big Data experience.
• Experience in installing, configuring and troubleshooting Hadoop ecosystem components like Map Reduce, HDFS, Sqoop, Impala, Pig, Flume, Hive, HBase, Zoo Keeper
• Experience on Hadoop CDH3 and CDH4,CDH5
• Experience in upgrading the existing Hadoop cluster to latest releases.
• Experienced in using NFS (network file systems) for Name node metadata backup.
• Experience in using Cloudera Manager 4.0, 5.0 for installation and management of Hadoop cluster.
• Experience in supporting data analysis projects using Elastic Map Reduce on the Amazon Web Services (AWS) cloud.
• Exporting and importing data into S3.
• ConfiguredName node HA on the existingHadoopcluster using Zookeeper quorum
• Expertise in writing Shell scripting in UNIX using ksh and bash
• Experienced in developing and implementing Map Reduce jobs using java to process and perform various analytics on large datasets
• Good experience in writing Pig Latin scripts and Hive queries
• Good understanding of Data Structure and Algorithms
• Good experience on developing of ETL Scripts for Data cleansing and Transformation
• Experience in Data migration from existing data stores and mainframe NDM (Network Data mover) to Hadoop.
• Experience in designing both time driven and data driven automated workflows using Oozie.
• Experience in supporting analysts by administering and configuring HIVE.
• Hands - on programming experience in various technologies like JAVA, JSP, Servlets, SQL, JDBC, HTML, XML, REST, Eclipse, Visual Studio on Windows.
• Experience writing SQL queries and working with Oracle and My SQL.
• Expertise in Object-oriented analysis and programming(OOAD) like UML and use of various design patterns
• Have dealt with end users in requirement gathering, user experience and issues
• Experience in preparing deployment packages and deploying to Dev and QA environments and prepare deployment instructions to Production Deployment Team
• Team player with excellent analytical, communication and project documentation skills
• Agile Methodology and Iterative development

TECHNICAL SKILLS

Programming Languages: Java, C, C++, SQL

Hadoop Ecosystems: HDFS, Hive, Pig, Flume, Impala, Oozie, Zookeeper, HBASE and Sqoop.

Operating System: Linux, Windows XP, Server 2003, Server 2008.

Databases: Oracle, MySQL and SQL Server.

Web Technologies: Applets, JavaScript, CSS, HTML and XHTML

Tools: Ant, Maven, TOAD, AgroUML, WinSCP, Putty

Version Control: VSS, SVN and CVS

PROFESSIONAL EXPERIENCE

Confidential, MD

Hadoop Developer

Responsibilities:

• Gathered the business requirements from the Business Partners and Subject Matter Experts.
• Involved in installing Hadoop Ecosystem components on 50 nodes production.
• Installed and configured the Hadoop name node ha service using Zookeeper.
• Installed and configured Hadoop security and access controls using Kerberos, Active Directory
• Responsible to manage data coming from different sources into hdfs through Sqoop, Flume
• Troubleshooting and monitoring Hadoop services using Cloudera manager.
• Monitoring and tuning Map Reduce Programs running on the cluster.
• Involved in HDFS maintenance and loading of structured and unstructured data.
• Developed several Map Reduce Programs for data preprocessing
• Imported data using Sqoop to load data from MySQL to HDFS on regular basis.
• Wrote Hive queries for data analysis to meet the business requirements.
• Exporting data from RDBMS to HIVE, HDFS and HIVE, HDFS to RDBMS by using SQOOP.
• Prepared System Design document with all functional implementations.
• Involved in Data model sessions to develop models for HIVE tables.
• Understanding the existing Enterprise data warehouse set up and provided design and architecture suggestion converting to Hadoop using MapReduce, HIVE, SQOOP and Pig Latin.
• Converting ETL logic to Hadoop mappings.
• Extensive hands on experience in Hadoop file system commands for file handling operations.
• Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for hive performance enhancement and storage improvement.
• Worked with parsing XML files using Map reduce to extract sales related attributed and store it in HDFS.
• Involved in building TBUILD scripts to import data from Teradata using Teradata Parallel transport APIs.

Environment: CDH, Hadoop, HDFS, MapReduce, Hive, Sqoop, Pig, XML, Cloudera Manager, Teradata

Confidential, MI

Hadoop Devops Consultant

Responsibilities:

• Development and ETL Design in Hadoop
• Developed Mapreduce Input format to read visa specific data format.
• Performance tuning of Hive Queries written by data analysts.
• Developing Hive queries and udf’s as per requirement.
• Migrating existing Ab initio transformation logic to Hadoop Pig Latin and Udf's.
• Used Sqoop to efficiently Transfer data from DB2 to HDFS, Oracle Exadata to HDFS
• Designing ETL flow for several newly on boarding Hadoop Applications.
• Worked on implementing Hadoop Streaming, Python mapreduce for visa analytics
• Implemented NLine Input Format to split a single file into multiple small files
• Designed and Developed oozie workflows, integration with Hcatalog/Pig.
• Documented ETL Best Practices to be implemented with Hadoop
• Monitoring and Debugging Hadoop jobs/Applications running in production.
• Worked on Hadoop CDH upgrade from CDH4.x to CDH5.x.
• Worked on Providing User support and application support on Hadoop Infrastructure.
• Reviewing ETL application use cases before on boarding to Hadoop.
• Worked on Evaluating, comparing different tools for test data management with Hadoop.
• Helping testing team to get up to speed on Hadoop Application testing.
• Worked on Integration of Hiveserver2 with Tableau.
• Worked on impala performance tuning with different workloads and file formats.
• Worked on Installing 20 node UAT Hadoop cluster
• Worked on POC of Talend integration with hadoop

Technologies: Hadoop, HDFS, MapReduce, Hive, Sqoop, Pig, DB2, Oracle, XML, Cloudera Manager

Confidential

Hadoop Developer

Responsibilities:

• Worked on the Hadoop File System Java API to develop or Compute the Disk Usage Statistics.
• Experience in Developing the Hive queries for the transformations, aggregations and Mappings on the Maps Data.
• Developed map reduce programs for applying business rules on the data
• Developed and executed Hive Queries for denormalizing the data.
• Automated workflow using Shell Scripts.
• Performance Tuning on Hive Queries.
• Involved in migration of data from Hadoop Cluster to the Hadoop Cluster.
• Worked on configuring multiple Map Reduce Pipelines, for the new Hadoop Cluster.
• Worked on Configuring the New Hadoop Cluster.

Environment: Java, CDH, Hadoop, HDFS, Map Reduce, Hive and Sqoop

Confidential

Java Developer

Responsibilities:

• Developed the user interface screens using Swing for accepting various system inputs such as contractual terms, monthly data pertaining to production, inventory and transportation.
• Involved in designing Database Connections using JDBC.
• Involved in design and Development of UI using HTML, JavaScript and CSS.
• Involved in creating tables, stored procedures in SQL for data manipulation and retrieval using SQL SERVER 2000, Database Modification using SQL, PL/SQL, Stored procedures, triggers, Views in Oracle.
• Developed the business components (in core Java) used for the calculation module (calculating various entitlement attributes).
• Involved in the logical and physical database design and implemented it by creating suitable tables, views and triggers.
• Created the related procedures and functions used by JDBC calls in the above components.
• Involved in fixing bugs and minor enhancements for the front-end modules.

Environment: Java, HTML, Java Script, CSS, Oracle, JDBC, Swing and Eclipse.