SUMMARY

• Over 7+ years of experience in IT and 3+ years of experience in Hadoop ecosystem's implementation, maintenance, ETL and Big Data analysis operations.
• Excellent understanding / knowledge of BigData and Hadoop ecosystems.
• Experience with configuration of Hadoop ecosystem components such as MapReduce, HDFS, HIVE, HBase, PIG, Sqoop, Scala, Zookeeper, Oozie and Flume.
• Experienced in using NFS (network file systems) for Name node metadata backup.
• In depth knowledge of Hadoop architecture and various components such as HDFS, JobTracker, NameNode, DataNode and MapReduce concepts.
• Good Exposure to Map Reduce programming, PIG Scripting and Distributed Application and NoSQL Databases like HBase, Mongo DB.
• Experience in administrative tasks such as installing, configuring, commission and de - commission nodes, troubleshooting, backups and recovery of Hadoop and its ecosystem components such as Hive, Pig, Sqoop, HBase, Spark.
• Experience in maintaining multiple Hadoop clusters (prod, dev etc.).
• Experience in performance tuning of the Hadoop Cluster by gathering and analyzing the existing infrastructure.
• Worked on both Hadoop distributions: Cloudera and Hortonworks.
• Debugged and solved the major issues with Cloudera manager by interacting with the Cloudera team
• Experience in performing minor and major upgrades and applying patches for Ambari and Cloudera Clusters.
• Experience in understanding Big Data business requirements and providing them Hadoop based solutions.
• Experience importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
• Experience in automating the Hadoop installation, configuration and maintaining the cluster by using the tools like puppet.
• Developed applications using Scala.
• Aggregate data using RDD and RDF data structures.
• Apply breaks and data validation using defined business logic.
• Experience in managing and monitoring Hadoop cluster using Cloudera Manager.
• Knowledge on designing both time driven and data driven automated workflows using Oozie.
• Excellent analytical, multi-tasking, problem-solving, time-management and communication skills with particular emphasis on clearly communicating and documenting detailed Test Requirements & Tests.
• Experience in support analysts by administering and configuring Hive.
• Experience in providing support to data analyst in running Pig and Hive queries.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Developed MapReduce programs to perform analysis.
• Experience in writing shell scripts to dump the Shared Data from MySQL server to HDFS.
• Worked in Classic Map Reduce and YARN architecture for working with data.
• Experience in Core Java, Java Virtual Machine (JVM) and multi-thread processing.
• Over 3 years of extensive experience as a Netcool Management Service (NMS) Consultant, involved in developing architectural solutions for customers in the areas of Fault Management, Performance Management, Operational Support Systems (OSS) and other related disciplines of included IBM Netcool.
• Responsible for monitoring hadoop cluster using ambari by configuring various alerts for troubleshooting, setting parameters for enhancing performance.
• Experience in Web based portal system such using Edge tool (JARVIS) that displays and consolidates web enabled network management applications into a single view.
• Experience in Unix Shell Scripting, SQL, Reporting and validating complex Stored Procedures, Triggers.
• Designed and maintained databases using Python.
• Tested and implemented applications built using Python.
• Installing and configuring MongoDB on Linux machine.
• Backup and Recovery in MongoDB using mongodump and snapshots methods.
• Worked on UNIX platform and got a clear understanding of the VI editor and UNIX commands.
• Excellent communication and inter-personal skills, self-motivated and detail oriented.

TECHNICAL SKILLS

Hadoop Ecosystem Development: HDFS, HBase, MapReduce, YARN, Hive, Pig, Sqoop, Flume, Oozie, Zookeeper, Spark, Scala

Fault Management Tool: IBM Tivoli suite of products ( OMNIbus, ITNM, Web-GUI, TBSM, TCR, Impact, ITM

Languages: C, Java, SQL, Pig Latin, Unix Shell Scripting, Python

IDE/ Tools: Eclipse

Databases: Oracle, MySQL, MS SQL Server 2000, DB2

Operating Systems: Linux, Windows XP, Server 2003, Server 2008

Others: JARVIS, HP-BSM

PROFESSIONAL EXPERIENCE

Confidential, Sunnyvale, CA

Hadoop Developer

Responsibilities:

• Analysing the requirement to setup a cluster.
• Installed and configured Hadoop, MapReduce, HDFS (Hadoop Distributed File System), developed multiple MapReduce jobs in java.
• Worked with the infrastructure and admin team in designing, modelling, sizing and configuring Hadoop cluster of 15 nodes.
• Developed Map Reduce programs in Java for parsing the raw data and populating staging Tables.
• Created Hive queries to compare the raw data with EDW reference tables and performing aggregates.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Experienced in analyzing data with Hive and Pig.
• Developed applications using Scala.
• Writing Pig scripts to process the data.
• Developed PIG Latin scripts to extract the data from the web server output files to load into HDFS.
• Integrating bulk data into Cassandra file system using MapReduce programs.
• Got good experience with NOSQL database.
• Involved in HBASE setup and storing data into HBASE, which will be used for further analysis.
• Experienced in managing and reviewing Hadoop log files.
• Experienced in defining job flows.
• Experienced in managing and reviewing Hadoop log files.
• Installed and configured Hive and also written Hive UDFs.
• Aggregate data using RDD and RDF data structures.
• Apply breaks and data validation using defined business logic.
• Involved in creating Hive tables, loading with data and writing hive queries using the HiveQL which will run internally in map reduce way.
• Extracted the data from MySQL into HDFS using Sqoop.
• Used HiveQL to analyse the partitioned and bucketed data and compute various metrics for reporting.
• Managing the configuration of the cluster to the meet the needs of analysis whether I/O bound or CPU bound
• Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
• Installed and configured multiple CDH Hadoop clusters.
• Deployed Hadoop Cluster in Fully Distributed and Pseudo-distributed modes.
• Experience in managing and monitoring Hadoop cluster using Cloudera Manager.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig, Hive and Sqoop.
• Designed and maintained databases using Python.
• Tested and implemented applications built using Python.
• Worked in Classic Map Reduce and YARN architecture for working with data.
• Configuring and managing MongoDB sharding cluster and MangoDB Master-Slave replication.
• Unit tested a sample of raw data and improved performance and turned over to production.

Environment: Java (JDK1.7), Java 7, Hadoop 2.6.0, MapReduce, HDFS, Hive 0.13.0, Scala, Sqoop 1.4.4, HBase, Pig 0.12.0, Oozie, Kerberos, Linux, Shell Scripting, Oracle 11g, PL/SQL, SQL*PLUS, HDInsight.

Confidential, Camp bell, CA

Hadoop Developer

Responsibilities:

• Installed and configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in Java for data cleaning and pre-processing.
• Installed and Configured Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Scala and Sqoop.
• Good understanding and related experience with Hadoop stack - internals, Hive, Pig and Map/Reduce.
• Wrote MapReduce jobs to discover trends in data usage by users.
• Involved in defining job flows.
• Involved in database connection by using SQOOP.
• Involved in creating Hive tables, loading with data and writing hive queries using the HiveQL.
• Experienced in SQL DB Migration to HDFS
• Got good experience with NOSQL database.
• Used Oozie to orchestrate the workflow.
• Installed and configured Pig for ETL jobs.
• Designed and maintained databases using Python.
• Tested and implemented applications built using SCALA, Python.
• Used big data technologies with hands on experience on Hadoop Distributed File System.
• Worked on Map Reduce programming and Pig latin scripts.
• Experience with Core Distributed computing and Data Mining Library using Apache Spark.
• Integrating bulk data into Cassandra file system using MapReduce programs.
• Involved in creating data-models for customer data using Cassandra Query Language.
• The data is collected from distributed sources into Avro models. Applied transformations and standardizations and loaded into HBase for further data processing.

Environment: Hadoop, Hive, Linux, MapReduce, HDFS, Hive, Scala, Python, Pig, Sqoop, Cloudera, Cassandra, Spark, Shell Scripting, Java (JDK 1.6), Java 6, Oracle 10g, PL/SQL, SQL*PLUS.

Confidential, Houston, TX

Hadoop Developer

Responsibilities:

• Installed and configured fully distributed Hadoop cluster.
• Hadoop cluster environment administration that includes adding and removing cluster nodes, cluster capacity planning, performance tuning, cluster monitoring, and trouble shooting.
• Extensively used Cloudera Manager to manage the Hadoop cluster.
• Used Oozie to automate/schedule business workflows which invoke Sqoop, MapReduce and Pig jobs as per the requirements.
• Developed Sqoop scripts to import and export the data from relational sources and handled incremental loading on the customer and transaction data by date.
• Worked with various HDFS file formats like Avro, SequenceFile and various compression formats like Snappy, bzip2.
• Developed efficient MapReduce programs for filtering out the unstructured data.
• Developed the Pig UDF's to pre-process the data for analysis.
• Developed Hive queries for data sampling and analysis to the analysts.
• Loaded data into the cluster from dynamically generated files using Flume and from relational database management systems using Sqoop.
• Developed custom Unix SHELL scripts to do pre and post validations of master and slave nodes, before and after configuring the name node and data nodes respectively.
• Involved in HDFS maintenance and administering it through Hadoop-Java API.
• Supported Map Reduce Programs those are running on the cluster.
• Developed Java Map Reduce programs using Mahout to apply on different datasets.
• Identified several PL/SQL batch applications in General Ledger processing and conducted performance comparison to demonstrate the benefits of migrating to Hadoop.
• Configured Sentry to secure access to purchase information stored in Hadoop.
• Involved in several POCs for different LOBs to benchmark the performance of data-mining using Hadoop.

Environment: RedHat Linux 5, MS SQL Server, Mongo DB, Oracle, Scala, Hadoop CDH 3/4/5, PIG, Hive, ZooKeeper, Mahout, HDFS, HBase, Sqoop, Python, Java, Oozie, Hue, Tez, UNIX Shell Scripting, PL/SQL, Maven, Ant.

Confidential

Java Developer

Responsibilities:

• Responsible for Design and Development of business and presentation layers.
• Developed the application using the MVC design pattern and encapsulated front end from the underlying business logic, and data model.
• Responsible for generation of XSLT.
• Worked on Struts Framework at later stages of the project to support ongoing development by the developers.
• Created Dynamic web pages using HTML, and XSLT.
• Developed front end templates and hand-code pages using HTML, JavaScript, and CSS.
• Developed EJB’s which invokes the Unit-Tel web services using apache axis SOA.
• Implemented different EJBs like State full session and stateless session beans.
• Developed the JSPs as web based user interface validated all client Connect screens using Java Scripts.
• Used the J2EE components like, JSPs including tag libraries, Servlets, EJBs.
• Conducted Code review with team members.
• Implemented STRUTS Framework including, creating templates for Action Form classes.
• Implemented internationalization concepts using Struts Framework.
• Wrote Session Beans to handle Business Logic.
• Developed XML file for each server. The XML files have information of all the components reporting to that server. XSL was used to transform XML document. GUI will use this data to draw Tables and Menus with specified attributes.
• Created unit test cases, tested application using J Unit testing tool.

Environment: Core Java, J2EE, CSS, Servlets, JSP, Struts, Tiles, XML, Tomcat, and MySQL, EJB.

Confidential

Java Developer

Responsibilities:

• Responsible for Business Analyst activities for critical functionality for the business interfacing projects.
• Responsible for successful deployment of all Major and Minor Releases.
• Developed key modules of the Application using Frameworks / Languages / Tools as JMS, Spring 2.5, Oracle 9i. And Hibernate 3.0.
• As application developer I was actively involved in designing of various Business Layer and Data Management components of this web based system over J2EE architecture using JSTL, JSP, Servlets, and JavaScript.
• Implemented MVC architecture using Spring Framework, coding involved writing Action Classes/Custom Tag Libraries, JSP.
• Have been involved in the design and key component(Matching Engine) of the system using PL/SQL procedure on Oracle Database.
• Built packages and procedures for designing business rules for the applications on the database side.
• Provided request and reports and Supporting Client Services for all customer requests being an SME of the application Experience developing for Unix/Linux based systems Development of Tools and Value adds to assist Performance Testing.
• Used AGILE methodology.
• Was involved in creating front-end pages using HTML 4.01, CSS and JavaScript.
• Used log4j to log exception and modified the xml to roll up, zip and archive logs.

Environment: Java 1.5, JSP, JSTL, JMS, Spring 2.5, Servlets, JavaScript, Oracle 9i, HTML 4.01, CSS, Log4j, Hibernate 3.0, PL/SQL, AGIL.