SUMMARY

• 8+ years of overall experience with strong emphasis on Design, Development, Implementation, Testing and Deployment of Software Applications.
• Over 3+ years of comprehensive IT experience in Big Data and Big Data Analytics, Hadoop, HDFS, Map Reduce, YARN, Hadoop Ecosystem and Shell Scripting.
• 5+ years of development experience using Java, J2EE, JSP and Servlets.
• Highly capable of processing large sets of Structured, Semi - structured and Unstructured datasets and supporting Big Data applications.
• Hands on experience with Hadoop Ecosystem components like Map Reduce (Processing), HDFS (Storage), YARN, Sqoop, Pig, Hive, HBase, Oozie and Zookeeper.
• Expertise in transferring data between a Hadoop ecosystem and structured data storage in a RDBMS such as MY SQL, Oracle, Teradata and DB2 using Sqoop.
• Expertise in moving large amounts of log, streaming event data and Transactional data using Flume.
• Experience in developing Map Reduce jobs in Java for data cleaning and preprocessing.
• Expertise in writing Pig Latin, Hive Scripts and extended their functionality using User Defined Functions (UDF's).
• Expertise in handling structured arrangement of data within certain limits (Data Layout's) using Partitions and Bucketing in Hive.
• Expertise in preparing interactive Data Visualization's using Tableau Software from different sources.
• Knowledge on NoSQL databases like HBase and Cassandra.
• Hands on experience developing workflows that execute Map Reduce, Sqoop, Pig, Hive and Shell scripts using Oozie.
• Experience working with Cloudera Hue Interface and Impala.
• Expertise in Object-oriented analysis and design (OOAD) like UML and use of various design patterns.
• Experience in Java, JSP, Servlets, EJB, Web Logic, Web Sphere, Hibernate, Spring JBoss, JDBC, RMI, Java Script, Ajax, JQuery, XML and HTML.
• Fluent with the core Java concepts like I/O, Multi-threading, Exceptions, RegEx, Collections, Data-structures and Serialization.
• Performed unit testing using JUnit Testing Framework and Log4J to monitor the error logs.
• Expertise in working with transactional databases like Oracle, SQL server, My SQL, and Db2.
• Expertise in developing SQL queries, Stored Procedures.
• Has very good development experience with Agile Methodology.
• Ability to adapt to evolving technology, strong sense of responsibility and accomplishment.
• Excellent leadership, interpersonal, problem solving and time management skills.
• Excellent communication skills both written (documentation) and verbal(presentation).
• Very responsible and good team player. Can work independently with minimal supervision.

TECHNICAL SKILLS

Languages: SQL, C, C++, Java, J2EE, Pig Latin, Hive.

Big Data Technologies: Hadoop, HDFS, Map Reduce, Hive, Pig, HBase, Impala, Hue, Sqoop, Oozie, Flume, Zookeeper, Cassandra, Cloudera CDH5, Python and Horton works.

Databases: Oracle, MySQL, SQL Server, DB2, Teradata, HBase, Cassandra.

Scripting and Query Languages: UNIX Shell scripting, SQL and PL/SQL.

Web Technologies: JSP, Servlets, JavaBeans, JDBC, AWT, Swing, JSF, XML, CSS, HTML, XHTML, JavaScript, AJAX.

Operating Systems: Windows 8, Windows 7, UNIX, Linux, CentOS, Ubuntu.

Tools: Eclipse, Tableau, GIT, SVN, Concurrent versions system (CVS).

PROFESSIONAL EXPERIENCE

Confidential, Dallas, TX

Sr. Hadoop Developer

Responsibilities:

• Responsible for gathering requirements from the business partners.
• Developed a process for Sqooping data from multiple sources like SQL Server, Oracle and Teradata.
• Developed Oozie workflow’s for executing Sqoop and Hive actions
• Responsible for developing Python wrapper scripts which will extract specific date range using Sqoop by passing custom properties required for the workflow.
• Developed scripts to run Oozie workflows, capture the logs of all jobs that run on cluster and create a metadata table which specifies the execution times of each job.
• Responsible for creation of mapping document from source fields to destination fields mapping.
• Developed a shell script to create staging, landing tables with the same schema like the source and generate the properties which are used by Oozie jobs.
• Developed Hive scripts for performing transformation logic and also loading the data from staging zone to final landing zone.
• Worked on Parquet File format to get a better storage and performance for publish tables.
• Involved in loading transactional data into HDFS using Flume for Fraud Analytics.
• Developed Python utility to validate HDFS tables with source tables.
• Responsible to check-in the developed code into Harvest for release management.
• Involved in using CA7 tool to setup dependencies at each level (Table Data, File and Time).
• Involved in developing Spark code using Scala and Spark-SQL for faster testing and processing of data and exploring of optimizing it using Spark Context, Spark-SQL, Pair RDD's, Spark YARN.

Environment: Hadoop, HDFS, Hive, HBase, Zookeeper, Oozie, Impala, Java(jdk1.6), Cloudera, Oracle, Teradata, SQL Server, UNIX Shell Scripting, Flume, Scala, Spark, Sqoop, Python.

Confidential, Pittsburg, PA

Hadoop Developer

Responsibilities:

• Responsible for Managing, Analyzing and Transforming petabyte’s of data and also quick validation check on FTP file arrival from S3 Bucket to HDFS.
• Developed and extended the functionality of the existing Map Reduce programs to perform Time zone Lookup on the source data.
• Experienced in creation of Hive tables and loading data incrementally into the tables using Dynamic Partitioning and Worked on Avro Files, JSON Records.
• Experienced in writing Hive Validation Scripts which are used in validation framework (for daily analysis through graphs and presented to business users).
• Responsible for creating Dashboards on Tableau Server.
• Involved in development and usage of UDTF’s and UDAF’s for decoding Log Record Fields and Conversion’s, Generating Minute Buckets for the specified Time Interval’s and JSON Field Extractor.
• Responsible for Debug, Optimization of Hive Scripts and also implementing Deduplication Logic in Hive using a Rank Key Function (UDF).
• Responsible for Scheduling using Active Batch jobs and Cron jobs.
• Involved in Jar builds that can be triggered by commits to Github using Jenkins.
• Exploring new tools for data tagging like Tealium (POC Report).

Environment: Hadoop, Map Reduce, HDFS, Pig, Hive, HBase, Zookeeper, Oozie, Impala, Java(jdk1.6), Cloudera, Oracle 11g/10g, Windows NT,UNIX Shell Scripting,Tableau, Tealium.

Confidential, Oaks, PA

Hadoop Developer

Responsibilities:

• Responsible for understanding the scope of the project and requirements gathering.
• Responsible for analyzing large data sets and derive customer usage patterns by developing new Map Reduce programs.
• Used Map Reduce to Index the large amount of data to easily access specific records.
• Load log data into HDFS using Flume. Worked extensively in creating Map Reduce jobs to power data for search and aggregation.
• Worked extensively with Sqoop for importing data from Oracle.
• Utilized Apache Hadoop ecosystem tools like HDFS, Hive and Pig for large datasets analysis.
• Developed Pig and Hive UDF's to analyze the complex data to find specific user behavior.
• Designed a data warehouse using Hive and created partitioned tables in Hive.
• Experienced in using Pig for data cleansing and developed Pig Latin scripts to extract the data from web server output files to load into HDFS.
• Developed Map Reduce ETL in Java/Pig and data validation using HIVE.
• Worked on Hive by creating external and internal tables, loading it with data and writing Hive queries.
• Created HBase tables to store data from different sources.
• Developed workflow in Oozie to automate the tasks of loading data into HDFS and pre-processing with Pig and Hive.
• Generated reports for hive tables in different scenarios using Tableau.
• Automated all the jobs for pulling data from FTP server to load data into Hive tables using Oozie workflows.
• Worked with various Hadoop file formats, including Text, Sequence File and RC File.
• Configured Zookeeper for Cluster co-ordination services.

Environment: Hadoop, Map Reduce, HDFS, Pig, Hive, HBase, Zookeeper, Oozie, Impala, Java(jdk1.6),Cloudera, Oracle 11g/10g, Windows NT,UNIX Shell Scripting, Tableau.

Confidential, MA

Hadoop Developer

Responsibilities:

• Involved in collecting business requirements from the Business partners and subject Matter Experts.
• Responsible to manage data coming from different data sources.
• Developed Map Reduce programs to perform data filtering for unstructured data.
• Designed and developed UDF’S to extend the functionality in both PIG and HIVE.
• Wrote Hive queries for data analysis to meet the business requirements.
• Created Partitioned Hive tables and worked on them using Hive.
• Import and Export of data using Sqoop between MySQL to HDFS on regular basis.
• Used Flume to channel data from different sources to HDFS.
• Created HBase tables to store data depending on column families.
• Worked with administrator to set up and monitor the Hadoop cluster
• Supported Map Reduce Programs which are running on the cluster.
• Involved in HDFS maintenance and loading of structured and unstructured data.
• Developed scripts and batch jobs to schedule various Hadoop programs.
• Designed and Maintained Oozie workflows to manage the flow of jobs in the cluster.
• Actively updated the upper management with daily updates on the progress of project that include the classification levels that were achieved on the data.

Environment: Java, Hadoop, Map Reduce, HDFS, Pig, Hive, HBase Linux, MySQL, Ubuntu.

Confidential, MO

Sr. Java Developer

Responsibilities:

• Designed the application by implementing Struts Framework based on MVC Architecture.
• Designed and developed the front end using JSP, HTML and JavaScript and JQuery.
• Developed framework for data processing using Design patterns, Java, XML.
• Developed custom Tags in Struts.
• Used the light weight container of the Spring Framework to provide architectural flexibility for Inversion of Controller (IOC).
• Used Spring IOC for dependency injection to Hibernate and Spring Frameworks.
• Designed and developed Session beans to implement the Business logic.
• Developed EJB components that are deployed on Web logic Application Server.
• Written unit tests using Junit Framework and Logging is done using Log4J Framework.
• Designed and developed various configuration files for Hibernate mappings.
• Designed and Developed SQL queries and Stored Procedures.
• Developed Web Services for sending and getting data from different applications using SOAP messages.
• Actively involved in code reviews and bug fixing.
• Applied CSS (Cascading style Sheets) for entire site for standardization of the site.
• Offshore co-ordination and User acceptance testing support.

Environment: Java 5.0, Struts, Spring 2.0, Hibernate 3.2, Web Logic 7.0, Eclipse 3.3, Oracle 10g, Junit 4.2,Maven, Windows XP, HTML, CSS, JavaScript, and XML.

Confidential

Java Developer

Responsibilities:

• Involved in the analysis & design of the application using Rational Rose.
• Object Oriented Analysis and Design using UML include development of class diagrams, Sequence diagrams and State diagrams and implemented these diagrams in Microsoft Visio.
• Developed the various action classes to handle the requests and responses.
• Designed and created Java Objects, JSP pages, JSF, JavaBeans and Servlets to achieve various business functionalities. Created validation methods using JavaScript and Backing Beans.
• Involved in writing client side validations using JavaScript, CSS.
• Involved in the design of the Referential Data Service module to interface with various databases using JDBC.
• Used Hibernate framework to persist the employee work hours to the database.
• Spring framework features were extensively used.
• Developed and configured using BEA Web Logic Application Server.
• Developed the build scripts using Ant.
• Involved in designing test plans, test cases and overall Unit testing of the system.
• Developed controllers and actions encapsulating the business logic.
• Developed classes and interface with underlying web services layer.
• Designed web services for the above modules.
• Prepared documentation and participated in preparing user's manual for the application.

Environment: Java, Rational Rose, Spring 2.0, Hibernate 3.2, Web Logic, Eclipse, SQL Server 2008, Junit 4.2,Ant, Windows XP, HTML, CSS, JavaScript, and XML.

Confidential

Jr. Software engineer

Responsibilities:

• Involved in gathering user requirements and followed by analysis and design.
• Involved in Developing JSP's, action classes, form beans, response beans, EJB's.
• Developed Struts configuration files and action classes the Transactional Model to handle many requests.
• Participated in developing helper classes which interact with middleware stubs..
• Extensively used XML to code configuration files.
• Developed PL/SQL stored procedures, triggers.
• Performed functional, integration, system and validation testing.

Environment: Java, JSP, Struts, SQL, PL/DSQL, Eclipse, Oracle, Windows XP, HTML, CSS, JavaScript, and XML.