SUMMARY

• Around 7 years of experience on gathering System Requirements, Analyzing the requirements, Designing and developing systems including 3+ years in Big Data.
• Highly dedicated and result oriented Hadoop Developer with 3+ years of strong end - to-end experience on Hadoop Development with varying level of expertise around different BIGDATA HADOOP projects.
• Experience in different hadoop distributions like Cloudera (CDH3 & CDH4) and Horton Works Distributions (HDP).
• Excellent Knowledge in understanding Big Data infrastructure, distributed file systems - HDFS, parallel processing - Map Reduce/YARN framework and complete Hadoop ecosystem - Hive, Pig, Scoop, Hbase, NoSQL, Oozie, Kafka and Flume.
• In depth understanding of Hadoop components such as Resource Manager, Node Manager, Application Master, Application Manager, Data Node and Zookeeper.
• Experience in analyzing data using HiveQL, Pig Latin and custom Map Reduce programs in Java.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Experience in using Ganglia, Nagios and Cloudera Manager for monitoring workload, job performance and collecting metrics of Hadoop cluster.
• Expertise with optimizing query performance and data load times in PIG, Hive and Map Reduce applications.
• Expert in designing both time driven and data driven automated workflows using Oozie.
• Exposure in using Cloud era Manager and Apache Ambari for monitoring jobs running on cluster.
• Experience in using different file formats like Sequence, AVRO, ORC files and CSV and using different compression Techniques.
• Hands on experience in NoSQL databases like HBase, Cassandra, Mongo DB.
• Exposure on customization of embedded Apache Solr, used it as search engine for our claims data.
• Experience in supporting data analysis projects using Elastic Map Reduce on the Amazon Web Services (AWS) cloud.
• Experienced Hadoop/Java developer having end to end experience in developing applications in Hadoop ecosystem
• Experience in using Flume/Kafka to load the log data from multiple sources directly into HDFS.
• Experienced with improving data cleansing process using Hive, Pig Latin operations, transformations and join operations files and CSV and using different compression Techniques.
• Implemented advanced procedures like text analytics and processing using the in-memory computing capabilities like Apache Spark written in Scala/Java.
• Handling and further processing schema oriented and non-schema oriented data using Pig.
• Experience on Core java technology, which includes multithreading, JDBC, Collections, network programming.
• Hands on experience in application development using Java, RDBMS, and Linux/Unix shell scripting.
• Experience in understanding the client’s Big Data business requirements and transform it into Hadoop centric technologies.
• Excellent analytical, problem solving, communication and interpersonal skills with ability to interact with individuals at all levels and can work as a part of a team as well as independently.
• Implemented Unit Testing using JUNIT testing during the projects.
• Ability to perform at a high level, meet deadlines, adaptable to ever changing priorities.

TECHNICAL SKILLS

Hadoop/Big Data: HDFS, Map Reduce, Hbase, Pig, Hive, Sqoop, Flume, OozieZookeeper, YARN

Technologies: Servlets, JSP, JDBC, XML, Java, SQL, PL/SQL, Cassandra, Mongo DB.

Programming Languages: C, C++, Java, Visual Basic, SQL, PL/SQL, Shell Scripting

Scripting Languages: HTML, CSS, JavaScript, Python

J2EE Technologies: Servlets, JSP, Custom Tag Libraries, JDBC, JNDI, Collections, Multi-Threading, JMS, Java Mail, JAXB, JAXP.EJB

Web Technologies: HTML, DHTML, JavaScript, Ajax, CSS, XML, DTD, XSD, XSLT, SAXDOM, JSON

Databases: Oracle 8.x/9.x, MySQL, SQL Server, Cassandra, Mongo DB

Application/Web Servers: Apache Tomcat, JBoss, Web Sphere, Web Logic

Web Services: Apache CXF/XFire, Apache Axis, SOAP, Jersey

IDE: Eclipse, Net Beans, JDeveloper, WSAD

Frameworks: Struts, spring, Hibernate

Testing/Logging tools: JUnit, Easy Mock, JMock, log4J

PROFESSIONAL EXPERIENCE

Confidential, NY

Hadoop Developer

Responsibilities:

• Worked closely with the business analysts to convert the Business Requirements into Technical
• Hands on using log files and to copy them into HDFS using flume
• Hands on writing Map Reduce code to make unstructured data as structured data and for ingesting data into HDFS/Hive.
• Experience in creating integration between Hive and HBase.
• Implemented business logic by writing Pig and Hive UDFs for some aggregative operations and to get the results from them.
• Design & Developed ETL workflow using Oozie for business requirements which includes automating the extraction of data from Oracle database into HDFS using Sqoop scripts.
• Created Hbase tables and loaded bulk data into Hbase using Hive/Pig scripts.
• Created Solr Collections and loaded data using Pig Scripts
• Performed data analytics in Hive and then exported this metrics back to Oracle Database using Sqoop.
• Developed customized UDF’s in java and used in Hive to meet business requirements
• Performed extensive hive tuning/optimization on hive tables like Partitioning, Bucketing, ORC file format, Cost Based Optimizations
• Performed different types of joins like Map only join, reduce side join, SMB Joins on Hive tables
• Implemented Custom JOINS to create tables containing the records of Items using Spark SQL.
• Conducting root cause analysis and resolve production problems and data issues.
• Proactively involved in ongoing maintenance, support and improvements in Hadoop cluster.
• Involved in Minor and Major Release work activities.
• Collaborating with business users/product owners/developers to contribute to the analysis of functional requirements.
• Hands on experience on Hadoop technology and framework, coding, testing and implementation.

Environment: JDK,JRE, Red Hat Linux, HDFS, Map-Reduce, Hive, Pig, Sqoop, Oozie, Oracle, Cassandra, HBase.

Confidential, Livingston, NJ

Hadoop Developer

Responsibilities:

• Extracted the data from Teradata/MySQL into HDFS using Sqoop export/import.
• Optimized Map Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Expertise in using Data organizational design patterns in Map Reduce to convert business data into custom format.
• Implemented Custom JOINS to create tables containing the records of Items using Spark SQL.
• Expertise in optimization of MapReduce algorithms using Combiners, Partitioners and Distributed Cache to deliver best results.
• Experienced with handling different compression codec's like LZO, GZIP, and Snappy.
• Experienced with handling data from different sources at a time to reducer using Object Writable in Mapreduce programs.
• Analyzed the data by performing Hive queries and running Pig scripts.
• Created and worked Sqoop jobs with incremental load to populate Hive External tables.
• Developed Hive scripts for end user / analyst requirements to perform ad hoc analysis.
• Expert in optimizing performance in hive using partitions and bucketing concepts.
• Experienced with handling different optimization techniques in Hive joins.
• Solved performance issues in Hive and Pig scripts with understanding of Joins, Group and aggregation and how does it translate to MapReduce jobs.
• Developed UDFs in Java as and when necessary to use in HIVE queries
• Developed Oozie workflow for scheduling and orchestrating the ETL process.
• Implemented authentication using Kerberos authentication using Apache Sentry.
• Deployed an Apache Solr search engine server to help speed up the search of the government cultural asset.
• Knowledge on visualization tools like Tableau.
• Experienced with writing build jobs using Maven and integrate that with Jenkins.
• Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Environment: Horton works, Hadoop, Map Reduce, HDFS, Hive, Sqoop, Avro, Oozie, Java (jdk6), Informatica and Oracle 4.0.8.

Confidential, Schaumburg, IL

Hadoop and Java Developer

Responsibilities:

• Installed and configured Hadoop Map Reduce, HDFS and developed multiple Map Reduce jobs in Java for data cleansing and preprocessing
• Experience in developing Shell Scripts for system management and for automating routine tasks.
• Worked on Hadoop Map Reduce tasks in Java to convert JSON format logs to text formats like CSV and TSV (Comma separated values, Tab separated values )
• Developed Map reduce programs in Java for parsing the raw data and populating staging tables
• Experienced in writing test classes for Mapreduce programs using MRUnit and Mockito.
• Experienced in loading data from UNIX local file system to HDFS.
• Involved in creating Hive tables, loading with data and writing Hive queries
• Worked on NoSQL database like HBase for storing unstructured image and signature data.
• Analyzed the data in HBase to get real time analytics using Java API.
• Imported bulk amount of data into HBase using Mapreduce Integration.
• Implemented performance tuning by using Map Joins, resizing the Mappers/reducers etc.
• Implemented Hive/Pig UDFs for common operations
• Experienced with handling Automatic failover in HBase using Zoo Keeper and HMaster.
• Performing real time analytics in Time Series data using HBase and Hadoop eco system.
• Experienced with performing ETL operations using Pig Latin operations and scripts.
• Parsed JSON and XML files in PIG using Pig Loader functions and extracted meaningful information from Pig Relations by providing a regex using the built-in functions in Pig.
• Experienced with processing Avro data files using Avro tools and Mapreduce programs.
• Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile and network devices and pushed to HDFS.
• Integrated spring schedulers with Oozie client to schedule nightly cron jobs.
• Being a part of a POC effort to help build new Hadoop clusters.
• Gained good Knowledge in Amazon AWS concepts like EMR and EC2 web services which provides fast and efficient processing of Big Data
• Worked hands on NoSQL databases like Cassandra, for POC purpose in storing images and URIs.
• Experienced in monitoring, configuring cluster using Ganglia.
• Experienced with writing Low level and high level design documents according to business requirements using Visio.

Environment: Hadoop, Map Reduce, HDFS, Hive, Apache Spark, Kafka, Couch DB, Flume, AWS, Cassandra, Java,, Servlets, HTML, XML, SQL, MRUnit, JDBC, Eclipse.

Confidential, Warren, NJ

JAVA Developer

Responsibilities:

• Developed the application using Struts Framework that leverages classical Model View Layer (MVC) architecture UML diagrams like use cases, class diagrams, interaction diagrams (sequence and collaboration) and activity diagrams were used
• Worked in an Agile work environment with Content Management system for workflow management and content versioning
• Involved in designing user screens and validations using HTML, jQuery, Ext JS and JSP as per user requirements
• Familiar with SQL Server Integration SSIS and SSRS
• Responsible for validation of Client interface JSP pages using Struts form validations
• Integrating Struts with Spring IOC
• Used Spring Dependency Injection properties to provide loose-coupling between layers
• Implemented the Web Service client for the login authentication, credit reports and applicant information using Apache Axis 2 Web Service
• UsedHibernateORM framework withSpringframework for data persistence and transaction management
• Used Hibernate 3.0 object relational data mapping framework to persist and retrieve the data from database
• Created physical and logical data model, design and implementation
• Wrote SQL queries, stored procedures, and triggers to perform back-end database operations
• Developed ANT Scripts to do compilation, packaging and deployment in the Web Sphere server
• Implemented the logging mechanism using Log4j framework
• Designing and implementing algorithms
• Wrote test cases in JUnit for unit testing of classes

Environment: JDK 1.6, J2EE 1.4, Agile Development Process, Struts 1.3, Spring 2.0, Web Services, Hibernate 3.0, RSA, JMS, JSP, Servlets 2.5, Web Sphere 6.1, SQL Server 2005, DB2, Windows XP, HTML, XML, IBM Rational Application Developer (RAD), ANT 1.6, Log4J, XML, XSLT, XSD, jQuery, JavaScript, Ext JS, JSON, JUnit 3.8.

Confidential

Junior JAVA Developer

Responsibilities:

• Involved in Design, Development and Support phases of Software Development Life Cycle (SDLC)
• Reviewed the functional, design, source code and test specifications
• Involved in developing the complete front end development using Java Script and CSS
• Author for Functional, Design and Test Specifications
• Implemented Backend, Configuration DAO, XML generation modules of DIS
• Analyzed, designed and developed the component
• Used JDBC for database access
• Used Data Transfer Object (DTO) design patterns
• Unit testing and rigorous integration testing of the whole application
• Written and executed the Test Scripts using JUNIT
• Developed XML parsing tool for regression testing
• Prepared the Installation, Customer guide and Configuration document which were delivered to the customer along with the product

Environment: Java, JavaScript, HTML, CSS, JDK 1.5.1, JDBC, Oracle10g, XML, XSL, Solaris and UML