Sr. Hadoop Developer Resume
Memphis, TN
SUMMARY:
- Over 7+ years of progressive experience in the IT industry with proven expertise in architecting and implementing Software Solutions using Big Data Technologies and Java based technologies.
- Over 3 years of experience on Hadoop working environment includes Map Reduce, HDFS, HBase,
- Zookeeper, Oozie, Hive, Sqoop, Pig, Cassandra and Flume.
- Domain experience in Insurance, Retail, Health Care industries
- Well versed in developing complex Map Reduce programs using Apache Hadoop for analyzing Big Data.
- Expertise in Database Design, Creation and Management of Schemas, writing Stored Procedures, Functions, DDL, DML SQL queries
- Good knowledge of No - SQL databases Cassandra and HBase
- Implemented batch processing solution to certain unstructured and large volume of data by using Hadoop
- Map Reduce framework.
- Hands on experience in writing Ad-hoc Queries for moving data from HDFS to HIVE and analyzing the data using HIVE QL.
- Working experience in importing and exporting data using Sqoop from Relational Database Systems
- (RDBMS) to HDFS
- Extending HIVE and PIG core functionality by using custom User Defined Function s (UDF), User Defined Table-Generating Functions (UDTF) and User Defined Aggregating Functions (UDAF) for Hive and Pig.
- Developed Pig Latin scripts for data cleansing and Transformation.
- Working Knowledge in configuring and monitoring tools like Ganglia and Nagios.
- Worked on HBase to load and retrieve data for real time processing using Rest API.
- Imported data from RDBMS to column families in Cassandra through Storage Handler
- Experienced with different file formats like CSV, Text files, Sequence files, XML, JSON and Avro files.
- Assisted in monitoring Hadoop cluster using tools like Cloudera manager.
- Involved in unit testing of Map Reduce programs using Apache MR unit.
- Good knowledge on data analysis with R.
- Experience developing applications using Java, J2EE, JSP, MVC, Hibernate, JMS, JSF, EJB, XML, AJAX and web based development tools.
- Experience working with popular frame works like Spring MVC, Hibernate.
- Implemented SOAP based web services using .
- Used Curl scripts to test RESTful Web Services
- Experience in database design using PL/SQL to write Stored Procedures, Functions, Triggers and strong experience in writing complex queries for Oracle.
- Experience working with Build tools like Maven and Ant.
- Experienced in both Waterfall and Agile Development (SCRUM) methodologies
- Strong Problem Solving and Analytical skills and abilities to make Balanced & Independent Decisions.
- Good Team Player, Strong Interpersonal, Organizational and Communication skills combined with Self- Motivation, Initiative and Project Management Attributes.
TECHNICAL SKILLS:
NO SQL Databases: HBase, Cassandra, Mongo DB Hadoop Data
Services: Hive, Pig, Sqoop, FlumeHadoop Operational
Services: Zookeeper, Oozie
Monitoring Tools: Gangila, Cloudera Manager
Programming Languages: C, Java, Python, SQL, PL/SQL, Unix Shell Scripting
IDE Tools: Eclipse, RAD, Net beans Cloud Computing
Services: AWS (Amazon Web Services)
Java & J2EE Technologies: Core Java, Servlets, JSP, JMS, EJB, Junit
Frameworks: Hibernate, Spring, Struts
Version Control tools: CVS, SVN, Rational Clear Case
Application Servers: Web Logic, Web Sphere, Jboss, Tomcat.
Databases: Oracle, MySQL, Postgress, Teradata
Operating Systems: UNIX, Windows, LINUX, Macintosh OSX
Build Tools: Jenkins, Maven, ANT
Visualization Tools: Tableau
WORK EXPERIENCE:
Confidential, Memphis, TN
Sr. Hadoop Developer
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop.
- Implemented nine nodes CDH3 Hadoop cluster on Red hat LINUX.
- Involved in loading data from Oracle database into HDFS using Sqoop queries.
- Implemented Map reduces programs to get Top K Results using Map Reduce programs by fallowing
- Map Reduce Design Patterns.
- Involved in loading the created HFiles into HBase for faster access of large customer base without taking
- Performance hit.
- Implemented working with different sources using Multi Input formats using Generic and Object
- Writable.
- Implemented best income logic using Pig scripts and Joins to transform data to Confidential custom formats.
- Implemented custom comparators and partioners to implement Secondary Sorting.
- Worked on tuning the performance of Hive queries.
- Implemented Hive Generic UDF's to implement business logic.
- Responsible to manage data coming from different sources.
- Configured Time Based Schedulers that get data from multiple sources parallel using Oozie work flows.
- Installed Oozie workflow engine to run multiple Hive and pig jobs.
- Used Zookeeper for providing coordinating services to the cluster.
- Coordinated with end users for designing and implementation of analytics solutions for User Based
- Recommendations using R as per project proposals.
- Assisted monitoring Hadoop cluster using Gangila
- Implemented test scripts to support test driven development and continuous integration.
- Configured build scripts for multi module projects with Maven and Jenkins CI.
- Involved in story-driven agile development methodology and actively participated in daily scrum meetings.
Environment: Hadoop, Map Reduce, HDFS, Pig, Hive, Oozie, Java, Linux, Maven, Oracle 11g/10g, Zookeeper, SVN, Gangila
Confidential, Dallas, TX
Hadoop Developer
Responsibilities:
- Imported Data from Different Relational Data Sources like RDBMS, Teradata to HDFS using Sqoop.
- Imported Bulk Data into HBase Using Map Reduce programs.
- Perform analytics on Time Series Data exists in HBase using HBase API.
- Designed and implemented Incremental Imports into Hive tables.
- Used Rest ApI to Access HBase data to perform analytics.
- Worked in Loading and transforming large sets of structured, semi structured and unstructured data
- Involved in collecting, aggregating and moving data from servers to HDFS using Apache Flume
- Written Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data.
- Involved in creating Hive tables, loading with data and writing hive queries that will run internally in map reduce way.
- Experienced in managing and reviewing the Hadoop log files.
- Migrated ETL jobs to Pig scripts do Transformations, even joins and some pre-aggregations before storing the data onto HDFS.
- Worked with Avro Data Serialization system to work with JSON data formats.
- Worked on different file formats like Sequence files, XML files and Map files using Map Reduce
- Programs.
- Involved in Unit testing and delivered Unit test plans and results documents using Junit and MRUnit.
- Exported data from HDFS environment into RDBMS using Sqoop for report generation and visualization purpose.
- Worked on Oozie workflow engine for job scheduling.
- Created and maintained Technical documentation for launching HADOOP Clusters and for executing Pig
- Scripts.
Environment: Hadoop, HDFS, Map Reduce, Hive, Oozie, Sqoop, Pig, Java, Rest API, Maven, MRUnit, Junit.
Confidential, Newark, NJ
Hadoop Developer
Responsibilities:
- Developed PIG UDF'S for manipulating the data according to Business Requirements and also worked on developing custom PIG Loaders.
- Developed Java Map Reduce programs on log data to transform into structured way to find user location, age group, spending time.
- Implemented Row Level Updates and Real time analytics using CQL on Cassandra Data.
- Collected and aggregated large amounts of web log data from different sources such as webservers, mobile and network devices using Apache Flume and stored the data into HDFS for analysis.
- Developed PIG scripts for the analysis of semi structured data.
- Worked on the Ingestion of Files into HDFS from remote systems using MFT (Managed File
- Transfer).
- Analyzed the web log data using the HiveQL to extract number of unique visitors per day, page views, visit duration, most purchased product on website.
- Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (such as Map-Reduce, Pig, Hive, and Sqoop) as well as system specific jobs (such as Java programs and shell scripts).
Environment: Hadoop, HDFS, Map Reduce, Hive, Pig, Sqoop, Oozie, Cassandra, Java, Shell Scripting, MySQL, SQL
Confidential, San Fransisco, CA
Senior Java / J2EE Developer
Responsibilities:
- Used HTML, JavaScript, Servlets, and JSP in the presentation tier. Developed business delegates to communicate with the Enterprise Java Beans (EJB) and web services residing on the business tier.
- Responsible and active in the analysis, definition, design, implementation, management, and deployment of full software development life cycle of project.
- Development of the application that was used to create dynamic JSPs, given input from a database table containing information about the contents of the JSP being developed.
- Implemented Business Logic Components using Spring Core and navigation using Spring MVC.
- Implemented Persistent entities and Persistent Layers using Hibernate API and Interfaces.
- Implemented SOAP based Web Services for Enterprise Logging Application Module.
- Responsible for coding SQL Statements and Stored procedures for back end communication using
- JDBC
- Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
- Involved in the development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.
Environment: J2EE, Java, XML, JAXB, JNDI, SOAP, Web sphere Application Server, Linux, SQL Server, Rational Rose, and Oracle
Confidential
Java Developer
Responsibilities:
- Used Hibernate ORM tool as persistence Layer - using the database and configuration data to provide persistence services (and persistent objects) to the application.
- Implemented Oracle Advanced Queuing using JMS and Message driven beans.
- Responsible for developing DAO layer using Spring MVC and configuration XML’s for Hibernate and to also manage CRUD operations (insert, update, and delete).
- Implemented Dependency injection of spring frame work.
- Developed and implemented the DAO and service classes.
- Developed reusable services using BPEL to transfer data.
- Participated in Analysis, interface design and development of JSP.
- Configured log4j to enable/disable logging in application.
- Developed Rich user interface using HTML, JSP, AJAX, JSTL, Java Script, JQuery and CSS.
- Implemented PL/SQL queries, Procedures to perform data base operations.
- Wrote UNIX Shell scripts and used UNIX environment to deploy the EAR and read the logs.
- Implemented Log4j for logging purpose in the application.
- Involved in code deployment activities for different environments.
- Implemented agile development methodology.
Environment: Java, Spring, Hibernate, JMS, EJB, Web logic Server, JDeveloper, Sql Developer, Maven, XML, CSS, JavaScript, JSON.