Hadoop Developer Resume
DallaS
SUMMARY
- Over 8 years of experience in Analysis, Design, Implementation and Testing of Object Oriented Applications using Java/J2EE which includes 3 years of experience in Hadoop Technologies
- Experience in performance tuning the Hadoop cluster by gathering and analyzing the existing infrastructure.
- Hands on experience in Big Data processing using Apache Hadoop and its ecosystem (MapReduce, Pig, Hive, Sqoop, Flume and HBase).
- Experience in installing, configuring and maintaining the HadoopCluster.
- Excellent ability to use analytical tools to mine data and evaluate the underlying patterns.
- Experience in working with flume to load the log data from multiple sources directly into HDFS.
- Experience in configuring the Zookeeper to coordinate the servers in clusters and to maintain the data consistency.
- Experience in upgrading the existing Hadoop cluster to latest releases.
- Experienced in using NFS (network file systems) for Name node metadata backup.
- Experience in using Cloudera Manager 4.0 for installation and management of Hadoop cluster.
- Experience in supporting data analysis projects using Elastic Map Reduce on the Amazon Web Services (AWS) cloud. Exporting and importing data into S3.
- Experience in supporting analysts by administering and configuring HIVE.
- Experience in providing support to data analyst in running Pig and Hive queries.
- Expert in working with Hive data warehouse tool - creating tables, data distribution by implementing partitioning and bucketing, writing and optimizing the HiveQL queries.
- Developed Map Reduce programs to perform analysis.
- Performed Importing and exporting data into HDFS and Hive using Sqoop.
- Experience in writing shell scripts to dump the Sharded data from MySQL servers to HDFS.
- In depth understanding/knowledge of HadoopArchitecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce concepts.
- Extensive experience with SQL, PL/SQL and database concepts.
- Knowledge of NoSQL databases such as Hbase, Cassandra.
- Expertise in RDBMS namely Oracle, MS-SQL Server, MySQL and DB2, Sybase and Stored procedures & Triggers.
- Good working Experience in client-side development with HTML, XHTML, CSS, JavaScript, JQuery, JSON and AJAX.
- Extensive experience with Java compliant IDE’s like Eclipse 3.0.
- Extensive experience in developing Internet and Intranet related applications using J2EE, Servlets, JSP, WebLogic, Tomcat.
- Hands on experience with JUnit, and Log4j in developing test cases and determining application functionality and good experience in using ANT and MAVEN tool.
- Experience using an Integrated Development Environment including Eclipse, MyEclips, and NetBeans.
- Possesses excellent communication skills and good team player with good problem solving and trouble-shooting capabilities and will prove a valuable asset to any project.
TECHNICAL SKILLS:
Languages: Java, J2EE, C++, C, SQL/PLSQL, PIG LATIN, UNIX Shell Scripting
Hadoop Ecosystem Development: HDFS, Hive, Pig, Flume, Oozie, Zookeeper, HBASE and Sqoop
Web Technologies: JSP, JQuery, XML, JSON, HTML, XSLT, JavaScript, CSS, DHTML, Servlets, JSF, Ajax
Databases: ORACLE, DB2, SQL Server, MYSQL
Design & Modelling: UML, Design Patterns, Microsoft Visio, Rational Rose 3.0, Agile SCRUM
Tools: /IDES RAD 7.5,Net Beans and Eclipse
Build Tools: ANT, Maven
PROFESSIONAL EXPERIENCE
Confidential
Hadoop Developer
Responsibilities:
- Gathered the business requirements from the Business partners and subject Matter Experts.
- Responsible to manage data coming from different sources.
- Supported MapReduce Programs which are running on the cluster.
- Developed MapReduce programs to perform data filtering for unstructured data.
- Wrote MapReduce jobs using Java API.
- Experienced in writing MapReduce jobs using Pig Latin scripts and Pig UDF's in Java.
- Prepare dataset with Hadoop and import it into Neo4J to be able to query and visualize the data.
- Used Flume to channel data from different sources to HDFS.
- Created HBase tables to store data depending on column families.
- Developed scripts and batch jobs to schedule various Hadoop programs.
- Wrote Hive queries for data analysis to meet the business requirements.
- Created Partitioned Hive tables and worked on them using HiveQL.
- Designed and Maintained Oozie workflows to manage the flow of jobs in the cluster.
- Actively updated the upper management with daily updates on the progress of project that include the classification levels that were achieved on the data.
Environment: Java, Hadoop, MapReduce, HDFS, Pig, Hive, Linux, Ubuntu.
Confidential, Dallas
Hadoop Developer
Responsibilities:
- Developed Map Reduce program for parsing and loading the streaming data into HDFS information regarding messaging objects
- Developed Hive queries to pre-process the data for analysis by imposing read only structure on the stream data
- Hands on experience in leading complex integration between AWS services (EMR, EC2, S3) and distributed legacy data centers in both Research and development and production environments.
- Developed workflow using Oozie for running Map Reduce jobs and Hive Queries
- Used Sqoop for exporting data into MYSQL
- Worked with Agile methodologies and have use scrum in the process
- Worked on defining job flows, jobs management using Fair scheduler
- Worker on Cluster coordination services through Zookeeper
- Worked on loading log data directly into HDFS using Flume.
- Worked with Agile Methodologies and have used scrum in the process.
Environment: Hadoop, HDFS, HIVE, Oozie, Sqoop, Flume, Zookeeper, MYSQL, Agile, Linux.
Confidential, CA
Hadoop developer
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop
- Installed and configured Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster
- Setup and benchmarked Hadoop/HBase clusters for internal use
- Developed Simple to complex Map/reduce Jobs using Hive and Pig
- Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop
- Analyzed the data by performing Hive queries and running Pig scripts to study customer behavior
- Used UDF’s to implement business logic in Hadoop
- Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and other sources.
- Continuous monitoring and managing the Hadoop cluster using Cloudera Manager
- Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team
Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Java, SQL, Cloudera Manager, Sqoop, Flume, Java (jdk1.6), Eclipse.
Confidential
JavaDeveloper
Responsibilities:
- Worked on Agile Methodology to meet timelines with quality deliverables.
- Create functional and technical specifications for products.
- Involved in entire life cycle development of the application.
- Developed the Application using Spring MVC Framework by implementing Controller, Service classes.
- Involved in the development of JSPs and Servlets for different User Interfaces.
- Used AJAX for high intensive user operations.
- Implemented the web services to access the order management applications at different geographical locations. Utilized JAXM to send the SOAP based messages to the service.
- Developed JavaBeans and mapping files for Hibernate Object Relation Mapping for data persistence & storage.
- Resolve application problems, support end-users, and manage the processes of reporting, diagnosing, and troubleshooting.
- Implemented Maven build tool to build jar files.
- Used Clear Case Version Control for Project Configuration Management.
- Build and manage relationship with business, development, change management, IT Operations, and project managers to resolve time critical issues and troubles.
Environment: Java, Agile, MVC, Spring, Hibernate, JSP, AJAX, JSON, Web services, WebLogic, Oracle(SQL/PLSQL), JBuilder, Toad, Oracle SQLDeveloper, Maven, Clear Case.
Confidential
Java/j2ee Developer
Responsibilities:
- Involved in analysis and design of the application.
- Involved in preparing the detailed design document for the project.
- Developed the application using J2EE architecture.
- Involved in developing JSP forms.
- Designed and developed web pages using HTML and JSP.
- Designed various applets using JBuilder.
- Designed and developed Servlets to communicate between presentation and business layer.
- Used EJB as a middleware in developing a three-tier distributed application.
- Developed Session Beans and Entity beans to business and data process.
- Used JMS in the project for sending and receiving the messages on the queue.
- Developed the Servlets for processing the data on the server.
- The processed data is transferred to the database through Entity Bean.
- Used JDBC for database connectivity with MySQL Server.
- Used CVS for version control.
- Involved in unit testing using Junit.
Environment: CoreJava, J2EE, JSP, Servlets, XML, XSLT, EJB, JDBC, JBuilder 8.0, JBoss, Swing, JavaScript, JMS, HTML, CSS, MySQL Server, CVS, Windows 2000
