Hadoop Engineer Resume
Roseville, CA
SUMMARY:
- 7+ years of overall experience in Systems Administration and Enterprise Application Development in diverse industries which includes hands on experience in Big data ecosystem related technologies.
- 2 years of comprehensive experience as a Big Data & Analytics Administrator.
- Experience in working with MapReduce programs using Apache Hadoop for working with Big Data.
- Experience in installation, configuration, supporting and monitoring Hadoop clusters using Apache, Cloudera distributions and AWS.
- Experience in using Pig, Hive, Scoop, HBase and Cloudera Manager.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa.
- Hands on experience in application development using Java, RDBMS, and Linux shell scripting
- Extending Hive and Pig core functionality by writing custom UDFs.
- Experience in analyzing data using HiveQL, Pig Latin, and custom Map Reduce programs in Java.
- Familiar with Java virtual machine (JVM) and multi-threaded processing.
- Worked on NoSQL databases including HBase, Cassandra and MongoDB.
- Knowledge in job workflow scheduling and monitoring tools like oozie and Zookeeper
- Experience in designing, developing and implementing connectivity products that allow efficient exchange of data between our core database engine and the Hadoop ecosystem.
- Experience as a Java Developer in Web/intranet, client/server technologies using Java, J2EE, Servlets, JSP, JSF, EJB, JDBC and SQL.
- Good understanding of XML methodologies (XML,XSL,XSD) including Web Services and SOAP
- Familiar with data warehousing and ETL tools like Informatica and Pentaho.
- Techno-functional responsibilities include interfacing with users, identifying functional and technical gaps, estimates, designing custom solutions, development, leading developers, producing documentation, and production support.
- Excellent interpersonal and communication skills, creative, research-minded, technically competent and result-oriented with problem solving and leadership skills.
TECHNICAL SKILLS:
Hadoop/Big Data: HDFS,Mapreduce,HBase,Pig,Hive,Sqoop,Flume,MongoDB,Cassandra, Power pivot, Puppet, oozie, Zookeeper
Java & J2EE Technologies: Core Java, Servlets, JSP, JDBC, JNDI, Java Beans
IDE s: Eclipse, Net beans
Big data Analytics: Datameer 2.0.5
Frameworks: MVC, Struts, Hibernate, Spring
Programming languages: C,C++, Java, Python, Ant scripts, Linux shell scripts
Databases: Oracle 11g/10g/9i, MySQL, DB2, MS-SQL Server
Web Servers: Web Logic, Web Sphere, Apache Tomcat
Web Technologies: HTML, XML, JavaScript, AJAX, SOAP, WSDL
Network Protocols: TCP/IP, UDP, HTTP, DNS, DHCP
ETL Tools: Informatica, Pentaho
Testing: Win Runner, Load Runner, QTP
PROFESSIONAL EXPERIENCE:
Confidential, Roseville, CA
Hadoop Engineer
Environment:: Hadoop, MapReduce, HDFS, Hive, Java, SQL, Cloudera Manager, Pig, Sqoop, Oozie
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop.
- Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
- Worked hands on with ETL process.
- Upgrading the Hadoop Cluster from CDH3 to CDH4 and setup High availability Cluster Integrate the HIVE with existing applications
- Configured Ethernet bonding for all Nodes to double the network bandwidth
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from Teradata into HDFS using Sqoop.
- Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
- Continuous monitoring and managing the Hadoop cluster through Cloudera Manager.
- Installed Oozie workflow engine to run multiple Hive and Pig jobs.
- Developed Hive queries to process the data and generate the data cubes for visualizing.
Confidential, San Francisco, CA
Hadoop Admin/ Developer
Environment:: Hadoop, MapReduce, HDFS, Hive, Oracle 11g, Java, Struts, Servlets, HTML, XML, SQL, J2EE, JUnit, Tomcat 6.
Responsibilities:
- Installed and configured Hadoop MapReduce, HDFS and developed multiple MapReduce jobs in Java for data cleansing and preprocessing.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Involved in defining job flows, managing and reviewing log files.
- Extracted files from CouchDB through Sqoop and placed in HDFS and processed.
- Load and transform large sets of structured, semi structured and unstructured data.
- Responsible to manage data coming from different sources.
- Supported Map Reduce Programs those are running on the cluster.
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive UDFs.
- Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
- Gained very good business knowledge on health insurance, claim processing, fraud suspect identification, appeals process etc.
Confidential, MD
Java/J2EE Developer
Environment:: Java 1.4,Struts, JSP, Servlets API, HTML, JDBC, Web Sphere 5.1,MQ Series, MS SQL server, XSLT, XML, EJB, Edit Plus, EJB, JUnit, CSS,JMS, Hibernate, Eclipse, and WSAD
Responsibilities:
- Responsible for the design and development of the framework. The system is designed using J2EE technologies based on MVC architecture.
- Developed Session Beans using J2EE Design Patterns.
- Implemented J2EE Design patterns like Data Access Objects, Business Objects, and Java Design Patterns like Singleton.
- Extensively used MQ series.
- Extensive use of Struts framework.
- Used JSP and Servlets, EJBs on server side.
- Implemented Home Interface, Remote Interface, and Bean Implementation class.
- Implemented business logic at server side using Session Bean.
- Wrote PL/SQL queries to access data from Oracle database.
- Set up Web sphere Application server and used Ant tool to build the application and deploy the application in Web sphere.
- Developed the application using WSAD.
- Prepared test plans and writing test cases
- Worked on Hibernate.
Confidential, Raleigh, NC
Application Developer J2EE
Environment:: Java, Web Sphere 3.5, EJB, Servlets, JavaScript, JDBC, SQL, JUnit, Eclipse IDE. Apache Tomcat 6
Responsibilities:
- Developed JavaScript behavior code for user interaction.
- Created database program in SQL server to manipulate data accumulated by internet transactions.
- Wrote Servlets class to generate dynamic HTML pages.
- Developed Servlets and back-end Java classes using Web Sphere application server.
- Developed an API to write XML documents from a database.
- Performed usability testing for the application using JUnit Test.
- Maintenance of a Java GUI application using JFC/Swing.
- Created complex SQL and used JDBC connectivity to access the database.
- Involved in the design and coding of the data capture templates, presentation and component templates.
- Part of the team that designed, customized and implemented metadata search and database synchronization.
- Used Oracle as Database and used Toad for queries execution and also Involved in writing SQL scripts, PL SQL code for procedures and functions
Confidential, NC
JAVA Developer
Environment: Oracle 11g, Java 1.5, Struts, Servlets, HTML, XML, SQL, J2EE, JUnit, Tomcat 6.
Responsibilities:
- Responsible and active in the analysis, design, implementation and deployment of full Software Development Lifecycle (SDLC) of the project.
- Designed and developed user interface using JSP, HTML and JavaScript.
- Developed Struts action classes, action forms and performed action mapping using Struts framework and performed data validation in form beans and action classes.
- Extensively used Struts framework as the controller to handle subsequent client requests and invoke the model based upon user requests.
- Defined the search criteria and pulled out the record of the customer from the database. Make the required changes and save the updated record back to the database.
- Validated the fields of user registration screen and login screen by writing JavaScript validations.
- Developed build and deployment scripts using Apache ANT to customize WAR and EAR files.
- Used DAO and JDBC for database access.
- Developed stored procedures and triggers using PL/SQL in order to calculate and update the tables to implement business logic.
- Design and develop XML processing components for dynamic menus on the application.
- Involved in post production support and maintenance of the application.
Confidential
Junior JAVA Developer
Environment:: Java, JSP, Servlets, JDBC, JavaScript, MySQL, JUnit, Eclipse IDE.
Responsibilities:
- Involved in the analysis, design, implementation, and testing of the project.
- Implemented the presentation layer with HTML, XHTML and JavaScript.
- Developed web components using JSP, Servlets and JDBC.
- Implemented database using SQL Server.
- Designed tables and indexes.
- Wrote complex SQL and stored procedures.
- Involved in fixing bugs and unit testing with test cases using JUnit.
- Developed user and technical documentation.