We provide IT Staff Augmentation Services!

Big Data/ Hadoop Developer Resume

0/5 (Submit Your Rating)

Boston, MA

SUMMARY

  • 7+ years of experience in Analysis, Architecture, Design, Development, Testing, Maintenance and User training of software application which includes 3+ Years in Big Data, Hadoop and HDFS environment and around 4 Years of experience in JAVA, J2EE
  • Hands on experience on major components in Hadoop Ecosystem like HDFS, HIVE, PIG, Hbase, Zookeeper, Ambari, Sqoop, Oozie, Flume and Avro.
  • Experience in writing with Map Reduce Programs using Apache Hadoop for working with Big data as per requirement.
  • Experience in importing and exporting the data using Sqoop using HDFS to relational database systems/mainframe and vice versa
  • Hands on dealing with log files to extract data and to copy into HDFS using flume.
  • Expertise in writing Hadoop jobs for analyzing data using Hive and Pig.
  • Hands on writing custom UDFs for extending Hive and Pig core functionality.
  • Experience with Oozie workflow Engine in running workflow jobs with actions that run Hadoop Map Reduce and Pig jobs.
  • Wrote Hadoop Test Cases in Hadoop for checking Input and Outputs.
  • Hands on integrating Hive and HBase.
  • Experience in working with Data Mining and Machine Learning techniques.
  • Experienced the integration of various data sources like Java, RDBMS, Shell Scripting, Spreadsheets, and Text files.
  • Expertise in tools and utilities like Eclipse, TOAD for Oracle, Rational Rose (UML tool), JCreator, WSAD, RAD, Ant and Maven.
  • Hands on experience in writing Ad - hoc Queries for moving data from HDFS to HIVE and analyzing the data using HIVE QL
  • Experience in NOSQL databases: MongoDB, HBase, Cassandra
  • Knowledge on reporting tools like Tableau which is used to do analytics on data in cloud
  • Knowledge on installing, configuring and using Hadoop components like Hadoop Map Reduce (MR1), YARN (MR2), HDFS, Hive, Pig, Flume and sqoop.
  • Extensive experience with SQL, PL/SQL, Shell Scripting and database concepts
  • Experience in working with popular frameworks likes Struts, Hibernate, Spring, and Spring MVC
  • Expertise in developing Pig Latin Scripts and using Hive Query Language
  • Collected data from different sources like web servers and social media using flume for storing in HDFS and analyzing the data using Hadoop technologies
  • Experience in writing batch scripts in Ubuntu/UNIX to automate sequential script entry.
  • Hands on JAXWS, JSP, Servlets, Struts, Web Logic, Web Sphere, Hibernate, Spring, JBoss, JDBC, RMI, Java Script, Ajax, jQuery, Linux, Unix, WSDL, XML, HTML,AWS and Scala and Vertica.
  • Developed applications using Java, RDBMS, and Linux shell scripting

TECHNICAL SKILLS

Language: Java,J2EE, JavaScript

Big Data: Hadoop, Map Reduce, Hive, Pig, Sqoop and MRUnit Hive, Puppet, Ambari, HBase, MongoDB, Cassandra, PowerPivot, Pentaho, Zookeeper and Flume

Methodologies: Agile/Scrum, Waterfall

Java Technologies: CoreJava 1.5,Servlets 2.4,JSP 2.0,JDBC, JNDI, Java Beans

Web Technologies: Javascript, JSP, Servlets, JDBC, Shell scripting HTML, XML, AJAX, SOAP, WSDL

Framework: Spring, Hibernate, Struts

Web/Application Servers: Apache Tomcat, Jboss4.0, WebLogic, WebSphere and Bastion.

RDBMS: Oracle11g 10g 9i, DB2, MySQL

IDE: Eclipse, Flex Builder, RAD

Source Control: SVN, Visual Source Safe, Clearcase, CVS

PROFESSIONAL EXPERIENCE

Confidential, Boston, MA

Big Data/ Hadoop Developer

Responsibilities:

  • Installed, configured, and maintained Apache Hadoop clusters for application development and major components of Hadoop Ecosystem: Hive, Pig, HBase, Sqoop, Flume, Oozie and Zookeeper
  • Implemented six nodes CDH4 Hadoop Cluster on CentOS
  • Extracted files from Cassandra throughSqoopand placed in HDFS and processed.
  • Importing and exporting data into HDFS and Hive from different RDBMS using Sqoop
  • Experienced in defining job flows to run multiple Map Reduce and Pig jobs using Oozie
  • Importing log files using Flume into HDFS and load into Hive tables to query data
  • Used HBase-Hive integration, written multiple Hive UDFs for complex queries
  • Created multiple Hive tables, implemented Partitioning, Dynamic Partitioning and Buckets in Hive for efficient data access
  • Written multiple Map Reduce programs in Java for data extraction, transformation and aggregation from multiple file formats including XML, JSON, CSV and other compressed file formats
  • Experienced in running batch processes using Pig Scripts and developed Pig UDFs for data manipulation according to Business Requirements
  • Written MapReduce program in Python with the Hadoop streaming API
  • Developed unit test cases for Hadoop MapReduce jobs with MRUnit
  • Excellent experience in ETL analysis, designing, developing, testing and implementing ETL processes including performance tuning and query optimizing of database
  • Experience in using Pentaho Data Integration tool for data integration, OLAP analysis and ETL process
  • Experience integrating R with Hadoop with RHadoop for statistical analysis and predictive modelling

Environment: Java, Javascript, Apache Hadoop, Hive, Pig, HBase, Sqoop, Flume, Java, Linux, MySQL Server5.155, MS SQL Server 2012, SQL, PL/SQL, SQL Server Data Tools, SQL Server

Confidential, Long island, NY

Hadoop Developer

Responsibilities:

  • Installed and configured Hadoop Mapreduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Installed and configured Pig and also written PigLatin scripts.
  • Developed PIG scripts using Pig Latin.
  • Involved in managing and reviewing Hadoop log files.
  • Exported data using Sqoop from HDFS to Teradata on regular basis.
  • Developing Scripts and Batch Job to schedule various Hadoop Program.
  • Written Hive queries for data analysis to meet the business requirements.
  • Creating Hive tables and working on them using Hive
  • Experienced indefining jobflows.
  • Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
  • Designed and implemented Mapreduce-based large-scale parallel relation-learning system.
  • Setup and benchmarked Hadoop clusters for internal use.
  • Monitoring the log flow from LM Proxy to ES-Head.
  • Used secportal as front end of Gracie where we perform the search operations.
  • Wrote the Map Reduce code for the flow from Hadoop Flume to ES Head

Environment: Java EE 6, IBM WebSphere Application Server 7, Apache-Struts 2.0, EJB 3, Spring 3.2, JSP 2.0, WebServices, JQuery 1.7, Hadoop, Hive, Pig, HBase, Sqoop, Flume, Java, Linux, MySQL Server

Hadoop Developer

Confidential, San Jose, CA

Responsibilities:

  • Proactively monitored systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
  • Worked with systems engineering team to plan and deploy new Hadoop environments and expand existing Hadoop clusters.
  • Monitored workload, job performance and capacity planning using Cloudera Manager.
  • Installed and configured Flume, Hive, Pig, Sqoop, HBase and Oozie on the Hadoop cluster.
  • Used Flume to collect, aggregate, and store the web log data from different sources like web servers, mobile devices and pushed to HDFS.
  • Good understanding of job schedulers like Fair Scheduler which assigns resources to jobs such that all jobs get, on average, an equal share of resources over time and an idea about Capacity Scheduler.
  • Responsible for performing peer code reviews, troubleshooting issues and maintaining status report.
  • Analyzed data using Hadoop components Hive and Pig.
  • Responsible for running Hadoop streaming jobs to process terabytes of xml's data.
  • Load and transform large sets of structured, semi structured and unstructured data using Hadoop/Big Data concepts.
  • Handled importing data from various data sources, performed transformations using Hive, Map Reduce, and loaded data into HDFS.
  • Imported and exported the data using Sqoop Export and Sqoop Import.

Environment: CDH4, Flume, Hive, HBase, Sqoop, Pig, Oozie, Cloudera Manager, Java, Linux, CentOS.

Confidential

Java Developer

Responsibilities:

  • Responsible for requirement gathering and analysis through interaction with end users.
  • Involved in designing use-case diagrams, class diagram, interaction using UML model with Rational Rose.
  • Designed and developed the application using various design patterns, such as session facade, business delegate and service locator.
  • Worked on Maven build tool.
  • Involved in developing JSP pages using Struts custom tags, JQuery and Tiles Framework.
  • Used JavaScript to perform client side validations and Struts-Validator Framework for server-side validation.
  • Involved in creating Database SQL and PL/SQL queries and stored Procedures.
  • Implemented Singleton classes for property loading and static data from DB.
  • Debugged and developed applications using Rational Application Developer (RAD).
  • Developed a Web service to communicate with the database using SOAP.
  • Developed DAO (data access objects) using Spring Framework 3.
  • Deployed the components in to WebSphere Application server 7.
  • Actively involved in backend tuning SQL queries/DB script.
  • Worked in writing commands using UNIX, Shell scripting.
  • Involved in developing other subsystems’ server-side components.
  • Production supporting using IBM clear quest for fixing bugs.
  • Used JSP for presentation layer, developed high performance object/relational persistence and query service for entire application utilizing Hibernate
  • Developed the XMLSchema and Web services for the data maintenance and structures

Environment: Java EE 6, IBM WebSphere Application Server 7, Apache-Struts 2.0, EJB 3, Spring 3.2, JSP 2.0, WebServices, JQuery 1.7, Servlet 3.0, Struts-Validator, Struts-Tiles, Tag Libraries, ANT 1.5, JDBC, Oracle 11g/SQL, JUNIT 3.8, CVS 1.2, Rational clear case, Eclipse 4.2,JSTL,DHTML

Confidential

Java Developer

Responsibilities:

  • Involved in Analysis of the requirements.
  • Prepared the High and Low level design document.
  • Created Creating UML artifacts - Use Cases, Class and Sequence Diagrams.
  • Developed Oracle PL/SQL procedures and functions.
  • Implemented Connection pool object for database connectivity.
  • Wrote hbm files and BO classes using Hibernate 3.3.1
  • Used XML parsers to parse incoming data and populating the database with the data
  • Designed the GUI screens using Struts and Configured log4j to debug the Application.
  • Involved in the development of test cases for the testing phase..
  • Performed End to end integration testing of online scenarios and unit testing usingJUnitTesting Framework.

Environment: Java, Windows XP, RAD 7.0.1, IBM Web Sphere Application Server 6.1, JSP, Struts, SQL, JDBC, Java Script, CSS, Log4j, Hibernate, HQL.

We'd love your feedback!