We provide IT Staff Augmentation Services!

Big Data/ Hadoop Developer Resume

TexaS

PROFESSIONAL SUMMARY:

  • Over 7 years of IT experience in Analysis, Design, Development and user training of software applications which includes over 4 Years in Big Data, Hadoop and HDFS environment and around 3 Years of experience in JAVA, J2EE.
  • Experience in developing and Implementing MapReduce programs using Hadoop to work with Big Data as per the requirement.
  • Hands on experience in Apache Hadoop ecosystem components like Hadoop Distributing File System (HDFS), MapReduce, Hive, Sqoop, Maven, HBase, PIG, Kafka, Zoo Keeper, Scala, Flume, Storm and Oozie.
  • Worked on Hive and HBase integration.
  • Good Knowledge of Hadoop architecture and various components such as HDFS Framework, Job Tracker, Task Tracker, Name Node, Data Node and MRV1 and MRV2 (YARN).
  • Experience in developing MapReduce jobs in Java for data cleansing, transformations, pre - processing and analysis. Multiple mappers are implemented to handle data from multiple sources.
  • Experience in installation, configuration and management of Hadoop Clusters.
  • Experienced in installing, configuring, and administrating Hadoop cluster of major Hadoop distributions Hortonworks, Cloudera.
  • Knowledge on Hadoop daemon functionalities, resource utilizations and dynamic tuning in order to make cluster available and efficient.
  • Hands-on writing custom UDF’s for extending Hive and Pig core functionality.
  • Knowledge on installing, configuring, and using Hadoop components like Hadoop Map Reduce (MR1), YARN (MR2), HDFS, Hive, Pig, Flume and Scoop.
  • Experience in setting up data gathering tools such as Flume and Sqoop.
  • Experience in working with Flume to load the log data from multiple sources directly into HDFS.
  • Knowledge on NOSQL Databases like Cassandra, MongoDB and HBase.
  • Experience in Hive Partitioning, bucketing and perform different types of joins on Hive tables and implementing Hive serdes like REGEX, JSON and Avro.
  • Experience in Scripting using UNIX shell script.
  • Experience in analysing, designing and developing ETL strategies and processes, writing ETL specifications, Informatica development.
  • Extensively used Informatica Power Centre for Extraction, Transformation and Loading process.
  • Extensively worked on the ETL mappings, analysis and documentation of OLAP reports requirements. A good understanding of OLAP concepts working especially with large data sets.
  • Experience in Dimensional Data Modelling using star and snow flake schema.
  • Good knowledge on Data Mining and Machine Learning techniques.
  • Proficient in Oracle 9i/10g/11g, SQL and PL/SQL.
  • Experience in integration of various data sources like Oracle, DB2, Sybase, SQL server and MS access and non-relational sources like flat files into staging area.
  • Developed core modules in large cross-platform applications using JAVA, J2EE with experience in Java core concepts like OOPS, Multi-threading, Collections and IO.
  • Hands-on JDBC, java script, Jquery, Linux, Unix HTML, AWS.
  • Developed applications using Java, RDBMS, and Linux shell scripting.
  • Have good interpersonal skills, good communication, problem solving skills and a motivated team player.
  • Have the ability to be a value contribution to the company.

TECHNICAL SKILLS:

Hadoop Eco System: Hadoop, Map Reduce, Sqoop, Hive, Oozie, Pig, HDFS, ZooKeeper, Flume, HBase, Impala, Spark, Storm, Hadoop( Cloudera, Horton Works and Pivotal), Solr

No SQL Databases: HBase, Cassandra, MongoDB

Java & J2EE Technologies: Core Java, Servlets, JSP, JDBC, NetBeans, Eclipse

Languages: C, Java, SQL, PL/SQL, PIG Latin, HiveQL, Unix Shell Scripting

Databases: Oracle 11g/10g/9i, My SQL, DB2, MS SQL Server

Application Server: Apache Tomcat, JBoss, IBM Web sphere, Web Logic

Web Services: WSDL, SOAP, REST

Methodologies: Agile, Scrum

Web Technologies: AJAX

PROFESSIONAL EXPERIENCE

Confidential, TEXAS

Big Data/ Hadoop Developer

Responsibilities:

  • Responsible for building scalable distributed data solutions using Hadoop and migrate legacy Retail applications ETL to Hadoop.
  • Extracted data from Teradata to HDFS using Sqoop.
  • Hands on extracting data from different databases and to copy into HDFS using Sqoop and have an expertise in using compression techniques to optimize the data storage.
  • Implemented ETL code to load data from multiple sources into HDFS using pig scripts.
  • Hands on creating different applications in social networking websites and obtaining access data from them.
  • Wrote Map Reduce jobs using the access tokens to get the data from the customers.
  • Developed simple to complex Map Reduce jobs using Hive and Pig for analyzing the data.
  • Used different Serdes for converting JSON data into pipe separated data.
  • Implemented some business logics by writing UDFs in Java and used various UDFs from Piggybanks and other sources to get some results from the data.
  • Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
  • Used Oozie workflow to automate all the jobs.
  • Hands on exporting the analyzed data into relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Performed unit testing using MRUnit and JUnit Testing Framework and Log4J to monitor the error log.
  • Involved in installing and configuring Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster.
  • Experience in building analytics for structured and unstructured data and managing large data ingestion by using Avro, Flume, Thrift, Kafka and Sqoop.
  • Deep understanding of scalable distributed computing systems, software architecture, data structures and algorithms using Hadoop, Apache Spark, Apache Storm etc.
  • Ingesting streaming data into Hadoop using Spark, Storm Framework and Scala.
  • Implemented Flume, Spark framework for real time data processing.
  • Exported the patterns analyzed back to Teradata using Sqoop.
  • Developed Bankers Rounding UDF for Hive/Pig or Implemented Teradata Rounding in Hive/Pig.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Continuously monitored and managed the Hadoop Cluster using Cloudera Manager.
  • Hands on experience in Tableau for Data Visualization and analysis on large data sets, drawing various conclusions.

Environment:: Hadoop, Map Reducer, Cloudera Manager, HDFS, Hive, Pig, Spark, Storm, Flume, Thrift, Kafka, Sqoop, Oozie, Impala, SQL, Scala, Java (JDK 1.6), Hadoop(Cloudera), Tableau, Eclipse and Informatica 9.1.

Confidential, ILLINOIS

Big Data/ Hadoop Developer

Responsibilities:

  • Responsible for building scalable distributed data solutions using Hadoop and migrate legacy Retail applications ETL to Hadoop.
  • Accessed information through mobile networks and satellites from the equipment.
  • Hands on extracting data from different databases and to copy into HDFS file system using Sqoop.
  • Implemented ETL code to load data from multiple sources into HDFS using pig scripts.
  • Hands on creating different applications in social networking websites and obtaining access data from them.
  • Wrote Map Reduce jobs using the access tokens to get the data from the customers.
  • Developed simple to complex Map Reduce jobs using Hive and Pig for analysing the data.
  • Used different Serdes for converting JSON data into pipe separated data.
  • Implemented some business logics by writing UDFs in Java and used various UDFs from Piggybanks and other sources to get some results from the data.
  • Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
  • Used Oozie workflow engine to run multiple Hive and Pig jobs.
  • Hands on exporting the analysed data into relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Developed wrapper using shell scripting for Hive, Pig, Sqoop, Scala jobs .
  • Worked on monitoring log input from several data centers, via Spark Stream , was analysed in Apache Storm.
  • Performed unit testing using MRUnit and JUnit Testing Framework and Log4J to monitor the error log.
  • Hands on experience in streaming data using Kafka.
  • Involved in installing and configuring Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Knowledge on pushing data as delimited files into HDFS using Talend Big Data Studio.

Environment:: Hadoop, Map Reducer, HDFS, Hive, Pig, Spark, Storm, Kafka, Flume, Sqoop, Oozie, SQL, Scala, Java (jdk 1.6), Hadoop(Horton Works), Eclipse, Talend Studio and Informatica 9.1.

Confidential, AR

Hadoop Developer

Responsibilities:

  • Gathered business requirements from the Business Partners and subject matter experts and prepared Business Requirement document.
  • Involved in converting Business Requirements into Technical Design.
  • Developed simple to complex Map/Reduce jobs using Hive and Pig.
  • Optimized Map/Reduce jobs to use HDFS efficiently by using various compression mechanisms.
  • Handled importing of data from various data sources performed transformations using Hive, MapReduce, loaded data into HDFS and extracted the data from MySQL into HDFS using sqoop.
  • Analysed the data by performing Hive queries and Pig scripts to study customer behaviour.
  • Used UDF’s to implement business logic in Hadoop.
  • Experience using NoSQL database Cassandra for information retrieval.
  • Have experience in implementation of Regression analysis using MapReduce.
  • Exported the analysed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Experience in developing scripts and Batch Jobs to schedule various Hadoop programs using Oozie.
  • Experience in testing MapReduce code using JUnit testing.
  • Cloudera manager is used to monitor the health of the jobs which are running on the cluster.

Environment: Java (jdk 1.6), Hadoop, MapReduce, Pig, Hive, Cassandra, Sqoop, Oozie, HDFS, Hadoop (Cloudera), MySQL, Eclipse, Oracle.

Confidential

JAVA Developer

Responsibilities:

  • Prepared High Level and Low Level Design document implementing applicable Design Patterns with UML diagrams to depict components, class level details.
  • Interacting with the system analysts & business users for design & requirement clarification.
  • Developed Web Services using SOAP, SOA, WSDL Spring MVC and developed DTDs, XSD schemas for XML (parsing, processing, and design) to communicate with Active Directory application using Restful API.
  • Developed JSPs according to requirement.
  • Developed the Web Based Rich Internet Application (RIA) using Adobe Flex.
  • Excellent knowledge of NOSQL on Mongo and Cassandra DB.
  • Developed integration services using SOA, Mule ESB, Web Services, SOAP, and WSDL.
  • Designed, developed and maintained the data layer using the ORM framework in Hibernate.
  • Involved in Analysis, Design, Development, and Production of the Application and develop UML diagrams.
  • Developed web applications using Spring MVC, HTML4, Bootstrap.
  • Presented top level design documentation to the transition of various groups.
  • Used Spring framework's JMS support for writing to JMS Queue, Hibernate Dao Support for interfacing with the database and integrated spring with JSF.
  • Wrote AngularJS controllers, views, and services.
  • Ant is used for building, and the application is deployed on JBoss application server.
  • Taken care of Java Multithreading part in back end components.
  • Experience in web development, client-server and n-tier Enterprise applications using Java/J2ee technologies and Adobe Flex .
  • Implemented by using MYSQL.
  • Developed HTML reports for various modules as per the requirement.
  • Analyse known information into concrete concepts and technical solutions.
  • Assisted in writing the SQL scripts to create and maintain the database, roles, users, tables in SQL Server.

Environment: Java, JDBC, Spring, JSP, JBoss, Servlets, Maven, Jenkins, Flex, HTML, AngularJS, Mongo DB, Hibernate, JavaScript, Eclipse, Struts, SQL Server2000.

Confidential

Jr. JAVA Developer

Responsibilities:

  • Analysed Object Oriented Design and presented with UML Sequence, Class Diagrams.
  • Developed Admission & Census module, which monitors a wide range of detailed information for each resident upon admission.
  • Developed Plans module, which provides a comprehensive library of problems, goals and approaches. You have the option of tailoring (adding, deleting, or editing problems, goals and approaches) these libraries and the disciplines you will use for your plans.
  • Developed General Ledger module, which streamlines analysis, reporting and recording of accounting information. General Ledger automatically integrates with a powerful spreadsheet solution for budgeting, comparative analysis and tracking facility information for flexible reporting.
  • Developed UI using HTML, JavaScript, and JSP, and developed Business Logic and Interfacing components using Business Objects, XML, and JDBC.
  • Designed user-interface and checking validations using JavaScript.
  • Managed connectivity using JDBC for querying/inserting & data management including triggers and stored procedures.
  • Developed components using Java multithreading concept.
  • Developed various EJBs (session and entity beans) for handling business logic and data manipulations from database.
  • Involved in design of JSP’s and Servlets for navigation among the modules.
  • Designed cascading style sheets and XSLT and XML part of Order entry Module & Product Search Module and did client side validations with java script.
  • Hosted the application on Web Sphere.

Environment: : J2EE, Java/JDK, PL/SQL, JDBC, JSP, Servlets, JavaScript, EJB, JavaBeans, UML, XML, XSLT, Oracle 9i, HTML/ DHTML, UML, JavaScript.

Hire Now