We provide IT Staff Augmentation Services!

Hadoop Developer Resume

3.00/5 (Submit Your Rating)

CA

PROFESSIONAL SUMMARY

  • 8+ years of extensive IT experience with multinational clients in variety of industries, which include expertise in Bigdata developing Bigdata/Hadoop applications.
  • Solid experience in Hadoop MRV1 and Hadoop MRV2 (or) YARN Architecture.
  • Extensive experience in Loading and Analyzing large datasets with Hadoop framework (MapReduce, HDFS, PIG, HIVE, Flume, Sqoop, Spark, Impala), NoSql databases like MongoDB, HBase, Cassandra.
  • Excellent working knowledge HDFS FileSystem, JobTracker, TaskTracker, NameNode, DataNode and MapReduce programming paradigm.
  • Strong experience in installing, configuring and using Hadoop Stack on multiple distributions i.e.Cloudera, MapR, Hortonworks and NoSQL platforms (HBase, Cassandra & MongoDB).
  • Strong experience in Object Oriented Analysis and Design, to develop Web/intranet, client/server technologies using Java, J2EE, Servlets, JSP, JSF, Springs, Hibernate.
  • Strong experience in Amazon AWS concepts like EMR, EC2, S3, Redshift, VPC, EBS, WorkSpaces which provides fast and efficient processing of Big Data Solutions.
  • Strong experienced in building highly scalable Big - data solutions using Hadoop.
  • Experience in managing and reviewing Hadoop log files.
  • Strong experience in writing MapReduce scripts using Scala, Java with Java API, Apache Hadoop API and Spark API for analyzing the data.
  • Extensive experience in writing custom UDF and UDAF MapReduce scripts.
  • Expert in deployment of Hadoop, Yarn, Spark and Storm integration with Cassandra, ignite and RabbitMQ, Kafka etc.
  • Strong experience in performance tuning of MapReduce, Pig jobs and Hive queries.
  • Extensive experience in migrating ETL operations into HDFS systems using Pig Scripts.
  • Experience with Oozie Workflow Engine in running workflow jobs with actions that run Hadoop MapReduce, Pig jobs and Hive Queries.
  • Experience in application development using Scala, Java and Linux/Unix Shell Scripting.
  • Extensive experience in Extraction, Transformation and Loading(ETL) of data from multiple sources into Data Warehouse and DataMarts.
  • Have extensive working knowledge on Advanced Machine Learning Language (AML), Agile environment.
  • Extensively worked on database applications using SAP Business Intelligence, Business Objects.
  • Solid understanding of the High volume, High Performance Systems.
  • Experienceinoptimizing the queries by creating variousclustered, non-clustered indexesand indexed views using and Data Modeling.
  • Experience in SAP BIBO Reporting (Crystal Reports, Web Intelligence), Data services and Rapid marts.
  • Developed, Configured and Customized Data Extractions from SAP R/3 and Legacy Systems. Created Transformation, DTP, Transfer and Update rules and Routines in transfer and update rules.
  • Experience in integration of SAP BI system with Business Objects.
  • Experience with all the phases Project preparation, requirement gathering and analysis and worked with SAP HR cubes and reporting.
  • Deployed MapR 15-node cluster onAWSand integrated Cassandra with it.
  • Migrated Cassandra, Hadoop cluster onAWSand defined different read/write strategies for geographies.
  • Demonstrated leadership abilities and teamwork skills as well as the ability to accomplish tasks under minimal direction and supervision.

Skills

TECHNICAL SKILLS

Operating Systems: Windows Vista/XP/NT/2000/98/95, Windows server 2003, Unix, Linux, Mac OS, Ubuntu.

Big Data Technologies: HDFS, Hive, MapReduce, Pig, HbaseSqoop, Flume, Spark, ZookeeperImpala, Maven,Yarn, Kafka, Storm.

Databases: Oracle 10g & 11g, MYSQL, SQL Server 2008/2005, Redshift.

Languages: Java, JavaScript, CSS, Scala, XML, SQL, PL/SQL, C, C++, Python.

Java Technologies and Frameworks: Struts Framework, Spring, Hibernate, J2EE,, JDBC, Multi-threading, JSP, Servlets, JSF, SOAP, XML, XSLT, JSON, MessagePack and DTD

NoSQL Database: Hbase, MongoDB, Cassandra, DynamoDB

ERP: SAP R/3 4.7/ 4.6C.

Office Tools: MS-OFFICE - Excel, Word, PowerPoint, Access.

Cloud Technologies: AWS (EMR, EC2, EBS)

PROFESSIONAL EXPERIENCE

Confidential, CA

Hadoop Developer

Environment: Hadoop, Hive, Yarn, Spark, Hbase, Zookeeper, Sqoop, Flume, Oozie, HDFS, MapReduce, Java, Eclipse, Maven, Pig, Storm, Ruby, Unix, Sql, Hortonworks Ambari, MySQL

Responsibilities:

  • Implemented solutions utilizing Advanced Big Data/Hadoop Distribution Frameworks: MapReduce, HBase, Zookeeper, Yarn, Hive, Spark, Pig, Oozie, Flume, Sqoop, Storm, Nagios, etc on Hortonworks Environment.
  • Performed Data Ingestion, Batch Processing, Data Extraction, Transformation, Loading and Real Time Streaming using Hadoop Frameworks.
  • Performed enhancement to the existing application and create new applications.
  • Written complex HiveQL Queries and Pig Latin Scripts for retrieving data from Hadoop in required format.
  • Worked with Spark, improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.
  • Involved in developing and debugging Scala.
  • Prepared and maintained Java methods(UDF, UDAF) and Scala Classes wherever necessary for creating the MapReduce jobs using Pig latin and Java API.
  • Expert in importing and exporting terabytes of data into HDFS and Hive using Sqoop from other Traditional Relational Database Systems.
  • Deployed Hive and HBase integration to perform OLAP operations on HBase data.
  • Incremental data moments using sqoop on oozie workflow scheduler.
  • Used Flume in Loading log data into HDFS.
  • Imported Bulk Data into HBase using HiveQL and MapReduce programs.
  • Import the data from different sources like HDFS/Hbase into Spark RDD.
  • Processed the flat files using Pig and load them into Hive.
  • Written pig scripts for moving the data from Local file system to Hadoop file system.
  • Managed the Hive database which involves ingest and index of data.
  • Performed Hive partitioning, bucketing and complex joins on Hive tables and implementing Hive serdes.
  • Developed Hive and Impala scripts on Avro and Parquet file formats.
  • Written custom UDfs programs to extract data from HBase for Statistical Analysis.
  • Performed analytics on Time Series Data exists in HBase using HBase API.
  • Involved in Performance Tuning for optimizing the jobs in Hive, Pig and Hbase.
  • Exported the data from Avro files and indexing the documents in sequence file format.
  • Used Oozie scheduler to automate the pipeline workflow and orchestrate the sqoop, hive and pig jobs that extract the data on a timely manner.
  • Customized and developed Oozie workflow engine to run complete flow of self-contained model.
  • Monitoring clusters using Nagios to send timely email for the alerts.
  • Monitored the status of the Hadoop HDFS periodically and performed root-cause analysis on the issues identified.
  • Developed Unit test cases and automated the scripts.
  • Involved in bringing up the Data and Name Nodes, whenever they are down.
  • Maintaining different cluster security settings and involving in creation and termination of multiple cluster environment.
  • Worked in Agile/Scrum environment this uses Jira to maintain the story points and Kanban model.

Confidential

Hadoop/ Java Developer

Environment: Hadoop, HDFS, MapReduce, Pig, Hive, Sqoop, Kafka, Solr, Impala, Cassandra, HBase, Oozie, Flume, java, SQL Scripting, Linux Shell Scripting, CDH 5

Responsibilities:

  • Processed data into HDFS by developing solutions, analyzed the data using MapReduce, Pig, Hive and produce summary results from Hadoop to downstream systems.
  • Worked with Hadoop stack (HDFS, Yarn, Pig, MapReduce, Hive, Hbase, Flume).
  • Installed, configured Hadoop, MapReduce, HDFS, AWS and developed multiple MapReduce jobs in Java(UDFs), Pig and Hive for data cleaning and pre-processing.
  • Expertise in Unix/Linux Shell Scripting.
  • Involved in designing and developing of Data-Centric Solutions for Clients Requirements.
  • Worked in Agile/Scrum methodologies and stored procedures.
  • Worked with the Production Environment on AWS, high availability practices and deploying backup/restore infrastructure.
  • Developed SOAP and RESTful service framework for data warehouse team to extract master reference data(Post information, operating hours, and mission information).
  • Performed Hbase modeling/development supporting customer behavior/life cycle analysis.
  • Involved in Design and Deployment of Storm cluster integration with Kafka and HBase.
  • Expert in Oozie and workflow scheduler to manage hadoop jobs by Direct Acyclic Graph (DAG) of actions with control flows.
  • Used JUnit, EasyMock and MRUnit testing frameworks to develop Unit test cases.
  • Incrementally loaded data into Hbase database.
  • Involved in importing and exporting terabytes of data into HDFS and Hive using Sqoop from other Traditional Relational Database Systems.
  • Created and Implemented highly scalable and reliable distributed data design using NoSQL: HBase, Cassandra.
  • Performed Fine Tuning and Enhance Performance MapReduce jobs and tuning Hadoop performance with high availability and involved in recovery of Hadoop clusters.
  • Worked extensively with Data migration, Data cleansing, Data profiling, and ETL Processes features for data warehouses.
  • Worked on NoSQL databases like HBase, Cassandra, integrated with Hadoop.
  • Managed NoSQL database on large Hadoop distribution systems such as: Cassandra, Hbase.
  • Refactored Cassandra-access code to allow either Hector or Thrift access, replacing the original thrift code interspersed throughout the application.
  • Performed optimization for a Cassandra cluster
  • Performed data analytics using PIG, Impala, Hive, Language R for Data Scientists within the team to improve Business and future Strategy development.
  • Worked with Business Intelligence Data Visualization Tools with specialization on Tableau.
  • Expertise in different Data Modeling and Data Warehouse Design and Development.

Confidential

Sap BI Consultant

Environment: Business Objects XI R3.1 (sp2) Enterprise, Performance Manager, Dashboard Manager, XCELSIUS 2008, SAP BI 7.0, ECC 6.0 QAAWS, WEBI, SQL Server 2005,Oracle 10g, ActiveX, Microsoft Visio, Access 2000, Windows XP.

Responsibilities:

  • Involved in gathering, analyzing and documenting business requirements, functional requirements and data specifications for business objects universes and reports.
  • Designed universes using SAP BW InfoCubes.
  • Designed, developed and managed Universes in Business Objects Designer for generating reports from SAP BW.
  • Created and Configured Roles in BW and Imported the roles into BOE(Business Objects Enterprise system ).
  • Involved in Creating backup and Recovery Plan for BOE servers.
  • Involved in Creating Single Sign On Between Sap enterprise portal and BI and BO systems.
  • Involved in Installing SAP Integration kit for Business Objects 3.1 Enterprise.
  • Designed and developed Functional Analysis and Technical Design like Joins, Shortcut Joins, Contexts and Cardinalities of the tables.
  • Importing and Exporting the Universes to repository to make it accessible to end users.
  • Migrated universe in different environments like Development to Production.
  • Created reports in Business Objects using functionalities such as Master/Detail, Slice and Dice, CrossTab, Drill Down, Alerters, Breaks, Sorting, Ranking and Charts.
  • Developed critical WebI reports like drill down, hyperlinks, Slice and Dice, master/detail for analysis of parts benefits and headcount.
  • Performed in depth analysis of data and prepared weekly, monthly and quarterly reports.
  • Reports are made user friendly by implementing filters, conditions, calculations and cascading prompts.
  • Using the Powerful business objects functionalities like functions, Variables, user response and formulae.
  • Involved in Designing AD Hoc reports and Canned Reports.
  • Developed Dashboards and metrics for finance users using Dashboard builder.
  • Created Dash boards for the top level Management using SAP BW data.
  • Involved in designing Aggregate Universe for all the Dashboards.
  • Linked different WebI Reports from dashboard to Infoview.
  • Exclusively Used QAAWS, Live Office, XML mappings as per the Dashboard requirements.
  • Developed web services using Query as a Web Service Wizard (QAAWS) and deployed it to the server for developing Xcelsius dashboards.
  • Performed QAAWS Sizing on the servers to perform the Queries run fast and display the dashboards.
  • Created multiple Flash variables in Xcelsius and used them to pass data from one dashboard to other.
  • Involved in Setting up the XML mapping files to pick up data from ODS on a daily basis and produce KPI results in the dashboard.
  • Developed On Demand WebI Reports which reads parameters from Crystal Xcelsius.
  • Developed Dashboard reports based on Geography for upper level management using Crystal Xcelsius and posted onto Finance Portal.
  • Preparing test cases for the created Webi and dashboard reports that incorporate application business rules for QA team to perform testing before rolling to production.
  • Involved in Training Portal Administrators about Xcelsius Model and How to Configure Xcelsius with different Methods to connect to universe (Using Live Office, QAAWS, XML, WSDL).
  • Involved in writing the High-level and Detail-level documentation of the project.
  • Worked in production support, User Training and also UAT.

Confidential

Sap BI Consultant

Environment: Business Objects XI 3.0 Enterprise, Dashboard Manager, Performance Manager ECC 6.0, SQL Server 2005, Crystal Xcelsius 2008, WEBI, Info Burst, DESKI, BO SDK, QAAWS, Live Office, Access 2000, Windows XP.

Responsibilities:

  • Involved in functional requirements gathering meeting different Business Groups and preparing the Technical Specifications.
  • Involved in Developing user friendly Universes using Business Objects Designer.
  • UsingJavaSDKpackages adding, modifying user account and managing the group membership.
  • Created links in BO reports to link summary and detail reports using open document syntax.
  • Designed and developed Web Intelligence Infoview interface according to the Client requirement by using Web Intelligence SDK, JSP, JAVASCRIPT and HTML.
  • Created tables, Table indexes and Wrote Stored Procedures in SQL Server 2005 and Creating Views as per the requirements.
  • Complex reports were developed using Web Intelligence and complex full client Business Objects.
  • Created the reports using Universes as the main data provider.
  • Created the reports using Business Objects functionalities like Queries, Slice and Dice, Drill Down, Cross Tab, Master Detail and Formulae etc.
  • Organized data in the report Inserting Filters, user prompts, conditions, Sorting, Ranking and highlighting data.
  • Extensively worked on drill mode to view data in different Perspectives.
  • Used the formatting options like crosstab representation, section breaks, data sort options, calculations, font, colors etc.
  • Created reports in table, cross table, 2-D, 3-D Column, Line, Area and Pie charts.
  • Created different scorecards and graphs using set analyzer in performance manager.
  • Used InfoBurst for scheduling, refreshed, logical bursting and then distributing and publishing reports to servers and users anywhere.
  • Involved in gathering Dashboard requirements from Higher Level Executives.
  • Worked extensively with Crystal Xcelsius 2008 to display summary-level business data in a rich and intuitive format.
  • Designed a data model and Universe to enable the write back and retrieve functionality using Xcelsius and Web Services.
  • Formatted the placement and layout of components using align and spacing features.
  • Developed web services using Query as a Web Service Wizard (QAAWS) and deployed it to the server for developing Xcelsius dashboards. .
  • Created multiple Flash variables in Xcelsius and used them to pass data from one dashboard to other.
  • Developed multiple dashboards so that the user can navigate from one dashboard to the other.
  • Involved in writing the High-level and Detail-level documentation of the project.

Confidential

Application Integrations Developer

Environment: Java, Java-J2EE, Struts, JavaBeans, JSP, Servlets, EJB, XML, MVC, Oracle 9i, Eclipse, IBM Secure way Directory Server (LDAP).

Responsibilities:

  • Responsible for Design of Payment Module and BUA Approval Module.
  • Responsible for User Interface designing using Action Forms and Strut taglibs.
  • Developed Action Classes and Controller Servlet.
  • Involved in Coding of various server side component for modules.
  • Design and Coding of Utility and Error framework.
  • Performed Design, Development and Testing of the Application.
  • Developed various Use Case Scenarios for Design and Testing.
  • Designed the Application Framework using UML.
  • Designed and developed the entire database for the application in SQL Server.
  • Developed the business layer of the application through EJBs.
  • Developed GUI layer using JSP and Struts.
  • Designed UI screens using JSP, Custom Tags, Struts tags and HTML.
  • Used JUnit framework for Unit testing of application.

Confidential

Integrations Developer

Environment: Rational Rose, EJB2.0, Struts 1.1, JSP 1.2, Servlets 2.3, JDBC, JavaScript, CSS, UML, HTML, JNDI, JMS, Log4J, JUnit, Tomcat Server, Eclipse, Linux, Windows 2000.

Responsibilities:

  • Involved in various phases of Software Development Life Cycle(SDLC) module.
  • Designed UI screens using JSP, Custom Tags, Struts tags and HTML.
  • Used JavaScript for client side validation.
  • Used Struts Framework in the application which is based on MVC2 design pattern.
  • Used EJBs in the application and developed Session beans to house business logic at the middle tier level and Entity Beans for persistence.
  • Full life cycle experience in development methodologies like Agile and RUP.
  • Actively involved in designing and implementing Session Facade, Service Locator, Data Access Objects, Singleton and Data Transfer Object design patterns.
  • Consumed a Web Services using WSDL and SOAP.
  • Used Liferay to convert text to HTML to be presented.
  • Developed the application on Eclipse and deployed it on Tomcat Server.
  • Developed SQL Queries to query the database to test the back end process of the application.
  • Used JUnit framework for Unit testing of application.
  • Used Log4J to create log files to debug as well as trace application.

Confidential

Junior Application Developer

Environment Oracle, Java, SQL Server, SQL, PL/SQL, Toad, IBM AIX, Web based Testing, MS Access, UNIX, JDK 1.5, ANT.

Responsibilities:

  • Involved in design, Development and Support of the application.
  • Used Oracle Collections, Cursors for developing PL/SQL subprograms(Packages, Procedures, and Functions).
  • Extensively worked on debugging Packages, Procedures, Functions and Triggers.
  • Actively involved in Review meetings and walkthroughs.
  • Designed and developed UNIX shell scripts to automate the process of loading, pulling the data.
  • Assisted in generating Post Deployment Scripts and UAT scripts.
  • Queried data from different database tables as per the requirement by writing complex SQL queries for data validation.
  • Extensively used UNIX commands for fetching and checking the Log files.
  • Worked as a peer code reviewer to check the design, vulnerability and scalability of the code.
  • For building and deploying project, ANT was extensively used.
  • Used front-end technologies like HTML for developing GUI.

We'd love your feedback!