We provide IT Staff Augmentation Services!

Hadoop Developer Resume

SUMMARY

  • Analysis, Data governance, Data integration and Data quality.
  • Experience in setting, configuring and monitoring of Hadoop cluster of Cloudera, Hortonworks distribution.
  • Deep and extensive knowledge with HDFS, Spark, Apache Nifi, MapReduce, Pig, Hive, HBase, Sqoop, Storm, Yarn, Flume, Oozie, Zookeeper, Cassandra, MongoDBetc.
  • Thorough knowledge on Hadoop architecture and various components such as HDFS, Name Node, Data Node, Application Master, Resource Manager, Node Manager, Job Tracker, Task Tracker and MapReduce programming paradigm.
  • Good understanding on Hadoop MR1 and MR2 (YARN) Architecture.
  • Experience in analyzing data using HIVEQL, PIG Latin and Map Reduce programs in JAVA.
  • Expertise in writing Map Reduce Programs and UDFs for both HIVE and PIG in JAVA. Extended HIVE and PIG core functionality by using custom UDF's.
  • Experience in developing scalable solutions using NoSQL databases including HBASE, CASSANDRA, MongoDB and Couch DB.
  • Extracted files from NoSQL database like Couch DB, HBase through Flume and placed in HDFS for processing.
  • Efficient in working with Hive data warehouse tool creating tables, data distributing by implementing Partitioning and Bucketing strategy, writing and optimizing the HiveQL queries.
  • Experienced in performing analytics on structured data using Hive queries, operations, Joins, tuning queries, SerDe's and UDF.
  • Good experience working with different Hadoop file formats like Sequence File, RCFile, ORC, AVRO and Parquet.
  • Experience in using modern Big - Data tools like SparkSQL to convert schema-less data into more structured files for further analysis. Experience in Spark Streaming to receive real time data and store the stream data into HDFS.
  • Experienced in building Storm topologies, spouts, boults to stream data from sources, pre-process data.
  • Extensive experienced in working with different Spark modules like Spark transformations, Mlib, Graphx, Streaming and Spark QL.
  • Good experience in writing Map Reduce jobs using Java native code, Pig, Hive for various business use cases.
  • Experience in processing data serialization formats like Xml, JSON and Sequence Files.
  • Experience in working with Apache Sqoop to import and export data to and from HDFS and Hive.
  • Good working experience in designing Oozie workflows for cleaning data and storing into Hive tables for quick analysis.
  • Implemented JavaMailAPIs to send e-mails.
  • Expertise in middle tier technologies like JMS (Point-to-Point & Publish and Subscribe).
  • Proficient in programming with Java/J2EE and strong experience in technologies such as JSP Servlets, Struts, Spring (IOC&AOP), Hibernate, EJBs, MDBs, Session Beans, JDBC and JNDI.
  • Expertise in developing both frontend and backend applications using Java, Servlets, JSP, JJDBC, XSLT, Web Services,ExtJS 4.0,Angular
  • Strong experience in developing persistence layers using JDBC, Hibernate, entity beans.
  • Knowledge on SOA that includes webservices and involves WSDL, SOAP, and XML.
  • Expertise in analyzing external client and target Systems involved Enterprise Architecture Integration.
  • Implemented Spring boot.
  • Expertise in designing enterprise applications using EJB 2.0, Struts (MVC model 2) and Spring MVC framework, Hibernate.
  • Experience in publishing and consuming Test Driven and Test First based unit testing frameworks using JUnit, Selenium Remote Control API, Selenium IDE, Http Unit, DBUnit, etc…
  • Experience in Planning, Installation and Administration of IBM MQSeries, MQSeries Integrator on various platforms like UNIX and Windows NT/2000.
  • Expertise in Object Oriented Programming, Client/Server Applications, Relational Databases and E-Business Applications

TECHNICAL SKILLS

Languages: Hadoop (Sqoop, Hive, Pyspark, Oozie, MapR, Cloudera Impala, Spark), Shell Programming, Core Java,Spring 2.0,Hibernate,Extjs,Angular 2.0

Office Tools: MS Word, MS Excel, MS Power Point, MS Visio

IDEs: Eclipse, PyCharm

Operating Systems: Win 98/ XP/ 7, UNIX (Solaris 9 / 10, HP UX)

Bug Reporting Tools: VersionOne, JIRA

Job Scheduling: Autosys

Configuration Management: Subversion (SVN) & GitHub

PROFESSIONAL EXPERIENCE

Confidential

Hadoop Developer

Responsibilities:

  • Core Hadoop Development Team Member involved in designing the end-to-end solution for Data and Transformations.
  • Providing design inputs and technical suggestions for any new integration work.
  • Discussing New or Enhancement requirements with the Business Analysts / Business Users.
  • Actively involved in Development and Release activities.
  • Developed various processes (using PySPARK, Shell Script etc.) to automate data movement within and outside of Hadoop.
  • Designed the End-to-End CDC (Change Data Capture) flow that resembles SCD on Hadoop
  • Preparation of the necessary documents for releases
  • Production deployment support for the code and debug any release issues
  • Production support of code for any issues
  • Analyzing the Existing data model and modify the data model to implement the New/Enhancement business requests.
  • Code review and Unit Testing of the new development.
  • Maintenance and support of the existing source codes and data.
  • As a Module Lead, ensuring the delivery of the requirements on time and reporting any issues / blockers to the higher management.

Confidential

Full Stack Developer

Responsibilities:

  • Daily programming in server and client side by using AngularJS, Java, J2EE, Node.JS and ExpressJS.
  • Work on Front end by using Angular JS, and JavaScript Frameworks.
  • Extensive use of AngularJS for developing the SPAs (Single Page Applications) for internal application/tool.
  • Develop multiple POCs to create a user interface in AngularJS.
  • Use various features in AngularJS (2.0/4.0) like data binding, routing, custom filters, custom directives to meet various customized requirements.
  • Involve in developing applications using AngularJS (2.0/4.0) like Grids, Toolbars, Panels, Combo-Box and Button.
  • Work on AngularJS directives, Controllers, Filters, Services, Templates, Events and Injectors.
  • Extensive knowledge of and use of AngularJS directives to bring additional functionality to HTML.
  • Good knowledge in AngularJS and Responsive Web Design.
  • Create Angular2 components, implement Interpolation, Input variables, Bootstrapping, ng-for, ng-If, Router Outlet, binding the click event, Component decorator, binding to the hidden property.
  • Responsible for design and development of the web pages from mock- ups (PSD).
  • Use Node JS, AngularJS and MVC Frameworks in the development of the web applications.
  • Design and develop CSS behaviors, CSS styles, CSS fonts, CSS buttons, Pseudo Classes, Pseudo elements in CSS.
  • Experience in Using SASS and LESS.
  • Effective use of CSS3featurres, such as creating multiple column, borders and background etc. and used LESS/SASS Preprocessor.
  • Develop Web API using Node.JS and hosted on multiple load balanced API instances.
  • Work on Big data (Splunk enterprise) to generate the Rest API and integrated that API to Client-side code using Node JS, Express JS.
  • Use Node.js as a proxy to interact with Rest services and interacting with Database.
  • Work on Apache Camel Routing to have the workflow pass through different Enterprise Application.
  • Work own Customized API’s built by Confidential .
  • Involve in agile software methodologies using TEST DRIVEN DEVELOPMENT (TDD).

Confidential

Full Stack Developer

Responsibilities:

  • Installed and configured Apache Hadoop clusters using YARN for application development and Apache toolkits like Apache Hive, Apache Pig, HBase, Apache Spark, Zookeeper, Flume, Kafka and Sqoop.
  • Developed and deployed successfully many modules on Spark, Hive, Sqoop, Shell, Pig, Scala and Python.
  • Successfully launched data transfer between Databases and HDFS with Sqoop, and used Flume in parallel to stream the log data from servers.
  • Modified Hive and SQL queries to Spark using Spark RDDs and Scala, python.
  • Designed and deployed multiple POCs using Scala and Yarn cluster, and checked the Performance of Spark, with Cassandra and SQL.
  • Involved in data loading from UNIX file system to HDFS.
  • Generated Sqoop scripts for data ingestion into Hadoop environment.
  • Implemented Spark API over YARN to achieve data analytics in Hive DB.
  • Created and scheduled multiple tasks for incremental load into staging tables.
  • Loaded the log data and data from UI apps into Hadoop lake using Apache Kafka service
  • Transformed data and performed data quality checks before loading onto HDFS with Pig.
  • Created Hive External tables in partitioned format to load the processed data obtained from MapReduce.
  • Operated analytical algorithms on HDFS data using MapReduce programs
  • Merged data from different sources using Hive joins and performed Adhoc queries.
  • Modified Hive and SQL queries to Spark using Spark RDDs and Scala, python.
  • Developed and deployed successfully many modules on Spark, Hive, Sqoop, Shell, Pig, Scala and Python.
  • Designed Java/J2EE web application using ExtJs and AngularJS UI Frameworks.
  • Designed JUnit Testcases using spring frameworks and Mockito.
  • Updated all Selenium Test framework scripts using Java with selenium 2.14 API.
  • Worked on Activiti BPM2.0 Framework API which involves End to End approval.
  • Effort Estimation, Impact Analysis for the change requests, Requirement Analysis and Design for the new features and enhancements.
  • Reviewed the code, implemented the enhancements, fixed the issues, and tracked the tickets and updating the status in HP OpenView.
  • Migrated current EXTJS4.0 project to Angular 1.3.
  • Followed test driven approach in Angular using karma.
  • Wrote unit tests using JUnit framework and Logging are done using Log4J.
  • Handled bugs in supporting existing application.
  • Designed, developed and maintained the data layer using the ORM framework called Hibernate.
  • Wrote HQL from complex queries.
  • Created design documents for major fixes and enhancements.
  • Involved in Major Enhancements in TMS Application.
  • Building and deployment of EAR, WAR, JAR files on test, stage and production systems in WebLogic Application Server.
  • WebApps modules are developed using JSF.
  • Extended and updated to REST API and created client API library.
  • Used the Node.js, Grunt/Gulp, Jasmine, backbone.js frameworks in the development of the responsive web application.
  • Implemented the Drools 5.5 (Business Rule Engine) for processing the workflows by decoupling it from the Java Code.
  • Integrated Sonar for effective programming and Splunk to debug the production issues.
  • Used JMSAPI for asynchronous communication by putting the messages in the Message queue, such as PDF, Excel report generation.
  • Used Web Services to extract customer related product data from machines and servers using WSDL.

Hire Now