SUMMARY

• Analysis, Data governance, Data integration and Data quality.
• Experience in setting, configuring and monitoring of Hadoop cluster of Cloudera, Hortonworks distribution.
• Deep and extensive knowledge with HDFS, Spark, Apache Nifi, MapReduce, Pig, Hive, HBase, Sqoop, Storm, Yarn, Flume, Oozie, Zookeeper, Cassandra, MongoDBetc.
• Thorough knowledge on Hadoop architecture and various components such as HDFS, Name Node, Data Node, Application Master, Resource Manager, Node Manager, Job Tracker, Task Tracker and MapReduce programming paradigm.
• Good understanding on Hadoop MR1 and MR2 (YARN) Architecture.
• Experience in analyzing data using HIVEQL, PIG Latin and Map Reduce programs in JAVA.
• Expertise in writing Map Reduce Programs and UDFs for both HIVE and PIG in JAVA. Extended HIVE and PIG core functionality by using custom UDF's.
• Experience in developing scalable solutions using NoSQL databases including HBASE, CASSANDRA, MongoDB and Couch DB.
• Extracted files from NoSQL database like Couch DB, HBase through Flume and placed in HDFS for processing.
• Efficient in working with Hive data warehouse tool creating tables, data distributing by implementing Partitioning and Bucketing strategy, writing and optimizing the HiveQL queries.
• Experienced in performing analytics on structured data using Hive queries, operations, Joins, tuning queries, SerDe's and UDF.
• Good experience working with different Hadoop file formats like Sequence File, RCFile, ORC, AVRO and Parquet.
• Experience in using modern Big - Data tools like SparkSQL to convert schema-less data into more structured files for further analysis. Experience in Spark Streaming to receive real time data and store the stream data into HDFS.
• Experienced in building Storm topologies, spouts, boults to stream data from sources, pre-process data.
• Extensive experienced in working with different Spark modules like Spark transformations, Mlib, Graphx, Streaming and Spark QL.
• Good experience in writing Map Reduce jobs using Java native code, Pig, Hive for various business use cases.
• Experience in processing data serialization formats like Xml, JSON and Sequence Files.
• Experience in working with Apache Sqoop to import and export data to and from HDFS and Hive.
• Good working experience in designing Oozie workflows for cleaning data and storing into Hive tables for quick analysis.
• Implemented JavaMailAPIs to send e-mails.
• Expertise in middle tier technologies like JMS (Point-to-Point & Publish and Subscribe).
• Proficient in programming with Java/J2EE and strong experience in technologies such as JSP Servlets, Struts, Spring (IOC&AOP), Hibernate, EJBs, MDBs, Session Beans, JDBC and JNDI.
• Expertise in developing both frontend and backend applications using Java, Servlets, JSP, JJDBC, XSLT, Web Services,ExtJS 4.0,Angular
• Strong experience in developing persistence layers using JDBC, Hibernate, entity beans.
• Knowledge on SOA that includes webservices and involves WSDL, SOAP, and XML.
• Expertise in analyzing external client and target Systems involved Enterprise Architecture Integration.
• Implemented Spring boot.
• Expertise in designing enterprise applications using EJB 2.0, Struts (MVC model 2) and Spring MVC framework, Hibernate.
• Experience in publishing and consuming Test Driven and Test First based unit testing frameworks using JUnit, Selenium Remote Control API, Selenium IDE, Http Unit, DBUnit, etc…
• Experience in Planning, Installation and Administration of IBM MQSeries, MQSeries Integrator on various platforms like UNIX and Windows NT/2000.
• Expertise in Object Oriented Programming, Client/Server Applications, Relational Databases and E-Business Applications

TECHNICAL SKILLS

Languages: Hadoop (Sqoop, Hive, Pyspark, Oozie, MapR, Cloudera Impala, Spark), Shell Programming, Core Java,Spring 2.0,Hibernate,Extjs,Angular 2.0

Office Tools: MS Word, MS Excel, MS Power Point, MS Visio

IDEs: Eclipse, PyCharm

Operating Systems: Win 98/ XP/ 7, UNIX (Solaris 9 / 10, HP UX)

Bug Reporting Tools: VersionOne, JIRA

Job Scheduling: Autosys

Configuration Management: Subversion (SVN) & GitHub

PROFESSIONAL EXPERIENCE

Confidential

Hadoop Developer

Responsibilities:

• Core Hadoop Development Team Member involved in designing the end-to-end solution for Data and Transformations.
• Providing design inputs and technical suggestions for any new integration work.
• Discussing New or Enhancement requirements with the Business Analysts / Business Users.
• Actively involved in Development and Release activities.
• Developed various processes (using PySPARK, Shell Script etc.) to automate data movement within and outside of Hadoop.
• Designed the End-to-End CDC (Change Data Capture) flow that resembles SCD on Hadoop
• Preparation of the necessary documents for releases
• Production deployment support for the code and debug any release issues
• Production support of code for any issues
• Analyzing the Existing data model and modify the data model to implement the New/Enhancement business requests.
• Code review and Unit Testing of the new development.
• Maintenance and support of the existing source codes and data.
• As a Module Lead, ensuring the delivery of the requirements on time and reporting any issues / blockers to the higher management.

Confidential

Full Stack Developer

Responsibilities:

• Daily programming in server and client side by using AngularJS, Java, J2EE, Node.JS and ExpressJS.
• Work on Front end by using Angular JS, and JavaScript Frameworks.
• Extensive use of AngularJS for developing the SPAs (Single Page Applications) for internal application/tool.
• Develop multiple POCs to create a user interface in AngularJS.
• Use various features in AngularJS (2.0/4.0) like data binding, routing, custom filters, custom directives to meet various customized requirements.
• Involve in developing applications using AngularJS (2.0/4.0) like Grids, Toolbars, Panels, Combo-Box and Button.
• Work on AngularJS directives, Controllers, Filters, Services, Templates, Events and Injectors.
• Extensive knowledge of and use of AngularJS directives to bring additional functionality to HTML.
• Good knowledge in AngularJS and Responsive Web Design.
• Create Angular2 components, implement Interpolation, Input variables, Bootstrapping, ng-for, ng-If, Router Outlet, binding the click event, Component decorator, binding to the hidden property.
• Responsible for design and development of the web pages from mock- ups (PSD).
• Use Node JS, AngularJS and MVC Frameworks in the development of the web applications.
• Design and develop CSS behaviors, CSS styles, CSS fonts, CSS buttons, Pseudo Classes, Pseudo elements in CSS.
• Experience in Using SASS and LESS.
• Effective use of CSS3featurres, such as creating multiple column, borders and background etc. and used LESS/SASS Preprocessor.
• Develop Web API using Node.JS and hosted on multiple load balanced API instances.
• Work on Big data (Splunk enterprise) to generate the Rest API and integrated that API to Client-side code using Node JS, Express JS.
• Use Node.js as a proxy to interact with Rest services and interacting with Database.
• Work on Apache Camel Routing to have the workflow pass through different Enterprise Application.
• Work own Customized API’s built by Confidential .
• Involve in agile software methodologies using TEST DRIVEN DEVELOPMENT (TDD).

Confidential

Full Stack Developer

Responsibilities:

• Installed and configured Apache Hadoop clusters using YARN for application development and Apache toolkits like Apache Hive, Apache Pig, HBase, Apache Spark, Zookeeper, Flume, Kafka and Sqoop.
• Developed and deployed successfully many modules on Spark, Hive, Sqoop, Shell, Pig, Scala and Python.
• Successfully launched data transfer between Databases and HDFS with Sqoop, and used Flume in parallel to stream the log data from servers.
• Modified Hive and SQL queries to Spark using Spark RDDs and Scala, python.
• Designed and deployed multiple POCs using Scala and Yarn cluster, and checked the Performance of Spark, with Cassandra and SQL.
• Involved in data loading from UNIX file system to HDFS.
• Generated Sqoop scripts for data ingestion into Hadoop environment.
• Implemented Spark API over YARN to achieve data analytics in Hive DB.
• Created and scheduled multiple tasks for incremental load into staging tables.
• Loaded the log data and data from UI apps into Hadoop lake using Apache Kafka service
• Transformed data and performed data quality checks before loading onto HDFS with Pig.
• Created Hive External tables in partitioned format to load the processed data obtained from MapReduce.
• Operated analytical algorithms on HDFS data using MapReduce programs
• Merged data from different sources using Hive joins and performed Adhoc queries.
• Modified Hive and SQL queries to Spark using Spark RDDs and Scala, python.
• Developed and deployed successfully many modules on Spark, Hive, Sqoop, Shell, Pig, Scala and Python.
• Designed Java/J2EE web application using ExtJs and AngularJS UI Frameworks.
• Designed JUnit Testcases using spring frameworks and Mockito.
• Updated all Selenium Test framework scripts using Java with selenium 2.14 API.
• Worked on Activiti BPM2.0 Framework API which involves End to End approval.
• Effort Estimation, Impact Analysis for the change requests, Requirement Analysis and Design for the new features and enhancements.
• Reviewed the code, implemented the enhancements, fixed the issues, and tracked the tickets and updating the status in HP OpenView.
• Migrated current EXTJS4.0 project to Angular 1.3.
• Followed test driven approach in Angular using karma.
• Wrote unit tests using JUnit framework and Logging are done using Log4J.
• Handled bugs in supporting existing application.
• Designed, developed and maintained the data layer using the ORM framework called Hibernate.
• Wrote HQL from complex queries.
• Created design documents for major fixes and enhancements.
• Involved in Major Enhancements in TMS Application.
• Building and deployment of EAR, WAR, JAR files on test, stage and production systems in WebLogic Application Server.
• WebApps modules are developed using JSF.
• Extended and updated to REST API and created client API library.
• Used the Node.js, Grunt/Gulp, Jasmine, backbone.js frameworks in the development of the responsive web application.
• Implemented the Drools 5.5 (Business Rule Engine) for processing the workflows by decoupling it from the Java Code.
• Integrated Sonar for effective programming and Splunk to debug the production issues.
• Used JMSAPI for asynchronous communication by putting the messages in the Message queue, such as PDF, Excel report generation.
• Used Web Services to extract customer related product data from machines and servers using WSDL.