SUMMARY:

• 8 Years and 6 Month of experience in Software Development, Automation testing and Maintenance activities.
• Recent experience includes developing reports, data cleansing to derive visualizations like pi - chart, bar chart, stacked bar chart, table, table map and word-count
• Experience in generating reports using PowerBI.
• Experience in NiFi workflow provided by HDP, PySpark, Hive, Sqoop, Oozie and Java.
• Exposure of intermediate knowledge of programming language ‘Python’.
• Exposure of Integration of different service such as Hive, OOZIE, SQOOP and Atlas as part of Hadoop Infrastructure.
• Good knowledge of data warehouse tool like DataStage.
• Exposure on Core Java.
• Experience around onsite and offshore coordination.
• Good result oriented, customer-focused interpersonal skills.
• Quest and zeal to learn modern technologies.
• Strong technical background and ability to learn modern technologies.

TECHNICAL SKILLS:

Programming Languages: Java1.7

JAVA Technologies: JDBC

Big Data Technologies: Hadoop (Apache Hadoop 2.2 and Cloudera)

Map: Reduce, HDFS, Hive, SQOOP, OOZIE

Server: Tomcat 7

Databases: MySQL, DB2

IDE Tools: Eclipse IDE

Operating System: Linux (Ubuntu), WINDOW.

PROFESSIONAL EXPERIENCE:

Confidential

Technical Lead

Responsibilities:

• During this tenure, parallel to below project, for almost 2-3 weeks I was involved for this POC and worked as a developer.
• I was responsible to collect the data, make it useful after cleansing.
• Using various functions provided by IBM cloud pak and use them to develop custom aggregated fields.
• Generating various kind of report using clean data like pi-chart, bar-chart, stacked-bar chart, Table, Tree-Map, Word Count and many more.
• This was a POC completed in short period of time and appreciated by Tavant as well as IBM senior management.

Confidential

Environment: ReactJS, Node, AWS, Jenkins, Elastic-Search, Java, Selenium

Developer

Responsibilities:

• Requirement gathering on daily basis.
• Setting up the complete system locally and sync-up with devops team to do the setup, as the project was divided into microservices like schemalookup, kraken and taskhandler and these microservices were maintained by docker.
• Was mainly responsible for writing code using react and redux to make changes in the UI as per the required implementation.
• Contributed towards backend development using node and other microservices.
• Sometimes worked for build activities via Jenkins.
• Writing automation scripts using selenium to test the application.

Technical Lead

Confidential

Environment: Hortonworks (HDP), NiFi, PySpark, Hive, Unix, HDFS, Sqoop, Oozie.

Responsibilities:

• Gathering client requirements on weekly basis.
• Was responsible in preparing of planning document, where we were maintaining naming standard for in and out of the project flow.
• As a data engineer, I was responsible for understanding data, cleanup activities, transform via PySpark and Spark-SQL and finally load data into a partitioned table into hive.
• Worked on documentation part to maintain source detail, desired format, business owner’s schema and table details.
• We were using various third-party python modules like pandas, pysftp, zipfile, smtplib and many more to bring data from ftp server, applying some transformations and cleanup activity over that, maintaining backup in HDFS location and finally loading of processed data into partitioned hive table or EDL. Also sending email notification in case of successful completion of the job.

Confidential

Technical Lead

Environment: Hortonworks (HDP), NiFi, Hive, Unix, HDFS, Java and NIFI REST APIs

Responsibilities:

• Gathering client requirements on daily basis.
• Worked as a data engineer in this project, So I was responsible for data validation, if data from Datorama is ingesting correctly in our EDL.
• Responsible for analyzing existing flows and providing suggestions how we can modify the flows to work them as expected.
• I worked for various NIFI flows to get most accurate data in our EDL and scheduled the flows on daily basis, which can be helpful to retrieve latest data from Datorama database.
• Worked on Sqoop jobs to retrieve data from data warehouse and scheduled the jobs using Oozie workflow.
• Responsible for writing hive DDL statements based on the API query received from the client.
• Worked on custom NiFi processor creation using Java Technology as it was not being possible to get data from EXCEL file. But in recent version it is available.

Confidential

Technical Lead

Environment: Hortonworks (HDP), NiFi, Hive, Unix, HDFS, DataStage, Oracle

Responsibilities:

• Involved in development of complete life cycle of this Project.
• Responsible for analyzing client requirement, based on that creating NiFi flows to ingest data from various source system.
• Responsible for writing DataStage jobs to get transactional data from Salesforce and storing them into files. On the top of these files external hive table was used to access the data for further analysis. Oracle database was using intermediately as staging area.
• Responsible for applying logic for validating data before ingesting and creating data lake.
• Responsible for deriving tables for salesforce after applying intermediate calculations which can be used to visualize to the end users.
• Responsible for writing complex Hive queries.
• Coordinated and deployed bi-weekly releases to the production environment and was responsible for production monitoring of services within team’s domain of ownership.
• I was involved in deriving reports using PowerBI based on the client requirement.
• Involved in development of complete life cycle of this Project.
• Responsible for analyzing existing ETL jobs to identify the required transformation and data.
• Responsible for analyzing source system and target system to identify the adapter to be used/developed for data ingestion and publish.
• Responsible for writing Map-Reduce jobs.
• Responsible for writing Hive queries

SYSTEM ENGINEER

Confidential

Environment: Hadoop, Map-reduce-2.0, Hive, OOZIE

Responsibilities:

• Using Map-reduce written dimension extraction for language, device and country.
• Using Hive queries on generated CSV file.
• Responsible for writing workflow of jobs using OOZIE.

Confidential

SYSTEM ENGINEER

Environment: Hadoop, Map-reduce-2.0, Hive, OOZIE, SQOOP, Facebook Graph APIFacebook4j API

Responsibilities:

• Involved in development of complete life cycle of this project.
• Responsible for creating Hadoop Infrastructure and Integration of different service as a part of Hadoop Infrastructure.
• Responsible for fetching Data from Facebook using Graph and Facebook4j API.
• Responsible for Parsing and arranging data from JSON format to required Formats.
• Responsible for writing Map-Reduce jobs to identify the Top-Trends.
• Responsible for writing Hive Queries to Calculate Response Rate.
• Responsible for writing workflow of jobs with OOZIE.
• Involved in server/ presentation side coding.
• Based on application design specification document understanding of business requirement.
• Handling of HPSM tickets for small issues and SRT tickets for code enhancement.
• Responsible for analysis, design, coding, unit testing and raising of request to promote code into production.
• Maintaining document in shared drive for each phase of code development like analysis, design, coding also tests results.
• Providing issue resolution and client support.