SUMMARY

• 10+ years of Professional experience in IT Software and Services in design, development and testing various applications in telecom and financial domains.
• Expertise in HDFS, MapReduce, YARN, Hive, HBase, Pig, Phoenix, Sqoop, Flume, Zookeeper, Oozie and various other ecosystem components with knowledge in Hadoop Cluster Setup, Integrations, and Installations.
• Good understanding on Business Intelligence, ETL Transformations and Hadoop Cluster.
• Experience in building data Ingestion, extraction, transformation for various datasets onto HDFS.
• Expertise in implementing HBase schemas with optimized Row - key design to avoid Hot-spotting.
• Experience in Populating HBase tables via phoenix DB to expose teh data for Spot-fire Analytics.
• Involved in designing Hive schemas, using performance tuning techniques like partitioning, bucketing.
• Optimized HiveQL by using execution engine like Tez.
• Performed Importing and exporting data into HDFS and Hive from DBMS using Sqoop.
• Experience in ETL Hive Transformation, HBase load via Phoenix to visualize data analytics.
• Analyzed teh data using Hive queries and running Pig scripts to study customer behavior.
• Developed Pig UDF'S to pre-process teh data for analysis.
• Used SFTP to transfer teh files to server.
• Experience in Talend Big Data Platform Studio, Implemented financial audit ETL transformations flows.
• Implemented Splunk Enterprise environment in HDP Cluster for log aggregation to analyze ecosystem issues.
• Implemented AppDynamics in HDP to understand teh cluster behavior for alerting and monitoring.
• Good understanding of Java Object Oriented Concepts and development of multi-tier enterprise web applications.
• Knowledge on cloud services like Azure and AWS E2 instances.
• Experience with Operating Systems like Windows, Linux, and Macintosh.
• Good Understanding in complete SDLC and STLC.
• Strong troubleshooting and production support skills and interaction abilities with end users.
• Have teh motivation to take independent responsibility as well as ability to contribute and be a productive team member.
• Self-motivated with an ardent desire to learn and an TEMPEffective Team Player.

TECHNICAL SKILLS

Hadoop Ecosystem: Hadoop 2.7.1, Map Reduce, YARN, HBase 1.1.2, Sqoop 1.4.6, Oozie 4.2.0, Hive 1.2.1, Pig 0.15.0, ZooKeeper 3.4.6, Splunk, AppDynamics, AWS E2, Talend Big Data Platform 6.2, MongoDB

Cluster: Hortonworks 2.4, Cloudera CDH5.7, IBM Big Insights 4.1

Languages: Core Java, C, JSP, EAI, Shell Script, R

Web Technologies: HTML5, CSS3, JavaScript, jQuery, XML, XHTML

Servers: Putty, WebSphere, WebLogic, JBoss, Apache Tomcat.

Database: MySQL, Oracle, PL-SQL, NO-SQL(HBase) IDE’s Eclipse Mars.1

PROFESSIONAL EXPERIENCE

Confidential, New York, NY

Big Data Architect

Responsibilities:

• Strategic technical interface with teh aligned business units for business technology strategy development, solution design & engineering, release management and relationship management.
• Data modeling using greenplum
• Engaged with cross functional teams to implement solutions and services as part of analytics platform
• Populated HDFS, Hadoop with data loaded from SQL production systems.
• Wrote Postgres scripts to extract data, create views, populate staging and populate teh data mart
• Marketing analytics data frames using pivotal R libraries
• Deficiency data analysis for R Libraries
• Designing and building large scale data loading, manipulation, processing, analysis, blending and exploration solutions using Hadoop/NoSQL technologies (e.g. HDFS, Hive, Sqoop, Flume, Spark, Kafka, HBase, Cassandra, MongoDB etc.
• Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.

Environment: Informatica 9.x, OS/390 Enterprise edition and server edition, DB2, Mainframe, DB2, Shell Scripts, Oracle, Green plum 4.2.3, Power center, Metadata Manager, Jaspersoft Reporting, Proactive Monitoring, Administrator, Web Services, Messaging Queues, Power exchange, hadoop, Vertica Analytics server, Mysql, Spark, kafka, Cassandra, Oracle 12c, Java, MS BI STACK, Postgres SQL

Mount Sinai Health Systems, Manhattan, New York

Big Data Architect

Responsibilities:

• Architecting business solutions for various projects within Mount Sinai.
• Ingest Clinical data of various client data into Azure Datalake HDFS.
• Implemented POC on Kyligence/Kylin - Analysed Dataset with in sub-seconds by building cubes on Kyligence (OLAP) on Azure HDP Platform.
• Implemented POC on MongoDB - Setup teh MongoDB Instance on Terminal Server. Built API connectivity by preparing JSON for Mulesoft consumption.
• Tuned teh Sqoop Jobs for TEMPeffective Automation in Pentaho.
• Import data from various sources to HDFS, Perform ETL Transformations using Hive based on client’s data.
• Transformation can be performed via Pentaho ETL. Oozie workflow defined for various stages of ETL.
• Writing UDF's in Hive for complex transformations in staging for Staging.
• Experience in loading raw files to HDFS and move teh data to hive database for analytics.
• Developing Pentaho packages using HQL and creating Hive Data warehousing in Hadoop.
• Used Hive to do analysis on teh data and identify different correlations.
• Tuning teh map reduce jobs in HDP and increase teh performance of teh job.
• Ongoing implementation of splunk to monitor teh machine critical information of customers.
• Used Tableau for visualizing clinical data and Patient 360 information.
• Timely Collaborating Hortonworks, Kyligence Enterprise Support Teams for any ongoing implementation issues.
• Reforming teh cluster into cloud - on going implementation of Azure Big Data Platform

Environment: HDP 2.5 (Cent OS)- Hadoop Eco System (HDFS, Map-Reduce, YARN, Sqoop, Hive, NoSQL (MongoDB and HBase), Phoenix, Oozie, Zookeeper, ETL-Pentaho, Mulesoft, Tableaue, Splunk, Kylin/Kyligence, DataIku, R-Programming and R Studio, Data Portal, DataWIKI and Machine Learning.

Confidential, Alpharetta, Georgia

Senior Hadoop Developer/Lead (ETL)

Responsibilities:

• Ingest SAP/Generic data of various client data into HDFS via EY- Helix UI Web application.
• Create clients, engagements, workspace and file-set for various clients for financial audit year to analyse meaningful business insights of GL, PP, and RR.
• Import data from various sources to HDFS, Perform ETL Transformations using Hive based on client’s data.
• SAP data transformation can be performed via Talend ETL. Generic data (Oracle, CSV etc.) transformations performed via Oozie workflow.
• Writing UDF's in Hive for complex transformations in staging for Staging.
• Experience in loading raw files to HDFS and move teh data to hive database for analytics.
• Developing Talend packages using HQL and creating Hive Data warehousing in Hadoop.
• Used Hive to do analysis on teh data and identify different correlations.
• Tuning teh map reduce jobs in HDP and increase teh performance of teh job.
• Parallelize teh sequential jobs in Talend and optimized teh query performance for SAP dataset.
• ETL data provision from various stages to CDM and RDM Transformation HBase
• Coordinate teh MTK central to upload teh various templates on Spot-fire to generate reports.
• Experience in Log management tool - Splunk. Provide splunk search queries to aggregate log events and analyse/monitor root causes of issues.
• Experience in AppDynamics - Application to monitor teh cluster environment, health and generate reports, alerts based on business requirements.
• Integrated App Dynamics and Splunk to monitor teh cluster to analyse events.
• Used Spot-fire for visualizing audit data and generate meaningful reports for clients.
• Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.
• Reforming teh cluster into cloud - on going implementation of Azure Big Data Platform

Environment: HDP 2.4 (Cent OS): HUEY (Helix User Interface), Map Reduce/YARN, Hive 1.2.1, HBase 1.1.2, Phoenix, Splunk, AppDynamics, Oozie 4.2.0, Ranger 0.6.0, Ambari 2.4.0, Spot-Fire, TFS (Team Foundation Server), Git, Db Visualizer. Talend Big Data platform 6.0

Hadoop Developer/Administration

Responsibilities:

• Worked on a live Big Data Hadoop production environment with 100 nodes.
• Loaded data into teh cluster from dynamically generated files using Flume and from relational database management systems using Sqoop.
• Worked with HBase databases for non-relational data storage and retrieval on enterprise use cases.
• Implemented POC Spark Cluster on AWS
• Involved in Analysing system failures, identifying root causes, and recommended course of actions.
• Imported logs from web servers with Flume to ingest teh data into HDFS.
• Retrieved data from HDFS into relational databases with Sqoop. Parsed cleansed and mined useful and meaningful data in HDFS using Map-Reduce for further analysis.
• Fine tuning hive jobs for optimized performance.
• Involved in implementing teh job workflows and scheduling for teh end to end application processing.
• Extending teh functionality of Hive and Pig with custom UDF’s and UDAF’s.
• Fine tuning Hive jobs for better performance.
• Used Flume to collect, aggregate and store teh web log data onto HDFS.
• Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems.
• Managed and reviewed Hadoop log files.
• Used Hive to do analysis on teh data and identify different correlations.
• Worked on importing and exporting data from Oracle and DB2 into HDFS using Sqoop.
• Automated all teh jobs, for pulling data from FTP server to load data into Hive tables, using Oozie coordinator.
• Utilized Agile Scrum Methodology to help manage and organize a team of 4 developers with regular code review sessions.
• Developed MapReduce/Spark modules for predictive analytics in Hadoop/Hive on AWS.
• Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.

Environment: HDP 2.3 (Cent OS) Apache Hadoop 2.7.1, HDFS, MapReduce/YARN HBase 1.1.2, Sqoop 1.4.6, Oozie 4.2.0, Hive 1.2.1, NoSQL, ETL, MySql, Teradata, AWS Amazon Web Services

Confidential, Fairfax, VA

Hadoop Developer (ETL)

Responsibilities:

• Configured Hadoop components including Hive, Pig, HBase, Sqoop, Oozie and Hue in teh client environment.
• Responsible to manage data coming from various sources and involved in HDFS maintenance and loading of structured and unstructured data.
• Developed MapReduce programs to parse teh raw data, populate staging tables and store teh refined data in partitioned tables in HDFS.
• Implemented knowledge of various java, J2EE and EAI patterns.
• Created Hive queries that helped market analysts spot emerging trends by comparing fresh data with HDFS reference tables and historical metrics.
• Enabled speedy reviews and first mover advantages by defining teh job flow in Oozie to automate data loading into teh Hadoop Distributed File System and PIG to pre-process teh data.
• Designed HBase schema to avoid Hotspotting and exposed teh data from HBase tables to REST API on UI.
• DevelopedPig scripts to transform raw datafrom several data sources into forming baseline data and loaded teh data into HBase tables.
• Involved in creating POCs to ingest and process streaming data using Spark and HDFS.
• Used Flume to collect, aggregate, and store teh log data from different web servers.
• Developed Shell Scripts to automate teh batch processing and processed teh daily jobs through Maestro scheduler.
• Provided design recommendations and thought leadership to sponsors/stakeholders thatimproved review processes and resolved technical problems.
• Co-ordinate with teh offshore team and cross-functional teams to ensure that applications are properly tested, configured, and deployed.
• Used Tableau for visualizing and to generate reports.

Environment: CDH 5.7.0(Cent OS): Apache Hadoop 2.7.1, MapReduce, HBase 1.1.2, Pig 0.15.0, Sqoop 1.4.6, Oozie 4.2.0, Java 8, Autosys, Hive 1.2.1, Impala, ZooKeeper 3.4.6, Oracle 11g, PL/SQL, SQL Developer 4.0, UNIX. Rest API, Web Services REST, SQL, ANT, Shell Script, JAVA, J2EE

Confidential

Technical Lead/Freelancer

Responsibilities:

• Designing and implementing new features and functionality
• Establishing and guiding teh website’s architecture
• Ensuring high-performance and availability and managing all technical aspects of teh CMS.
• Helping formulate an TEMPeffective, responsive design and turning it into a working theme and plugin.
• Network setup and set up applications as part of our service (Tomcat etc)
• Installing/configuring our 3rd party software
• Troubleshooting/incident investigation
• Monitoring and Capacity planning.
• Installing a configuring monitoring software to support others in their roles

Environment: Web development, Web Hosting, Configuration, WordPress/CMS, HTML, CSS, PHP, Google Analytics, Web deployment, Linux/Windows.

Confidential

Technical Lead

Responsibilities:

• Worked on SQOOP to import CDR data from various relational data sources to Big Data.
• Working with Flume in bringing click stream data from front facing application logs.
• Worked on strategizing SQOOP jobs to parallelize data loads from source systems.
• Participated in providing inputs for design of teh ingestion patterns.
• Participated in strategizing loads without impacting front facing applications.
• Worked on design on Hive data store to store teh data from various data sources.
• Developed MapReduce and MRUnit jobs to operate on streaming data.
• Involved in providing inputs to analyst team for functional testing.
• Worked with source system load testing teams to perform loads while ingestion jobs are in progress.
• Worked on performing data standardization using PIG scripts.
• Worked on building analytical data stores for data science team’s model development.
• Worked on design and development of Oozie works flows to perform orchestration of PIG and HIVE jobs.
• Worked on performance tuning of HIVE queries with partitioning and bucketing process.
• Worked with Ambari UI to configure alerts for Hadoop eco system components.
• Participated in tuning various components in Hadoop Eco System.

Environment: Hadoop 2.2 (IBM Big Insights) - Apache Sqoop 1.4.5, HDFS, Map Reduce, Hive 0.14.0, TEZ 0.5.2, PIG 0.14.0, Oozie 4.1.0, HBase 0.98.4, Ambari 2.0.0 JUnit, Zookeeper, maven, Hadoop Data Lake with Linux-Cent OS, Oracle 11g, SQL Developer 4.1.3, Unix/Shell Scripting.

Confidential

Application Developer

Responsibilities:

• Implemented CDR batch processing for MSC, IN, ADSL, and GPRS Streams to object-oriented Programming.
• Involved in teh review and analysis of teh Functional Specifications, and Requirements Clarification Defects etc.
• Involved in teh analysis and design of teh initiatives.
• Involved in teh development of teh mediation platform using Java.
• Involved in design and implementation of migration of Mediation legacy platform to mediation zone.
• Involved in writing Junit Test for Unit testing.
• Involved in Regression Testing of SAT and Development Environments.
• Involved in writing teh SQL queries and stored procedures.
• Participated in teh test case reviews, and manual testing of teh enhancements during Release 1.5.
• Involved in fixing teh defects during integration testing.
• Build and deployment of teh application using Ant script on to dev and testing environments.
• Participated in teh code reviews for various initiatives, Performed Static Code Analysis to follow teh Best Practices for Performance and Security.

Environment: Mediation Zone (Digital Route), Legacy Zone, Core Java 6

Confidential

Application Developer & Tester

Responsibilities:

• Analyzing teh High-Level Design (HLD) and Business Requirements.
• Providing daily updates to teh on-site team over call and making enhancements.
• Implemented SQL and PL/SQL scripts including Stored Procedures, functions, packages and triggers.
• Interpreting product requirements into test requirements, writing test plans and test cases.
• Made enhancements to teh application which presented me with teh opportunity to go through teh entire SDLC.
• Preparing teh Test Cases and Test Execution Plan.
• Code coverage and Test case presentation.
• Involved in Test Planning and Test case execution.
• Preparing TPI in teh test planning phase and teh execution phase.
• Raised defects in QC.
• Reviewing teh test plan document, so that there is no functionality or teh requirements missing.
• Used Web Services like SOAP and WSDL to communicate over internet.
• Automated teh tests using teh integration tool (SOAP-UI).

Environment: WebMethods, SOAP-UI, SEIBEL, CITRIX, Putty, IBM Rational - ClearQuest.