SUMMARY

• Over 9.5 years of experience in Software development lifecycle - Software analysis, design, development, testing, deployment and maintenance which includes 3.4 years of professional work experience on Hadoop (Cloudera distribution CDH3, 4 and 5) applications.
• Hands on experience with the Hadoop,MapReduce, HDFS, Sqoop, Pig, Hive, HBase, Oozie, and Zookeeper.
• Well versed in configuring and administering the Hadoop Cluster using major Hadoop
• Distributions like Apache Hadoop and Cloudera
• Have hands on experience in writing Map Reduce jobs on Hadoop Ecosystem including Hive and Pig
• Worked with Oozie workflow engine to schedule time based jobs.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems/ NonRelational Database Systems and vice-versa.
• Analyzed large amounts of data sets by writing Pig scripts and Hive queries.
• Hands on experience in writing pig Latin scripts and pig commands.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Experience with Sequence files, AVRO and HAR file formats and compression.
• Experience working on NoSQL databases including Hbase & MongoDB.
• Experience using Sqoop to import data into HDFS from RDBMS and vice-versa.
• Extensive experience in Java and J2EE technologies like Servlets and JSP.
• Experience with front end technologies like HTML, CSS and Javascript.
• Experienced in java GUI/IDE Tools using Eclipse and NetBeans.
• Proficient in design and development of various dashboards, ad-hoc reports using OBIEE.
• Experience in database development using SQL and PL/SQL and experience working on databases like Oracle 9i/10g and SQL Server.
• Expertise in debugging and optimizing the performance of Oracle SQL queries.
• Effective team player and excellent communication skills with insight to determine priorities, schedule work and meet critical deadlines.

TECHNICAL SKILLS

Big Data: Hadoop, Map Reduce, HDFS, HBase, Hive, Pig, mahout, Sqoop, Ambari,Oozie, Cassandra,MongoDB,ZooKeeper

Java: Java, J2EE

Databases: Oracle 9i/11g,My SQL,SQL Server 2000/2005

DWH (Reporting): OBIEE 10.1.3.2.0/11 g

DWH (ETL): Informatica Power Center 9.6.x

Languages: SQL, PL/SQL,Java

UI: HTML, CSS, Javascript

Defect Tracking Tools: Quality Center, JIRA

Tools: SQL Tools, TOAD

Query Tools: TOAD, SQL Developer

Version Control: Tortoise SVN, GitHUB

Operating Systems: Windows ..., Linux/Unix

PROFESSIONAL EXPERIENCE

Confidential, NY

Senior Big Data Architect

Responsibilities:

• Responsible for building scalable distributed data solutions using Hadoop installed and configured Hive, Pig, Sqoop and Oozie on the Hadoop cluster
• Developed Simple to complex Map/Reduce Jobs using Hive and Pig
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
• Handled importing of data from various data sources, performed transformations using Hive, MapReduce,loaded data into HDFS and extracted the data from Oracle into HDFS using Sqoop
• Ingesting the data from different databases like SQL, Oracle into Hadoop Data Lake using Sqoop. Writing Pig and Hive scripts to process the HDFS data.
• Writing several Shell scripts for cleansing and preparing the data before it is moved to Hadoop Data Lake.
• Used Pig UDF's to implement business logic in Hadoop
• Installed Oozie workflow engine to run multiple Hive and Pig jobs
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team
• Involved Extensively used Pig for data cleansing.
• Created partitioned tables in Hive to improve the perfromanace.
• Managed and reviewed Hadoop log files.
• Involved in creating Hive tables, loading with data and writing hive queries which run internally in MapReduce.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Load and transform large sets of structured, semi structured and unstructured data
• Responsible to manage data coming from different sources
• Use of Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports.
• Worked on NoSQL databases including HBase, MongoDB and Cassandra.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.

Environment: Hadoop, MapReduce, HDFS, Hive, Pig,HBase, ZooKeeper,Mahout,Ambari, Sqoop, Java, SQL, Java (jdk1.7),Eclipse

Confidential

Hadoop Consultant

Responsibilities:

• Provide technical designs, architecture, Support automation, installation and configuration tasks and upgrades and planning system upgrades of Hadoop cluster
• Design development and architecture of the Hadoop cluster, map reduce processes, Hbase system
• Design and develop process framework and support data migration in Hadoop system
• Experience in the analysis, design, development and MR Unit testing of Hadoop Cluster Structure using Java
• Developed Map reduce program using Java and deployed the Jobs at the customer environment.
• Worked on several apaches Hadoop projects. Maps reduce programs were developed using Hadoop java API and also using hives and pig
• Worked with sqoop to import/export data from relational database to Hadoop and flume to collect data and populate in Hadoop
• Implemented and integrated Hadoop based business intelligence and Data Warehouse system including implementations of searching, filtering, indexing, aggregation for reporting and report generation and general information retrieval
• Maintained Hadoop clusters for dev/staging/production. Trained the development, administration, testing and analysis teams on Hadoop framework and Hadoop eco system
• Give extensive presentations about the Hadoop ecosystem, best practices, data architecture in Hadoop
• Integrating Bigdata technologies and analysis tools into the overall architecture

Environment: Hadoop, HDFS, MapReduce, Hive, Pig,HBase,ZooKeeper,Mahout,Ambari, Sqoop, Oozie,Cassandra,MongoDB

Confidential, NY

Hadoop Consultant

Responsibilities:

• Lead the AML Cards North America development and DQ team successfully to implement the compliance project.
• Involved in the project from POC and worked from data staging till saturation of DataMart and reporting. Worked in an onsite-offshore environment.
• Completely responsible for creating data model for storing & processing data and for generating & reporting alerts. This model is being implemented as standard across all regions as a global solution.
• Involved in discussions and guiding other region teams on SCB Big data platform and AML cards data model and strategy.
• Responsible for technical design and review of data dictionary (Business requirement).
• Responsible for providing technical solutions and work arounds.
• Migrating the needed data from Data warehouse and Product processors into HDFS using Sqoop and importing various formats of flat files in to HDFS.
• Involved in discussion with source systems for issues related to DQ in data.
• Implemented partitioning, dynamic partitions, buckets and Custom UDF's in HIVE.
• Used Hive to process data and Batch data filtering
• Supported and Monitored Map Reduce Programs running on the cluster.
• Monitored logs and responded accordingly to any warning or failure conditions.
• Responsible for preserving code and design integrity using SVN and SharePoint.

Environment: Apache Hadoop, HDFS, Hive, Map Reduce, Hive, Pig, HBase, Zookeeper, Ambari, Mahout, Oozie, Cassandra, MongoDB, Java, Sqoop, Cloudera CDH3 Platform, SVN

Confidential

Java/J2EE Developer

Responsibilities:

• Involved in various phases of Software Development Life Cycle (SDLC) as design development and unit testing.
• Developed and deployed UI layer logics of sites using JSP, XML, JavaScript, HTML/DHTML
• Designed different design specifications for application development that includes front-end, back-end using design patterns.
• Developed prototype test screens in HTML and JavaScript.
• Involved in developing JSP for client data presentation and, data validation on the client side with in the forms.
• Collection framework used to transfer objects between the different layers of the application.
• Developed data mapping to create a communication bridge between various application interfaces using XML, and XSL.
• Developed Junit testing framework for Unit level testing.
• Actively involved in code review and bug fixing for improving the performance.
• Documented application for its functionality and its enhanced features.
• Created connection through JDBC and used JDBC statements to call stored procedures.
• Created UML diagrams like use cases, class diagrams, interaction diagrams, and activity diagrams.
• Extensively worked on User Interface for few modules using JSPs
• Wrote complex SQL queries and stored procedures.
• Developed the XML Schema and Web services for the data maintenance and structures.
• Designed the logical and physical data model, generated DDL scripts, and wrote DML scripts for Oracle 10g database.
• Involved in creating templates and screens in HTML and JavaScript

Environment: Java, JSP, XML, CSS, HTML, JavaScript 1.2, Oracle 9i/10g.

Confidential

Oracle SQL,PL/SQL and OBIEE Consultant

Responsibilities:

• Imported new tables into physical layer of Oracle BI Admin Tool according to new business requirements.
• Modified existing Physical tables by importing new columns from Oracle database.
• Implemented left outer, right oute r and inner joins in Oracle Admin Tool.
• Modified existing BMM layer by mapping columns from Physical layer to BMM layer.
• Rearranged columns in Presentation layer to order and organize columns in analytics.
• Created logical columns by using Calculation wizard.
• Modified existing logical columns and created new logical columns using CASE Statements.
• Altered existing OBIEE reports as per the new requirements.
• Created new dashboard prompts in OBIEE.
• Modified existing filters in OBIEE as per new requirements of reports.
• Created new filters in OBIEE as per requirements of new reports.
• Altered existing dashboard prompts in OBIEE Analytics/Answers according to new business requirements.
• Assisted in migration of rpd from DEV/TEST environment to Productions environment.

Environment: Oracle 9i/11g, OBIEE 10.1.3.2, Linux/Unix