Hadoop Developer Resume
ChicagO
SUMMARY
- Overall 8 years of IT experiences in Data Warehousing (ETL), OLTP, OLAP, ODS, Requirements Gathering, Analysis, Design, Development, Implementation, Integration, Testing and Validation of data using the technology, Informatica PowerCenter for Investments in different Methodologies.
- Extensive business domain experience in Investments.
- Excellent experience in developing and maintaining complex large scale batch interface and ETL processes across transactional data and data warehousing system.
- Skilled to Develop, Test, Tune and Debug the informatica Mappings, Sessions and Monitor the system.
- Skilled to interact with business users. Pioneered in different load strategies from heterogeneous sources to target. Successfully implemented SCD Type1/Type2 load, Capture Data Changes to maintain the Data history using informatica.
- Experienced to identify the Bottlenecks of data load and tuned it for better performance.
- Excellent to write the Stored Procedures, Triggers, Indexes, Functions by PL/SQL, SQL Scripts.
- Excellent experience inproblem solvingandsupporting applicationscomes under24/5.
- Timely and accurate response and/or resolution of production problems; initiates and/or updates problem management tracking including and timely communication to support teams.
- Provide 'Root Cause' for production issues to management.
- In depth understanding/knowledge of Hadoop Architecture and its components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce.
- Expertise in writing Hadoop Jobs for analyzing data using Hive and Pig.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa.
- Experienced in extending Hive and Pig core functionality by writing custom UDFs.
- Experienced in NoSQL databases such as HBase.
- Experienced in job workflow scheduling and monitoring tools like Oozie and Zookeeper
- Experienced in Migration of codes from Repository to Repository, wrote up Technical/Functional Mapping specification Documents for each Mapping along with unit testing for future development.
TECHNICAL SKILLS
Databases: Oracle 10g, 11g, MS Access
GUI tool: SQL-Developer, TOAD, SQL*Plus, SQL Loader, Eclipse 3.0,Putty, WinSCP, Service Now, IBM Tivoli job scheduler
ETL Tools: Informatica Power Center 9.1,IDQ
Languages: SQL, PL/SQL, Unix Shell Scripting, Core Java
HADOOP/BIG DATA: HDFS, MapReduce, HBase, Pig, Hive, Sqoop, Oozie.
PROFESSIONAL EXPERIENCE
Hadoop Developer
Confidential, Chicago
Technologies: Hadoop, HDFS, Hive, Pig, Linux, XML, Eclipse, Cloudera, CDH3/4 Distribution, Informatica 9.1, Oracle 11i
Responsibilities:
- Gathered the business requirements from the Business Partners and Subject Matter Experts.
- Created Hive tables and working on them using Hive QL.
- Handled 2 TB of data volume and implemented the same in Production.
- Responsible to manage data coming from different sources.
- Supporting Hbase Architecture Design with the Hadoop Architect team to develop a Database Design in HDFS.
- Involved in HDFS maintenance and loading of structured and unstructured data.
- Wrote Hive queries for data analysis to meet the business requirements.
- Wrote PigLatin scripts.
- Developed UDFs for Pig Data Analysis.
- Developed Scripts and Batch Job to schedule various Hadoop Program.
- Worked hands on with ETL process.
- Handled importing of data from various data sources, performed transformations using Hive, Pig.
- Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
- Developed Hive queries to process the data and generate the data cubes for visualizing.
- Imported Source/Target tables from the respective Business Warehouse systems and created reusable transformations (Joiner, Routers, Lookups, Rank, Filter, Expression and Aggregator) inside a Mapplets and created new mappings using Designer module of Informatica Power Center to implement the business logic and to load the customer healthcare data incrementally and full.
- Created Complex mappings using Unconnected Lookup, and Aggregate and Router transformations for populating target table in efficient manner.
- Optimized the mappings using various optimization techniques and also debugged some existing mappings using the Debugger to test and fix the mappings.
- Update maps, sessions and workflows as a part of ETL change.
- Modifications to existing ETL Code and document the changes.
Informatica Developer
Confidential, Chicago
Technologies: Informatica PowerCenter 9.1.0, Oracle 11g, SQL, PL/SQL, Putty, WinSCP, UNIX Shell Scripting, SQL-Developer.
Responsibilities:
- Participated in daily/weekly meetings, monitored the work progresses of teams and proposed ETL strategies.
- Designed and developed various complex SCD Type1/Type2 mappings in different layers, migrated the codes from Dev to Test to Prod environment. Wrote down the techno-functional documentations along with different test cases to smooth transfer of project and to maintain SDLC.
- For each Mapping prepared effective Unit, Integration and System test cases for various stages to capture the data discrepancies/inaccuracies to ensure the successful execution of accurate data loading.
- Tested mappings, workflows and sessions to figure out the bottleneck to tune them for better performance. Prepared effect Unit, Integration and System test cases for various stages to capture the data discrepancies/ inaccuracies to ensure the successful execution of accurate data loading.
- Performed exception handling, reporting and monitoring the system. Created different rules as mapplets, workflows. Deployed the workflows as an application to run them. Tuned the mappings for better performance.
- Created Pre & Post-Sessions UNIX Scripts, Functions, Triggers and Stored Procedures to drop & re-create the indexes and to solve the complex calculations on data. Responsible to transform and load of large sets of structured, semi-structured and unstructured data from heterogeneous sources.
- Used Debugger to validate the Mappings and gained troubleshooting information about the data and error conditions. Involved in fixing the invalid Mappings. Wrote various Functions, Triggers and Stored Procedures to drop, re-create the indexes and to solve the complex calculations.
Informatica Developer
Confidential, Chicago
Technologies: Informatica PowerCenter 9.1.0, Oracle 11g, SQL, PL/SQL, Putty, WinSCP, UNIX Shell Scripting, TOAD.
Responsibilities:
- Participated in daily/weekly meetings, monitored the work progresses of teams and proposed ETL strategies.
- Involved in the development of Stored Procedures, Functions, Views, Materialized Views, and Triggers, to drop & re-create the indexes and to process business data according to requirements.
- Developed mapping using various transformation such as Lookups, Source Qualifier, Joiner Transformation, Expression, Filter, Router, Aggregator, Update Strategy, Rank, Sequence Generator and other transformations to achieve the desire output efficiently
- Extensively used Informatica Debugger to analyze/resolve the data issues.
- Performance tuning of PowerCenter mappings, sessions for reducing the job times to load data in Data Marts.
- Worked with Business Analyst & SME on understanding requirements and convert them into the technical specification.
- Worked with Stake holders on coordinating and managing System Testing and User Acceptance Testing.
- Coordinate and worked on Break Fix Requests and Data Analysis requests.
- Coordinate and worked with operations teams for QA and Production implementations.
- Coordinate and manage the Production support activities.
- Leading the team (onsite-offshore model) in coding, testing, performance tuning in developing Informatica and Oracle objects.
- Presented Weekly/Monthly Status Reports to client for effective effort tracking.
- Involved in 24*5 production support for ETL job run.
