We provide IT Staff Augmentation Services!

Etl/hadoop Developer Resume

2.00/5 (Submit Your Rating)

Seattle, WA

SUMMARY:

  • Over 10+ years of IT experience in the Analysis, Design, Development, Testing and Implementation of Data ware house application systems for Health care, Banking, Financial and Telecom Sector.
  • Experience in Agile/Dev - ops tools.
  • Strong experience in the Analysis, design, development, testing and Implementation of Business Intelligence solutions.
  • Experience in using Filter, Expression, Sequence Generator, Update Strategy, Joiner, Router and Aggregator transformations to create the mappings in Informatica Power Center Designer.
  • Implemented Slowly Changing dimension type2 methodology for accessing the full history of accounts and transaction information.
  • Involved in extracting the data from the Flat Files and Relational databases into staging area.
  • Created Unix Scripts and used them as Pre-session and Post-Session Commands for reading the Data from flat files and archiving the Flat files at the specified Central repository.
  • Performed transformation tunings at Informatica transformation level and at the database level to achieve optimization.
  • Experience in using Automation Scheduling tools like Crontab, Tidal and Control-M.
  • Experience in preparation of Test scenarios, Test cases and executing the same in Test tool.
  • Expertise in Data Warehouse, Data mart and ODS implementations teamed with project scope, Analysis, requirements gathering, data modeling, Effort Estimation, ETL Design.
  • Development, System testing, Implementation and production support.
  • Extensive involvement in Unit testing, Regression Testing and UAT.
  • Experience in Data modelling activities using Erwin tool.
  • Experience in attribute validation, data flow testing, load strategy testing, Duplicate data validation, Data mapping with direct load, data load with derived attributes.
  • Experience in Defect life cycle and generating the system test plan and test report.
  • Experience in the Test Management Tool Confidential Quality Centre and Confidential ALM.
  • Experience on UNIX multi file system and mocking the data in UNIX environment.
  • Experience in Informatica upgrade activity from 8.6 to 9.5 versions.
  • Exposure in Sales Domain and Banking Domain - Core and Commercial banking.
  • Lead development team comprising of Offshore and Onshore associates.
  • Involved in Confidential Splitting activity workshop, Dev-ops and Scrum workshop.

TECHNICAL PROFICENCY:

ETL Tools : Informatica power center 9.6/8.5/7.1, Informatica Data Explorer, Teradata

Data Bases : Oracle 11g/10g/9i, SQL Server 2008, Teradata, Hadoop

Database tools : Toad data point, TD SQL assistance, SQL Navigator, Hive, PIGDeployment Tools: Jenkins, Nexus, Github

Testing tools : Confidential Quality Center 10/11.0, Confidential ALM, JIRA

Operating Systems : Windows 2003, XP, Linux, Unix

Other Tools : PUTTY, Winscp, HPSM, Service now, Cron-Tab, Tidal scheduler, Control-m, Rally, Ambari

EXPERIENCE SUMMARY:

Confidential, Seattle, WA

ETL/Hadoop Developer

Responsibilities:

  • Analyze requirements to assess testability and understand underlying ETL Framework logic to perform comprehensive test planning activities.
  • Interacting and conducting necessary meetings with Business counterparts and various teams to understand the business functionality and the value of the project.
  • Frame ETL Fact and Dimension test scenarios and write all-inclusive test cases comprising of White and Black box test suite along with boundary value test cases for the business requirements of the project.
  • Writing HIVE SQLs and prepare Sqoop, Hadoop, HBase, MySQL commands to validate the HADOOP Data loads and HIVE tables.
  • Perform extensive Data Mining on the test environment to do test data set up for the project, to validate the data as per the ETL framework.
  • Preparing complex Teradata SQL and PL/SQLs scripts to verify the ETL transformation logic and perform data validation to ensure the integrity of ETL code and Data loads as per the business requirements.
  • Writing UNIX commands and Shell scripts to execute and validate the data.
  • Tuning of Scripts to improve and test critical data flow paths.
  • Work with ALM and other CICD tools to maintain the software integrity and create a test case warehouse suite.
  • Review and Analysis of the developed ETL code (Informatica, Nifi, RabbitMQ), Hadoop and Teradata bteq scripts to expose security flaws, reveal defects and identify areas of optimization.
  • Performing ETL Test Lead activities like conducting defect triage meetings, publishing status reports of all open issues and gaps found during test phase and articulate the details to all stakeholders.
  • Implement automated ETL Quality Assurance processes using either SQL or Shell scripts for a continuous deployment pipeline.
  • Preparing and maintaining all artifacts of the project in respective project configuration and maintenance tools.

Technologies: Informatica Power Center 9.6, Oracle 11g, Teradata, Hadoop, HBase, Hive, Jenkins, Rally, Github.

Confidential, CA

ETL Developer

Responsibilities:

  • Analyzed the existing OLTP system
  • Extensively involved in Data Extraction, Transformation and Loading (ETL process) from Source to target systems using Informatica.
  • Followed the organization defined Naming conventions for naming the Flat file structure, Informatica Mappings, Workflows, sessions and daily batches for executing the Informatica workflows.
  • Performed data cleansing and transformation using Informatica.
  • Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Router, and Aggregator to create robust mappings in the Informatica Power Center Designer.
  • Worked extensively with Designer tools like Source Analyzer, warehouse Designer, Transformation Developer, Mapping and Mapplet Designers.
  • Created Workflows and defined the relational connections for different environments for the smooth running of the Conversion process using Workflow Manager Tools like Task Developer, Workflow Designer and Worklet Designer.
  • Created Unix Scripts and used them as Pre-Session and Post-Session Commands for reading the Data from flat files and archiving the Flat files at the specified Central repository.
  • Developing Informatica Mappings & tuning them when necessary.
  • Developed various worklets that were then included into the workflows.
  • Unit testing for validating the data to meet business requirements.
  • Extensively executed SQL queries on Oracle using Toad and SQL server tables to view successful transaction of data and to validate data.
  • Created the scripts using python and deployed in different environments.
  • Migrated Mappings, Sessions, Workflows from Development to Test and then to Production environment.

Technologies: Informatica Power Center 9.6, Oracle 11g, Toad data point 4.0, Putty, Solaris

Confidential, VA

Senior ETL Developer

Responsibilities:

  • Extensively involved in Data Extraction, Transformation and Loading (ETL process) from Source to target systems using Informatica.
  • Performed data cleansing and transformation activities.
  • Worked extensively with Designer tools like Source Analyzer, warehouse Designer, Transformation Developer, Mapping and Mapplet Designers.
  • Created data mappings to extract data from different source files, transform the data using Filter, Update Strategy, Aggregator, Expression, Joiner Transformations and then loaded into data warehouse.
  • Implemented Slowly Changing dimension type2 methodology for accessing the full history of accounts and transaction information.
  • Used Update Strategy Transformation to update the Target Dimension tables, type2 updates where we insert the new record and update the old record in the target, so we can track the changes in the future.
  • Developed various worklets that were then included into the workflows.
  • Used Workflow Manager to read data from sources and write data to target databases and manage sessions.
  • Worked on ECM4CRM and Mainframe applications to publish the data to different systems.
  • Assigning the work to the Team members and conducting the daily meetings to make sure no gaps in understanding the requirement.
  • Lead the Team and assisted the team on critical issues from onshore.

Technologies: Informatica Power Center 9.6 (Designer, Workflow Manager, Monitor, Repository Manager), Oracle, Tidal scheduler, Linux, TOAD

Confidential, San Francisco, CA

Informatica ETL Developer

Responsibilities:

  • Clearly understand source systems by going through the functional specification documents and one-to-one interaction with the business team.
  • Developed data mapping documents that contain transformation rules to implement the business logic.
  • Developed various mappings with the collection of all sources, targets, and transformations using designer. Used version mapping to update the slowly changing dimensions to keep full history to the target database.
  • Project planning, work scheduling and work tracking to ensure the adherence to the project plan.
  • Translating high level requirements documents to design documents.
  • Made substantial contributions in simplifying the development and maintenance of ETL by creating shortcuts, re-usable Mapplets and Transformation objects.
  • Used update strategy to effectively migrate slowly changing data from source system to target Database.
  • Used transformations like aggregator, filter, router, stored procedure, sequence generator, lookup, expression and update strategy to meet business logic in the mappings.
  • Created pre and post session Stored procedures to drop, recreate the indexes and keys of source and target tables.
  • Understand the components of a data quality plan (Data Profiling).
  • Designed data transformation to staging, fact and dimension tables in the warehouse.
  • Involved in designed tables and implementing Informatica mappings and workflows for extraction of the data from the source systems to populate Staging Area, Dimension and Fact Tables.
  • Involved in designed tables and implementing Informatica mappings and workflows for extraction of the data from the source systems to populate Staging Area, Dimension and Fact Tables.
  • Extensively worked with various lookup caches like Static Cache, Dynamic Cache and Persistent Cache.
  • Worked with PMCMD to interact with Informatica Server from command mode and execute the batch scripts.
  • Scheduled and ran Extraction, loading processes using Crontab.
  • Developed unit test case scenarios for thoroughly testing ETL processes and shared them with testing team.
  • Developed the strategies like CDC(Change Data Capture), Batch processing, Auditing, Recovery Strategy etc.
  • Tested the data and data integrity among various sources and targets. Used debugger by making use of breakpoints to monitor data movement, identified and fixed the bugs.
  • Used power center workflow manager for session management, database connection management and scheduling of jobs to be run.
  • Wrote Pl/Sql in Oracle, Sql server 2008 for data Audit and maintained them.
  • Developed Informatica workflows and sessions associated with the mappings.
  • Scheduled walkthroughs of design documents, specifications, code, test plans etc. as appropriate throughout project lifecycle.
  • Used Workflow Monitor to monitor the progress of workflow.
  • Tuning Informatica Mappings and Sessions for optimum performance.
  • Code promotion to SIT, UAT and Prod.

Technologies : Informatica 8.6.1/9.5, UNIX, DB2, Flat Files, Oracle 11g, Excel Files, XML Files, Toad for Oracle, SQL Server 2008, Confidential Quality Center.

Confidential, Richmond, VA

Informatica ETL Developer

Responsibilities:

  • Development of Business Requirements Design documents
  • Extensively worked with Repository Manager, Designer, Workflow Manager and Workflow Monitor.
  • Developed transformation logic and designed various Complex Mappings and Mapplets using the Designer.
  • Developed complex mapping to implement slowly changing Dimension (SDC).
  • Worked with the Lookup, Aggregator, Expression, Router, Filter, Update Strategy, Joiner Transformations.
  • Developed various worklets that were then included into the workflows.
  • Used Workflow Manager to read data from sources, and write data to target databases, and manage sessions
  • Developed complex mappings to implement type 2 slowly changing dimensions using transformations such as the Source qualifier, Aggregator, Expression, Static Lookup, Dynamic Lookup, Filter, Router, Rank, Union, Normalize, Sequence Generator, Update Strategy and Joiner.
  • Involved in designed tables and implementing Informatica mappings and workflows for extraction of the data from the source systems to populate Staging Area, Dimension and Fact Tables.
  • Performed tuning of Informatica sessions by implementing database partitioning, increasing block size, data cache size, sequence buffer length, target based commit interval.
  • Migrating code from different Environments.
  • Execution of queries in validating the Source and Target data tables.
  • Understanding the Business Requirements and Functional Requirements specification.
  • Defects tracking & Analyzing Test Results using test management tool, Confidential Quality Center.
  • Being end to end expertise in functionality, giving KT to new resources.
  • Actively participated in DRB meetings.
  • Participating in Status Calls with Onsite Clients and QA resources.
  • Mentored and coordinated 13 member’s team during testing activities for multiple releases.
  • Analyzed on GIS systems to load the dimension tables for North America region.
  • Tested all business functionalities and mapped the requirements in QC - analysis of its coverage.
  • Assisted in batch processing and verified the jobs status and data in database tables.
  • Collected and verified the necessary data for batch run across team and reported the same.
  • Responsible for defect tracking, retesting and closing of the defects as per defect life cycle.
  • Analyzed and developed software test strategies and executed the test methods; respective SQL scripts to validate the data. Peer reviewed results.
  • Troubleshoot failures, worked with development team to point and resolve problems.
  • Validation of transformation as per business rules performing peer reviews and walkthrough
  • Coded small macros in excel to run the daily testing reports efficiently.

Technologies : Informatica 8.5, UNIX, SQL, Oracle, Confidential ALM 11, Citrix, Putty, TOAD 10.6.

Confidential

ETL Developer

Responsibilities:

  • Extensively involved in Data Extraction, Transformation and Loading (ETL process) from Source to target systems using Informatica.
  • Followed the organization defined Naming conventions for naming the Flat file structure, Informatica Mappings, Workflows, sessions and daily batches for executing the Informatica workflows.
  • Performed data cleansing and transformation using Informatica.
  • Used various transformations like Filter, Expression, Sequence Generator, Update Strategy, Joiner, Router and Aggregator to create robust mappings in the Informatica Power Center Designer.
  • Worked extensively with Designer tools like Source Analyzer, warehouse Designer, Transformation Developer, Mapping and Mapplet Designers.
  • Created Workflows and defined the relational connections for different environments for the smooth running of the Conversion process using Workflow Manager Tools like Task Developer, Workflow Designer and Worklet Designer.
  • Created Unix Scripts and used them as Pre-Session and Post-Session Commands for reading the Data from flat files and archiving the Flat files at the specified Central repository.
  • Code peer review before migrating the code to production environment.
  • Developed various worklets that were then included into the workflows.
  • Unit testing for validating the data to meet business requirements.
  • Extensively executed SQL queries on Oracle using Toad and SQL server tables to view successful transaction of data and to validate data.
  • Migrated Mappings, Sessions, Workflows from Development to Test and then to Production environment.
  • Assigning the work to the Team members and conducting the daily meetings to make sure no gaps in understanding the requirement.

Technologies : Informatica Power Center 8.5 (Designer, Workflow Manager, Monitor, Repository Manager), Oracle, Tidal scheduler, Linux, TOAD

Confidential, TX

ETL Developer

Responsibilities:

  • Followed the organization defined Naming conventions for naming the Flat file structure, Informatica Mappings, Workflows, sessions and daily batches for executing the Informatica workflows.
  • Performed data cleansing and transformation using Informatica.
  • Worked extensively with Designer tools like Source Analyzer, warehouse Designer, Transformation Developer, Mapping and Mapplet Designers.
  • Understanding the Business Requirements and Functional Requirements specification.
  • Analyzed business requirements and worked closely with the various application teams and business teams to develop ETL procedures that are consistent across all applications and systems.
  • Involved in Analysis, Design, Development, test and implementation of Informatica transformations and workflows for extracting the data from the multiple systems.
  • Interacted with the Business Analysts in collecting the technical and business requirements for the project.
  • Experience in translating high level requirements documents to design documents.
  • Extensively worked with various Active transformations like Filter, Sorter, aggregator, Router and Joiner transformations. Passive transformations like Expression, Lookup and Sequence Generator.
  • Created complex mappings using unconnected and Connected Lookup transformations.
  • Worked with the Joiner transformation using Normal Join, Master outer join, Detail Outer Join and Full Outer Join. Implemented slowly changing dimension Type 1 and Type 2 for changed data capture.
  • Used Sorter and Aggregator transformations in combination for performance tuning of aggregations used in the mappings. Implementing active transformation like filter as early as possible in the mapping.
  • Worked with various Informatica Power center objects like Mappings, transformations, Mapplets, Workflows and Session Tasks.
  • Created pre and post session Stored procedures to drop, recreate the indexes and keys of source and target tables.
  • Created Transformations like Sequence generator, Lookup, joiner and Source qualifier transformations in Informatica Designer.
  • Monitored workflows and session using Power center workflows monitor and used Informatica Scheduler for scheduling the workflows.
  • Created Mapplet and used them in different Mappings.
  • Developed Informatica workflows and sessions associated with the mappings.
  • Performed error handling of sessions by using terse, normal, verbose initialization and verbose data tracing levels.
  • Scheduled walkthroughs of design documents, specifications, code, test plans etc. as appropriate throughout project lifecycle.
  • Used Workflow Monitor to monitor the progress of workflow.
  • Involved in post-production support once the code has moved to production.

Technologies: Informatica 7.1, Unix Confidential v7, Oracle 9i, Confidential Quality Center, Citrix, Putty and TOAD 9.

We'd love your feedback!