Senior Etl Developer / Data Engineer Resume
2.00/5 (Submit Your Rating)
San Francisco, CA
SUMMARY
- Data Engineer with 8 years of IT experience in Analysis, Design, Testing, Development and Implementation of client/server applications.
- Strong experience in performing ETL operations like Data Extraction, Data Transformation and Data Loading with Informatica Power Center 10.2/10.1/9.6.1/9.1.0/9.0/8.6.0.
- Worked intensively with Informatica Powercenter Designer. Strong knowledge on Source Analyzer, Warehouse designer, Transformation Developer, Mapplet Designer and Mapping Designer.
- Extensively used Informatica Workflow manager and Workflow monitor for creating and monitoring workflows, worklets and sessions.
- Extensively worked on Informatica PowerCenter Transformations such as Source Qualifier, Lookup, Filter, Expression, Router, Normalizer, Joiner, Update Strategy, Rank, Aggregator, Stored Procedure, Sorter, Sequence Generator and XML Source Qualifier.
- Identified the Facts and Dimensions using Erwin Data modeling tool to represent the Star Schema Data Marts and to build OLAP/OLTP models.
- Experience with ODS (Operational Data Store) to integrate data from different heterogeneous data sources in order to facilitate operational reporting in real - time or near real-time.
- Used ODS as data source frequently for the purpose data warehouse.
- Extensively performed unit testing and system or integration testing.
- Experience in testing/validating the Source, Stage and Target (End-to-End).
- Experience in writing Test cases and Testing Scripts for validating data.
- Involved in massive data profiling using IDQ (Analyst Tool) prior to data staging.
- Very strong in implementation of data profiling, creating score cards, Creating reference tables and documenting Data Quality metrics/dimensions like Accuracy, completeness, duplication, validity, consistency.
- Worked on analyzing data with Hive and Pig.
- Involved in Migration of mapping from IDQ to PowerCenter.
- Extensive experience in Informatica Power Center to implement Data Marts which involved creating debugging and executing Informatica Mappings, Sessions, Tasks and Workflow.
- Experience in databases like RDBMS systems Oracle 12c/11g/10g/9i/, SQL Server, Netezza, TeraData.
- Strong problem-solving skills, ability to develop strategies for clients and provide detailed documents to clients.
- Experience working with oracle EBS applications like ERP and CRM.
- Experience interacting with Oracle EBS analysts to understand business process and provide technical support.
- Experience in creating databases, users, tables, triggers, views, stored procedures, functions, joins, hash indexes in Teradata database.
- Loading of data from different source systems into Oracle and Teradata Target database.
- Excellent analytical and SQL writing skills to address different business scenarios.
- Experience with workload automation tools like Autosys, Tidal and Airflow.
- Exposure to full project life cycle development for implementation and integration.
- Hands on experience in Production Support and Maintenance by resolving issues to meet all Data Warehouse service Level agreements (SLAs).
- Extensive Experience on Production Support.
- Experienced in gathering of User requirements and responsible to provide efficient end-user training and designing report specifications.
TECHNICAL SKILLS
- Informatica PowerCenter 10.2/10.1/9.6.1/9.0/8.6.1/ , Power Connect for Mainframe/SAP/Peoplesoft/MQSeries/IBM MQ, Informatica IDQ, Trackwise, SQL*Loader, Flat Files (Fixed, CSV, Delimited, XML, COBOL), Salesforce.com
- Dimensional Data Modeling, Star Schema Modeling, Snow-Flake Modeling, FACT and Dimensions Tables, OLAP and OLTP.
- RDBMS Oracle 12c/11g/10g/9i, MS SQL Server 2000/7.0, DB2, TOAD 8.5.1/7.5., Netezza, Teradata, MongoDB, Hive
- Business Objects and OBIEE, Qlikmart, Tableau, Autosys, Redwood, Tidal, Airflow
- C, SQL, PL/SQL, SQL*Plus, Working knowledge of Unix Shell Scripting, Python
- Microsoft XP/NT/2000, UNIX, LINUX
PROFESSIONAL EXPERIENCE
Confidential, San Francisco, CA
Senior ETL Developer / Data Engineer
Responsibilities:
- Interact with the users to gather the requirements, and functional specifications to develop ETL procedures that are consistent across application and systems.
- Configure the connections in informatica to extract the data from SFDC API's.
- Extracted data using infromatica power center and data loaders by connecting to SFDC cloud.
- Implemented bulk api to extract huge volumes of data from the salesforce API's and address all the performance issues encountered.
- Tuned the Synchronization and Replication jobs to reduce the runtimes of the jobs and eliminate data contention in the bulk data jobs.
- Performed data profiling and analysis of various objects in SalesForce.com (SFDC) and MS Access database tables for an in-depth understanding of the source entities, attributes, relationships, domains, source data quality, hidden and potential data issues, etc.
- Wrote Ad - hoc queries for analyzing the data using HIVE QL.
- Developed Source-Target Map documents to describe the relationship between source and target data.
- Created multiple Hive tables, implemented partitioning, dynamic partitioning and buckets in Hive for efficient data access.
- Developed Python scripts for automated daily load processes and data analysis.
- Expertise in Salesforce-Informatica Integration. Experience in Sales Force CRM Configuration, Customization, and Testing of applications.
- Developed ETL Specification Design document containing detailed information on ETL processing, mapping/workflow specifications, exception handling process, staging and data warehouse schemas, etc.
- Designed and developed Informatica ETL Interfaces to load data incrementally from SFDC, MS Access databases and Flat files into Staging schema.
- Designed and developed Informatica ETL/SCD Interfaces to load data from staging schema into the Customer Reporting DW dimension tables.
- Designed and developed Informatica ETL Interfaces to load data from staging schema into the Customer Reporting DW Fact tables.
- Scheduled ETL jobs for the entire DW load process using Tidal Scheduler.
Environment: Informatica Power Center 10.2/10.1, Oracle 12c/11g, SFDC API’s, UNIX, Python, Tidal Automation Tool, Hive, Pig, WinSCP, Putty, Tableau.
Confidential, San Ramon, CA
Data Engineer
Responsibilities:
- Analyze user requirements, current operational procedures, and functional specifications to develop ETL procedures that are consistent across application and systems.
- Implemented Perform tuning and optimization of SQL queries, using Explain Plan and tkprof.
- Implemented and maintained multiple ETL processes to synchronize data between different source systems and database.
- Develop and Implement comprehensive data integration process, standardize the presentation of information derived from multiple sub ledger transaction systems and develop data necessary.
- Developed ETL code, control files, metadata, and lineage diagrams for ETL programs.
- Reviewed and altered database programs to increase operating efficiency.
- Extracted structured and unstructured data from source systems, transforming it to business needs.
- Created and Configured Workflows and Sessions to transport the data to target warehouse Oracle tables using Informatica Workflow Manager and Worked on different tasks in Workflows like sessions, events raise, event wait, decision, e-mail, command, worklets.
- End-to-end performance tuning of Hadoop clusters and Hadoop MapReduce routines against very large data sets.
- Coordinated with Data Architect in designing and developing re-usable components for ETL processes.
- Used Workflow Manager for creating, validating, testing and running the sequential and concurrent sessions and scheduling them to run at specified time and as well to read data from different sources and write it to target databases.
- Developed database code for the feature enhancement in CCAR project.
- Worked along with UNIX team for writing UNIX shell scripts to customize the server scheduling jobs and Used Shell Scripts which compares the incoming file with the existing file and creates a log file with differences and the changed values are loaded into the target.
- Created detailed Unit Test Document with all possible Test cases/Scripts and involved in Unit, Integration, System and Performance testing levels.
- Experience working with XLM files as source to stage the data and to generate XML files using Informatica.
Environment: : Informatica Power Center 10.1/9 .6.1, Oracle 12c/11g, UNIX , Python, Tidal Automation Tool, WinSCP,Hadoop,Hive,MongoDB, Putty .
