Data Engineer Resume
SUMMARY
- Highly skilled consultant bringing more than 12 years’ experience as Data and Business Analyst in financial domain. Advanced knowledge of various ETL Tools with expertise in all aspects of the software development life cycle, including requirements analysis, design, development and production support.
- 12+ years of industry experience in analysis, design, development, implementation and troubleshooting in Business Intelligence - Data warehousing.
- As a Data Architect/Modeler design the DataMart in the best possible way considering business requirements, data usage, ETL/ELT capabilities and to also future enhancements
- As an ETL developer/Data engineer have an extensive experience in ETL methodology for performing data profiling, data migration, data ingestion, extract, transform and load using various ETL tools such as Ab initio, informatica Talend Data Integration open Studio, Talend Big Data integration open Studio, from a wide variety of source systems including Teradata, hive, Hdfs, Db2, Oracle, Netezza and other non-relational sources such as mainframe files, XMLS and flat files.
- As a Data Analyst, collection and organization of data, finding correlations between that analyzed data to spot problems and initiate preventative measures or problem solve creatively
- As a Business Analyst, experienced in driving product features from Discovery to Delivery, participated in discussions with business stakeholders to understand the business requirements and translating them effectively into functional and technical requirements.
- As a scrum master, leading a scrum team of 10+ members from different time zones
- Experience in the IT industry in verticals like Banking and Financial Services domain with emphasis on design, development, implementation, and management of projects.
- Experience in streamlining online mortgage data and pushing the data in Hadoop(hive) environment using ACQUIRE>IT module of Ab Initio and Talend Big Data Integration Studio.
- As an ETL- Ab-initio lead have extensive experience in Extraction, Transformation, loading (ETL) of data from various sources into Data Warehouses and Data Marts using Ab Initio and Informatica Power Center 9.1/8.6/7.1x (Repository Manager, Designer, Workflow manager and Workflow Monitor).
- Expertise in Ab-initio, Acquire >IT, Conduct>It, PDL, MDH, EME version control, Data Lineage, Data dependency, Oracle, PL/SQL, UNIX-Shell Scripting, HIVE, XML, JSON & AVRO and CDC GoldenGate file format.
- Experienced in working with XMLs and Multifile systems (MFS).
- Experience working on Ab initio Batch flows, Continuous flows and Plans
- Knowledge of pig Latin and HiveQL for data analysis and ETL .
- Worked on various SQL and No-SQL databases - Teradata, Oracle 8i/9i/10g/11g, DB2 V9, SQL Server, Nettezza, Hive
- Experience on Teradata tools and utilities (BTEQ, Fast load, Multi Load, Fast Export, and TPUMP).
- Expertise with different types of data load strategies such as Historical Load, Truncate and Load, Dimensional Load.
- Experience in working on Near Real Time Data processing.
- Experienced in developing and maintaining overall Test Methodology and Strategy, Documenting Test Plans, Test Cases and editing, executing Test Cases and Test Scripts.
- Experienced in UNIX Shell and Python Scripting.
- Experience in Data Masking and Data Quality.
- Involved in Performance tuning of Ab-initio Graphs, Teradata SQLs and Good knowledge on logical and physical Data Modeling using normalization Techniques.
- Expertise in creating Joins and Sub queries for complex queries involving multiple tables with Strong Experience in writing advanced query concepts using having clause, group by etc.
- Strong work ethics, able to take ownership of all duties and responsibilities and efficient time management skills. Group and individual based problem solving and decision-making skills.
- Excellent analytical, programming, written and verbal communication skills with ability to interact with individuals at all levels.
TECHNICAL SKILLS
Agile Tools: JIRA, Hp Quality Center
ETL Tools: Ab Initio GDE & Co-operating system, Acquire>IT, Informatica Power center 9.1/8.6/7.1.3 , Talend DATA integration open Studio, Talend Big Data Integration open studio, pysprk
Databases: Teradata, Hadoop/Hive, Oracle 8i/9i/10g/11g, DB2 V9, SQL Server
Database Utilities: ERWIN, power designer, Teradata Sql Server, Teradata studio, Toad for Oracle 11.6, Toad for Data Analysts 3 (Netezza/Oracle), Toad for DB2, SQL Plus, Toad for SQL Server
Languages: Unix Korn Shell Scripting, python, Oracle PL/SQL, DB2 Procedural language, C, Core JAVA, Pig scripting (HiveQL)
Version Control Tools: VSS
Change and Configuration Management: Rational
Scheduling Tools: Autosys, Control-M
Operating Systems: Unix (AIX/Solaris), Linux, Windows 9X/2000/XP/2K8/Vista/7/ NT, MS-DOS, Mac (Intel/PPC)
Desktop Applications: MS Project, MS SharePoint, Microsoft Office Suite
PROFESSIONAL EXPERIENCE
Confidential
Data Engineer
Environment: Ab Initio GDE 3.3.4, CO-OP 3.0, Teradata SQL Assistant 13.10, Toad for Oracle 10.6, Unix Shell Scripting, Autosys/Control-M, Hadoop, Hive
Responsibilities:
- Interacting with business users, customers, data analysts and modelers to define the requirements into more concrete and detailed terms
- Provide solutions with optimized designs by considering the business needs and Building logical and physical data model using power designer /Erwin
- Develop with Talend studio building jobs for data migration and data warehousing on multiple projects
- Develop, Deploy and support Talend packages for home mortgage applications with various sources.
- Design and development software products targeted towards financial institutions using Ab - initio and Talend.
- Used ETL methodologies and best practices to create ETL jobs. Followed and enhanced programming and naming standards.
- Design and Implemented ETL for data load from heterogeneous Sources to SQL Server and Oracle as target databases and for Fact and Slowly Changing Dimensions SCD-Type1 and SCD-Type2.
- Get hold of security master central architecture and propose new solutions to improvise the existing system.
- Coordination with the team to handle the changes impacted to multiple projects.
- Followed the multi-lane(branching) policy for the development to work on projects simultaneously.
- Production fixes on the existing processes.
- Data analysis to provide various results to business.
- Design/Code review for modules in Talend/Ab-initio and Teradata.
- Prepare detailed documentation for Ab-initio used applications like Design Documents, Run Books, Code Checklists, Test plans and so on.
- Optimization for the existing Talend Mappings to reduce the running time. Also updated existing logics to make the graphs more sophisticated.
- CDC (change data capture) has been implemented in the Teradata instead of Ab Initio to lift the performance.
- Mainframe files have been accessed using the gateway to reduce the physical space used on UNIX server, and file transfer time for optimizing the run times.
- Plans used for running generic load/unloads with looping functionality, to reduce the number of objects and jobs scheduled.
- Worked with QA team to explain the test cases and fix the data issues
- 24 X 7 production support for the home mortgage origination and servicing applications and provide solutions to the production failure jobs.
- Responsibility of project role outs and implementations within the specified SLAs.
Confidential
Data Engineer
Responsibilities:
- Interacting with business and customers to understand the architecture of the applications.
- Provide solutions with optimized designs by considering the business needs.
- Design, Develop, deploy and support Teradata procedure to encrypt the data in database layer and created used defined functions to be called from ETL layer
- Used Teradata utilities FAST LOAD, MULTI LOAD, TPUMP to load data.
- Wrote, tested and implemented Teradata Fast load, Multiload and BTEQ scripts, DML and DDL.
- Data analysis to provide various results to business and users for encryption and decryption validation
- Develop, Deploy and support Ab Initio packages for Teradata encryption application for the entire mortgage data mart
- Design and development software products targeted towards financial institutions using Ab Initio.
- Get hold of security master central architecture and propose new solutions to improvise the existing system.
- Coordination with the team to handle the changes impacted to multiple projects.
- Followed the multi lane (Branching) policy for the development to work on projects simultaneously.
- Production fixes on the existing processes.
- Design/Code review for modules in Ab-initio and Teradata Bteq Scripts.
- Prepare detailed documentation for Ab-initio used applications like Design Documents, Run Books, Code Checklists, Test plans and so on.
- Optimization for the existing Ab-initio graphs to reduce the running time. Also updated existing logics to make the graphs more sophisticated.
- Encrypted data from 700 tables to create the required data extract from an Common Opportunities and research environment data mart.
- Used BTEQ and SQL Assistant (Query man) front-end tools to issue SQL commands matching the business requirements to Teradata RDBMS.
- Plans used for running generic load/unloads with looping functionality, to re- duce the number of objects and jobs scheduled.
- Worked with QA team to explain the test cases and fix the data issues.
- 24 X 7 production support for the home mortgage origination and servicing applications and provide solutions to the production failure jobs.
- Responsibility of project role outs and implementations within the specified