Data Engineer Resume
SUMMARY
- Over 9+ years of experience in ETL/ELT Development using various tools Informatica, Sas DI tool, DBT etc with a strong background in Analysing, Designing, Developing, Testing, Debugging, Migrating including writing SQL Programming, and Implementing of Data Warehouse development in domains such as Academic, Banking, Finance.
- Performance Tuning by Identifying the bottlenecks, creating subject areas in OBIEE using Datawarehouse techniques and visualization and analysis of data using Tableau.
- Strong Knowledge on RDBMS, Datawarehouse concepts, Star and Snowflake schema designing, OLTP, OLAP, Timestamp based CDC, slowly changing Dimensions and Fact loads.
- Experience on Snowflake on Bulk Loading data from AWS S3, creating stages, Warehouse scaling and continuous ingestion using snow pipe etc.
- Experienced in creating Databases, Derived Tables, DDL/DML, Views/Materialized Views, Indexes to facilitate efficient Data Manipulation and Consistency.
- Strong Knowledge in Normalizing/DE normalizing tables to maintain referential integrity using Primary and Foreign keys.
- Performed activities including execution of test plans, design of exception handling strategy and performance tuning.
- Experience in Creating Subject Areas (RPD Development work) by designing the Star and Snowflake schemas in OBIEE BI Reporting tool for generating Summarized, Detailed reports in the Oracle Analytical Cloud migration project.
- Worked under Agile Methodology in most of the projects, performed a decisive role in developing and planning as well as coordinating with assigned Sprints.
- Experience on UNIX Shell scripting.
TECHNICAL SKILLS
ETL /ELT Tools: Informatica PowerCenter V 10.2.0/9.6/8 x, Sas DI studio v4.9DataStage v11.5
Reporting Tools: Oracle BI Administration Tool V12.2, Oracle Analytical Cloud, Tableau
Databases: Oracle 10g/11g, SQL, PL/SQL, Hue
Data Modelling: ERWIN4.x/3.x,Star/SnowflakeSchema,Dimension Modelling
Data Analysis: Informatica Data Quality, Analytix DS
Ticketing Tool: Remedy v7.1/8.1, Atlassian Jira, Rally
Job Console: Atomic Web Interface v12.3, Control - M
Cloud DWH: Snowflake, AWS S3
PROFESSIONAL EXPERIENCE
Confidential
Data EngineerResponsibilities:
- Analysing and preparing designing documentation and registering the asset metadata in cloud and activation of files/tables in MFP.
- Integration of data from multiple sources and ELT the process to load data in a snowflake environment.
- Extracting files from various locations and creating an MFP drop zone for processing.
- Creating Pre-processing jobs using non talon batch.
- Creating test cases (qtest) for Unit Testing includes Rows Threshold, Key Column data format, QC Count, Dimension Check of overlapping data for keys, Late shout.
- Bulk loading of data from the external stage (AWS S3), internal stage to snowflake cloud using COPY commands.
- Writing complex snow sql scripts in snowflake cloud data warehouse for analysis and reporting.
- Created clone objects to maintain zero-copy cloning and Cloned Production data for code modifications and testing when needed.
- Created yaml files, packages for transforming the data using DBT tool.
- Developed ETL/ELT data pipeline flow to load data from various data sources to the staging database and apply complex business logic to populate normalized and denormalized data structure using DBT.
- Worked on Snowflake cloud-based project and to design dynamic ETL solution to load the data from On-prem to Cloud Data Warehouse.
- Implemented SCD type1 and type2 using DBT Pipelines to capture data in snowflake.
Environment: DataStagev11.5, Git Lab, Unix, Snowflake, AWS S3, Squirrel Sql Client v4.4.0 DBT, Control-M, Jira, WinSCP.
Confidential
ETL Developer
Responsibilities:
- Involved in the development of Client Analytical Repository by designing EDW mappings based on the provided mapping document.
- Worked for preparing design documents and interacted with the data modelers to understand the data model and design.
- Created new mappings and updating old mappings according to changes in Business logic.
- Involved in migrating project from UAT to Production.
- Performed data quality issue analysis using Snow SQL by building analytical warehouses on Snowflake.
- Perform troubleshooting analysis and resolution of critical issues.
- Architecting, designing and operationalizing large-scale data and analytics solution on MPP (Microsoft APS) and Cloud System (Snowflake).
- Designed, Configured and Fine-tuned the existing ETL workflows in Informatica.
- Worked with business users to ensure the architecture design meets the product requirements.
- Used Agile Development concept with the application being divided into modules and developed in cascading stages.
- Designed ETL/ELT process for periodic automated loads including FTP/SFTP file transfer to/from SQL Server to Unix.
- Coordinated in daily team meetings, technical code review meetings and interacted with businesspeople for better technical solutions and proposed ETL strategy based on Agile Methodologies.
Environment: Informatica V10.2, Microsoft Parallel Data Warehouse (APS 2015), Snowflake, AWS S3, Oracle 10g/9i, TOAD Data Point, Atomic Job Console, Rally, Hue, Winscp.
Confidential
Data Modeler/ETL Developer
Responsibilities:
- Responsible for designing, developing, validating and migrating data mappings.
- Involved in Cognos decommission project where all the data flows were designed, migrated to Informatica Power Center and Sas Di studio.
- Experience of writing SQL query scripts for independent data analysis.
- Performance tuned Informatica session for large data files by increasing block size, data cache size, buffer length and target-based commit interval and eliminated the bottlenecks to decrease the runtime of the job.
- Created and maintained technical documentation for source-to-target mapping and assisted in production support by resolving source data issues and refining transformation rules whenever needed.
- Involved in performance tuning and optimization of Informatica mappings and sessions using features like partitions to manage very large volumes of data.
- Knowledge on building interactive dashboards/charts to convey the story inside the data using Tableau.
- Prepared migration documents to promote the mappings from development to testing and then to production repositories. Worked on a migration project to extract all the data files from old server to new server and FTPing to target server.
- Involved in system and design performance tuning projects that helped to reduce the overall CPU and memory utilization by 15%.
- Involved in the RPD development work, created Subject areas by building the logical and physical models for dimensional modelling projects related to Academic, HR, Financial data.
- Created dashboards by extracting data from different sources using parameters, sets, groups and calculations to visualize fund expenditures and enrolments’ data for the last 10 years census point.
Environment: SAS DI Studiov4.9, Informatica, Oracle Business Intelligence Client, Tableau, Atomic Job Console, Atlassian Jira, Erwin, Mobaxterm Personalv12.2.
Confidential
Application Developer
Responsibilities:
- Developed and supported the ETL process for Heterogeneous source systems like as Oracle, Flat files, etc. using Informatica Power centre 9.1.
- Monitored activities for all the Production related jobs by checking logs and load details. Resolved issues related to long running jobs by implementing the performance tuning techniques.
- Created and enhanced Mappings based on Business requirement to load data from Source system into the target system.
- Designed and developed mappings to in corporate the existing SQL code logic into Informatica transformations like source qualifier, sorter, aggregator, expression, joiner, dynamic lookup and connected and unconnected lookups, filters, Sequence Generator, Router and update strategy.
- Extensively worked on developing and debugging mappings, mapplets, sessions and workflows.
- Worked on creating Unit Test Cases for mapping and code reviews.
- Worked with PMCMD to interact with Informatica Server from command mode and execute the Shell scripts.
- Participated in weekly status meetings and conducting internal and external reviews as well as formal walkthroughs among various teams and documenting the proceedings Production Support post go-live.
Environment: Informatica 8.6, Oracle 11g, XML, SQL Server 2008, Unix.
ConfidentialApplication Developer
Responsibilities:
- Understanding the business requirements and preparing the technical specification document using the function specification document provided by the Client team.
- Extensively used Transformations like Router, Aggregator, Joiner, Expression and Lookup, Update strategy and Sequence generator and Filter.
- Created Informatica Mappings, Sessions including Command tasks like Event Wait, Event Raise, and Timer and assignment workflows on business requirements.
- Doing System Testing and coordinating UAT with other vendors.
- Bridge triaging with customers and other stakeholders.
- Responsible for ETL design, development, testing and review of code along with support in documents like Unit test case and technical handover documents.
- Worked on bug fixes on existing Informatica Mappings to produce correct output.
- Managing Files in UNIX and Used FTP to transfer the files between UNIX and Windows. Prepared Data Mapping Documents and designed the ETL jobs based with required Tables in the Dev Environment.
- Developed and maintained ETL (Extract, Transformation and Loading) mappings to extract the data from multiple source systems like Oracle, SQL server and Flat files and loaded into Oracle.
- Worked with Data Profiling Team in analysing the source systems data for duplicative and quality issues.
- Usage of Constraint-based loading to load Target tables.
- Prepared ETL standards, naming conventions and process flow documents.
- Administered the repository by creating folders and logins for the group members and assigning necessary privileges.
Environment: Informatica 8.6, Oracle 11g, XML, SQL Server 2008, Web services, Unix, Cognos 8 (Reporting Tool), Control-M, and Erwin.
ConfidentialTechnical Analyst
Responsibilities:
- Provided Support for 18 applications in the Production arena. Worked on BMC Remedy tool ticket management and delivered solutions to users on time.
- Production activities in Business objects, Kronos, Cass, and other applications as a major part of the supported list of applications.
- Interacted with users directly to collect required information for analysis and provided on-time resolutions for low and medium priority issues.
- Created RFCs for code migration and change of code whenever required.
- Worked on Confidential Solar3 project for Data migration activities.
Environment: BMC REMEDY TOOLv8, Sql server2008, MS Access 2007.