Senior Etl Developer/ Snowflake Developer Resume
Pataskala, OH
SUMMARY
- Senior IT professional with over 8+ years of experiencein information technology, with specific focuson Data Architecture, Data Modeling, Data Conversion,Testing, design & implementation of large scale data warehouse, creation of data marts on various databases & operating systems.
- Experience in different project methodologies (SDLC) like Agile / Waterfall model.
- Extensive testing ETL experience using Informatica Power Center (Designer, Workflow Manager,Workflow Monitor and Server Manager).
- Experience of Relational Modeling and Dimensional Data Modeling using Star & Snow Flake schema, De normalization, Normalization, and Aggregations. Designed Data bases using Erwin 4.5
- Hands on experience in AWS cloud services like S3, EMR, EC2, Lambda, Cloud Watch, Athena, RDS, Airflow, and Redshift.
- Experience in writing UNIX Korn shell scripting
- Innovative in Approach, enjoys learning new methods and ideas and putting them into daily practices.
- Strong experience in migrating other databases to Snowflake
- Participates in the development improvement and maintenance of snowflake database applications.
- Experience in creating end user reports using SAP BI reports.
- Good understanding of Data warehouse concepts and principles (Kimball/Inman) Star Schema,Snowflake, Slowly Changing Dimensions (SCD), Surrogate keys, Normalization/De normalization.
- Experiencing in Designing and developing a new Redshift data warehouse.
- Extensive experience writingDatabase Triggers, Stored Procedures, FunctionsandPackagesusing PL/SQL.
- Involving in writing complex SQL queries to verify data from Source to Target
- Experience in migrating existing databases from on premise to AWS Redshift using various AWS services
- Experience in creating and modifying forms usingOracle Forms 9ifor data manipulation purpose and reports usingOracleReports.
- Data Migration usingSQL *Loader and TOADUtility. Experience in usingData warehousing toolslikeInformaticaforETL process
- Performed ETL procedure of source system analysis, data transformation and load data from different sources into data marts and Data Warehouse.
- Experience on different data sources such as Flat files, XML files and Excel filesfor the development of mappings
- Extensive testing experience usingInformatica Power Center, Big Data Management and Performance tuning.
- Expertise in Creating, Debugging, Scheduling and Monitoring jobs usingApacheAirflow
- Extensive experience with LoadUtilities Multiload, BTEQ, Fast Load, Fast Export, TPT of Teradata.
- Define virtual warehouse sizing for Snowflake for different type of workloads.
- Develop highly scalable, fault tolerant, maintainable ETL data pipelines to handle vast amount of data.
- Good Team player with excellent communication and problem solving skills.
TECHNICAL SKILLS
Operating Systems: Windows XP, 7,8,10, UNIX
Languages: SQL, PL/SQL, Python, Unix, Shell
Data Modeling Tools: Erwin and Ms Visio
OLAP Tools: SAP BI 4.2 SP06
ETL Tools: Informatica Power Center, Informatica BDM, Airflow
Database Tools: Oracle 9i/10g/11g/12c, ERP, SQL Server, DB2, DB2, Ms - Access, Teradata AWS S3 and AWS Redshift, Snowflake
AWS Environment: S3, EC2, Lambda, RDS, Redshift,DynamoDB, Cloud Watch
Web/Apps Server: IBM Web Sphere, IIS andTomcat.
Methodologies Tools: Waterfall, Agile and Scrum methodologies
Tools: JIRA, HP ALM, HP Quality Center, Autosys, IBM Clear case
PROFESSIONAL EXPERIENCE
Confidential, Pataskala, OH
Senior ETL Developer/ Snowflake Developer
Responsibilities:
- Used various Informatica transformations like Aggregator, Joiner, Filter, and Sorter to improve the ETL performance.
- Work with internal teams to create the migration process of Redshift to Snowflake
- Created reports on Metabase to see the Tableau impact on snowflake in terms of cost.
- Participated in sprint planning meetings, worked closely with manager on gathering the requirements.
- Created tables and views on snowflake as per the business needs.
- Involved in Data Modeling the Dimensional and FACT tables E-R relationship.
- Effectively used Informatica’s BULK loading and parallelism strategy to improve the ETL data loads.
- Developing ETL pipelines in and out of data warehouse using Snowflakes SnowSQL Writing SQL queries against Snowflake
- Designed and implemented efficient data pipelines (ETLs) in order to integrate data from a variety of sources into Data Warehouse.
- Performed data quality issue analysis using SnowSQL by building analytical warehouses on Snowflake.
- Implemented data intelligence solutions around Snowflake Data Warehouse.
- Creating Reports in Looker based on Snowflake Connections.
- Implemented Data Quality framework using AWS Athena, Snowflake, Airflow and Python.
- Developed reusable transformations and mapplets maximize the ETL load performance.
- Implemented parallelism and dynamic caching at Informatica session level to reduce the load durations
- Worked on SnowSQL and Snow pipe
- Unit tested the data between Redshift and Snowflake.
- Worked with Interface teams on Snowflake connectivity with reporting layers at SAP BO & Tableau
- Developed complex Stored Procedures for automation of ETLprocess and data capture from Oracle EBS to ODS and DM tables.
- Developed microservice on boarding tools leveraging Python and Jenkins allowing for easy creation and maintenance of build jobs and Kubernetes deploy and services.
- Develop alerts and timed reports Develop and manage Splunk applications.
- Migrate the database 500 + Tables and views from Redshift to Snowflake.
- Redesigned the Views in snowflake to increase the performance.
- Developed Python scripts to take backup of EBS volumes using AWS Lambda and Cloud Watch
- Provided 24*7 support for Production Migration.
Environment: Snowflake, SnowSQL, Redshift, SQL server, AWS, Redshift, EC2, S3, Athena, Lambda,Cloudwatch, Airflow, Talend, Jenkins, Kubernetes, SQL, Informatica Power Center, BDM, Oracle 12c, PL/SQL (Stored Procedures), Python, UNIX.
Confidential, Blue Ash, OH
Senior ETL Developer/ Snowflake Developer
Responsibilities:
- Designed, implemented and maintained ETL platforms.
- Coordinating with DBA team to design datasets, tables, views in data sources like Oracle, SQL server&DB2.
- Create Technical Specification documents for ETL Mappings
- Created roles and access level privileges and taken care of Snowflake Admin Activity end to end.
- Design and customize data models for Enterprise Data warehouse supporting data from multiple sources.
- This includes Relational and Dimensional Data Modelling using Star & Snow Flake schema, De normalization, Normalization, and Aggregations.
- Retrofitted 500 Talend jobs from SQL Server to Snowflake.
- Developing mappings to extract data from DB2, SQL Server, Oracle, Flat files, and load into Data warehouse using the Mapping Designer.
- Migrate the database 500 + Tables and views from Redshift to Snowflake.
- Redesigned the Views in snowflake to increase the performance.
- Provided proper resolutions for ETL related issues.
- Maintained documentations for ETL processes.
- Creating Reports in Looker based on Snowflake Connections.
- Validation of Looker report with Redshift database.
- Reporting testing - creating test cases, scripts and executing them. Tested the fixed reports in SAP BI environment as well as the Performance of the reports.
- Created data sharing out of snowflake with consumers.
- Worked on replication and data mapping of ODS tables to Guidewire ClaimCenter typelists and entities.
- Validating the data from SQL Server to Snowflake to make sure it has Apple to Apple match.
- Consulting on Snowflake Data Platform Solution Architecture, Design, Development and deployment focused to bring the data driven culture across the enterprises.
- Worked on SnowSQL and Snowpipe
- Driving replacing every other data platform technology using Snowflake with lowest TCO with no compromise on performance, quality and scalability.
- Created Airflow Scheduling scripts in Python
- Unit tested the data between Redshift and Snowflake.
- Creating scripts for system administration and AWS using languages such as BASH and Python.
- Responsible for Continuous Integration (CI) and Continuous Delivery (CD) process implementation- using Jenkins along with Python and Shell scripts to automate routine jobs.
- Created Pre-commit hooks in Python/shell/bash for authentication with JIRA-Pattern Id while committing codes in SVN, limiting file size code and file type and restricting development team to check-in while code commit.
Environment: Informatica Power Center, Snowflake,SnowSQL, AWS, Redshift, S3, Lambda, Airflow, SQL server, TALEND, JENKINS, SQL, Python, Shell, Bash Scripting, Jira
Confidential, NYC, NY
ETL Informatica Developer
Responsibilities:
- Analyzed the Business requirements and documented the technical specification documents.
- Provide level 1 support for the production jobs and monitored them. Ensure that the ETL jobs get completed as per SLA.
- Designed tables and columns in Redshift for data distribution across data nodes in the cluster keeping columnar database design considerations.
- Wrote PL/SQL procedures which are called from Stored Procedure transformation to perform database actions such as truncate the target before load, delete records based on a condition and rename the tables.
- Wrote Redshift UDFs and Lambda functions using Python for custom data transformation and ETL.
- Worked with UNIX shell scripts extensively for job execution and automation.
- Performed data profiling & cleansing by using IDQ and migrated objects from IDQ to PowerCenter.
- Developed naming convention document for various component of Informatica.
- Designed Sources to Targets mappings from SQL Server, Excel/Flat files to Oracleusing Informatica Power Centre.
- Used AWS Redshift, S3, Spectrum and Athenaservices to query large amount data stored on S3 to create a Virtual Data Lake without having to go through ETL process.
- Worked closely with the ETL SSIS Developers to explain the complex Data Transformation using Logic.
- Assisted with implementation/upgrade of AutoSys, troubleshoot errors, and documentation/procedures.
- Created various Oracle databaseSQL, PL/SQL objects like Indexes, stored procedures, views and functions for Data Import/Export.
- Provided seamless connectivity between BI tools like Tableau to Redshift endpoints.
- Installation and configuration of Informatica Server and Informatica Repository Server in Windows-UNIX Operating Systems.
- Data Warehouse Data modeling based on the client requirement using Erwin (Conceptual, Logical and Physical Data Modeling).
- Have done POC on Redshiftspectrum to create external tables by using S3 files.
- Involved in logging defects in JIRA and monitor the progress until it goes to UAT environment.
Environment: Informatica Power Center, AWS, Redshift, S3, Lambda, PL/SQL, Oracle 11g, SQL Server, AutosysErwin, Unix, Shell Scripting,Jira
Confidential, Bentonville, AR
ETL/ Informatica Developer
Responsibilities:
- Developed Logical and Physical data models that capture current state/future state data elements and data flows using Erwin 4.5.
- Data Migration from oracle to redshift using SCT and DMS.
- Extracted Data from various sources like Data Files, different customized tools like Meridian and Oracle.
- Extensively worked on Views, Stored Procedures, Triggers and SQL queries and for loading the data (staging) to enhance and maintain the existing functionality.
- Create, modify and execute DDL in table AWS Redshift tables to load data.
- Done analysis of Source, Requirements, existing OLTP system and identification of required dimensions and facts from the Database.
- Developed and executed a migration strategy to move Data Warehouse from an Oracle platform to AWSRedshift
- Designed the Dimensional Model of the Data Warehouse Confirmation of source data layouts and needs.
- Extensively used Oracle ETL process for address data cleansing.
- Developed and tuned all the Affiliations received from data sources using Oracle and Informatica and tested with high volume of data.
- Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Oracle and Informatica PowerCenter.
- Reviewed high-level design specification, ETL coding and mapping standards.
- Designed new database tables to meet business information needs. Designed Mapping document, which is a guideline to ETL Coding.
- Performance tuning the tables in Redshift.Reviewing the explain plan for the SQLs in Redshift.
- Used ETL to extract files for the external vendors and coordinated that effort.
- Performed Unit Testing and tuned for better performance.
Environment: Informatica Power Center, AWS, Redshift, OLTP, Oracle 10g, PL/SQL, Erwin, Windows
Confidential, Arlington, VA
ETL/Oracle Developer
Responsibilities:
- Gathered requirements from Business and documented for project development
- Installation, Creation and support ofOracledatabase environments.
- Created data maps in Informatica to extract data from Sequential files
- Coordinated design reviews, ETL code reviews with teammates
- Creating table spaces, tables, views, scripts for automatic operations of the database activities.
- Data conversion from flat file to intermediate tables usingSQL*Loader, Data mapping
- Created control files forSQL*Loader.
- Worked with InformaticaPower Center for data processing and loading files.
- Extensively worked with Informatica transformations.
- Designed/developed tables, views, various SQL queries, stored procedures, functions.
- Extensively worked on UNIX Shell Scripting for file transfer and error logging.
- Monitor database and system backups, developed Oracle Stored procedures, functions, packages, and triggers that pull data for reports.
Environment: Informatica Power Center, PL/SQL, Oracle 9i/10g, SQL, SQL*Loader, Unix, Shell Scripting, Windows