We provide IT Staff Augmentation Services!

Redshift Lead Resume

PA

SUMMARY

  • 8+ Years of Experience in analysis, design, development, implementation and troubleshooting of Data Warehouse applications.
  • Experience working in Data warehousing, Business Intelligence systems,OLAP environments.
  • Experiencing in Designing and developing a new Redshift data warehouse.
  • Good noledge on AWS key services like Amazon Redshift, Amazon S3, DMS, Athena, Glue, Kinesis, EMR, SNS, Amazon EC2, Data Pipeline, Amazon Lambda, Amazon CloudWatch and Amazon Glacier.
  • Good experience in performing on - prem migration to AWS cloud
  • Designed, developed and implemented datalake on S3
  • Experience in identifying and utilizing AWS services wherever needed
  • Good experience in writing complex queries on teh tables dat contains billions of records using Redshift SQL to transform and load data to Redshift database.
  • Experience in utilizing different design and coding techniques to improve teh performance of teh queries on Redshift Database.
  • Experience in working on Redshift cluster and taking teh snapshots on it and using those snapshots for restoring teh data.
  • Experience on working with Amazon management console, to setup and manage instances through Amazon EC2 console, and manage teh clusters and snapshots using Amazon Redshift console.
  • Experience in implementing performance optimization techniques.
  • Designed and developed ETL to load data from various sources like DB2, Oracle, Flat files and MS SQL Server into Amazon Redshift, Oracle, Netezza and SQL server targets.
  • Experience in creating Python scripts to extract and load data from flat files like CSV, JSON to Redshift/S3 and also moving data from different applications using APIs
  • Experience working on Informatica Cloud to extract and load data to Redshift and/or S3.
  • Extensively worked on PowerCenter Client tools - Designer, Repository manager, Workflow manager, Workflow monitor.
  • Good experience in creation of mappings, sessions, mapplets, tasks and workflows using Power Center tools to move data from multiple sources into target areas.
  • Experience in working with BI reporting tools such as Tableau, Business Objects and Crystal reports.
  • Hands on experience in identifying and resolving performance bottlenecks.
  • Worked on OLAP databases with different kinds of tables like dimensional, historical and aggregator and on OLTP databases.
  • Experience in SQL programming using tools such as Aginity, Toad, SQL Plus and SQL Developer.
  • Good in UNIX Shell scripting. Developed UNIX scripts using PMCMD utility and scheduled ETL load using utilities like CRON tab, Control-M, Automic(UC4) and Autosys.
  • Experience working on Github and PVCS version Manager to host and maintain teh project related work.
  • Involved in all phases of data warehouse project life cycle with good exposure to Development, Testing, Debugging, Implementation, Documentation, End-user training and Production support.
  • Good Knowledge on Normalization and De-normalization concepts and design methodologies like Ralph Kimball and Bill Inmon’s Data Warehouse methodology.
  • Knowledge on execution of jobs in Mainframe and understandable noledge of JCL code.
  • Performed Unit Testing, Integration Testing, System Testing and Data Validation during various stages of teh project.
  • Documented design procedures, mapping specifications, operating instructions, test plans/procedures and troubleshooting procedure for ease of application maintenance.
  • Developed TEMPeffective working relationships with client team to understand support requirements, develop tactical and strategic plans to implement technology solutions, and TEMPeffectively manage client expectation.
  • Worked in offshore and on-site model, and acted as an on-site coordinator for many of teh projects and directly interacted with teh Client and teh Business.
  • An excellent team member with an ability to perform individually and as team member with good interpersonal relations, strong communication skills, hardworking and high level of motivation.

TECHNICAL SKILLS

AWS Services: Amazon Management Console, AWS CLI, Amazon Redshift, Amazon S3, Amazon DMS, Amazon Kinesis, Amazon Athena, Amazon EMR, AWS Glue, Amazon EC2, Amazon SNS, Amazon Data Pipeline, Amazon Lambda, Amazon Glacier, Amazon Cloudwatch, Amazon IAM, Machine Learning

Data warehousing Tools: SQL Workbench SQL*Plus, Toad, SQL Navigator, Aginity for Netezza and Redshift, Informatica Power Center 9.x/8.x/7.x, Informatica Power Exchange, Informatica Cloud, Tableau, Crystal reports

Databases: Oracle11g/10g/9i, DB2, MSSQLServer2012/2008R2/2005, Amazon RedShift, Netezza 7.0.4, MS Access 2007/2003

Data Modeling: Star Schema, SnowFlake Schema, Kimball Methodology, Bill Inmon’s Data Warehouse methodology

Programming: SQL, PL/SQL, Redshift SQL, Unix Shell Scripting, Python.

Environment: UNIX (SunSolaris10, AIX 6.1, Linux 6.5), Windows 2008/07/XP

PROFESSIONAL EXPERIENCE

Confidential, PA

Redshift Lead

Responsibilities:

  • Performed POCs and implemented teh solutions to move Confidential environment from on-prem to AWS cloud.
  • Created Data lake on S3 which acts as central repository for all Confidential related data
  • Implemented various ETL and ELT flows to move data from different sources to S3 and Redshift
  • Used DMS and Schema Conversion Tool to move data from data sources like Oracle and SQL server to S3
  • Created, configured teh Redshift scalable clusters dat supports teh data warehouse and created databases on teh clusters
  • Create table structures with Distribution Keys, Sort Keys and encoding techniques to improve teh performance for data retrieval on AWS Redshift.
  • Designed teh tables and loaded teh data to teh Redshift databases using Amazon best practices.
  • Designed and developed tables in teh Data warehouse based on Star Schema Model.
  • Implemented performance enhancing techniques on Redshift tables to ensure teh jobs complete in schedule time.
  • Created AWS Data pipeline jobs to perform transformations on S3 data and load it to Redshift and/or S3
  • Implemented mechanism to convert CSV files to Parquet on S3 bucket to reduce space, improve data retrieval speed and decrease cost while retrieving data using AWS Athena
  • Created and used EMR clusters to implement data conversions from CSV to Parquet and automated them using data pipeline
  • Created Python scripts to pull marketing and campaign data from Eloqua application.
  • Created Python scripts, APIs and AWS Kinesis to pull data from Twitter, Facebook and Google to S3
  • Created Cloud Watch dashboards for Events, Alarms and SNS notifications
  • Created External tables on Athena and Redshift spectrum for teh users to query on S3 data
  • Worked on AWS Glue to perform some ETL operations
  • Implemented Version control, Life Cycle Management and archival on S3 buckets
  • Created and streamlined IAM polices to enable complete control on AWS for teh users
  • Working on POC on how AWS Machine Learning can provide predictions based on available data
  • Converted Informatica mappings to SQL scripts and used them in Data pipeline jobs to do teh transformations
  • Used synchronization tasks and mappings on Informatica Cloud to move data from Salesforce to S3
  • Prepared Master Document dat shows complete flow of data and tools and services dat are used as part of this data flow
  • Presented POC sessions and provided noledge transfer sessions on AWS services to teh Client

Environment: Amazon Management Console, Amazon Redshift, Amazon DMS, Amazon Athena, EMR, Kinesis, Glue, Amazon EC2, AWS Athena, Amazon S3, SNS, Amazon Glacier, Amazon Lambda, Amazon Data Pipeline, AWS Machine Learning, Aginity workbench for Redshift, SQL Workbench for Redshift, Informatica Cloud, Informatica Power Center, Python, Power BI, Linux 6.5

Confidential, Atlanta, GA

BI Redshift Developer

Responsibilities:

  • Working and interacting with business to gather all teh requirements for teh development activities.
  • Designing teh tables and loading teh data to teh Redshift data warehouse using Amazon best practices.
  • Designed and developing tables in teh Data warehouse based on Star Schema Model.
  • Creating, configuring and supporting teh Redshift scalable clusters dat supports teh data warehouse.
  • Working on Amazon management Console to manage teh services like Amazon Redshift, Amazon S3, SNS, Data Pipeline, Glacier and Lambda.
  • Create table structures with Distribution Keys, Sort Keys and encoding techniques to improve teh performance for data retrieval.
  • Creating new process flow using Redshift SQL scripts to extract transform and load teh data from different sources to Amazon Redshift database.
  • Configuring, modifying and managing teh pre-prod clusters of Amazon Redshift database as per teh management requests.
  • Taking teh snapshots and using those snapshots for restoring teh data and creating other clusters.
  • Resolving teh performance issues using various performance tuning techniques on teh Redshift Database.
  • Creating Data pipeline process using python to load data from S3 to Redshift and transfer of files between different applications.
  • Created Tables and Views in Redshift dat involves complex logic and are used for reporting purpose of teh business.
  • Using commands like unload and copy to move data between different Redshift Databases and used table STL LOAD ERRORS to fix teh data copy issues.
  • Implementing performance enhancing techniques on Redshift tables to ensure teh jobs complete in schedule time.
  • Setting up Events, Alarms and SNS notifications and responding to them when required.
  • Worked on SNS to create and publish topics and creating subscription for various users and groups to those topics.
  • Creating AWS Data Pipeline jobs to process data between S3 and Redshift and execute different sql statements as per teh requirements.
  • Archiving of old data and files to Amazon Glacier and retrieving when necessary.
  • Created Email notifications using Amazon Lambda and Python, dat will invoke SNS and send email to end users
  • Creating scripts to create new tables, views, queries for new enhancement in teh application using SQL Workbench and Aginity Workbench for Redshift.
  • Working on Informatica Cloud, to develop mappings and tasks to transfer data from sources like Salesforce, Flat Files to Redshift Database and S3.
  • Re-coded teh existing SQLs in to speed up teh run time dat helped business to generate reports much faster than before.
  • Work with Business users to create workbooks with worksheets, interactive dashboards related to Forecast, Budget, actuals and daily sales data in Tableau.
  • Worked in Tableau environment to create dashboards like monthly, daily reports using tableau desktop & publish them to server.
  • Used filters, drilldowns, parameters, blending to make teh views work more efficiently.
  • Created Custom Hierarchies to meet teh Business requirement in Tableau.
  • Enhance and Publish dashboards developed by Business groups to desktops of teh intended users.
  • Working with business in Navigating to resolve data issues faced by teh business on Tableau reports and dashboards.
  • Performing testing at different phases in teh life cycle of teh project.
  • Creating Test documents with Test queries to perform teh testing during pre-production and after teh deployment.
  • Working with teh production team in deploying teh code to production and execution of jobs.
  • Providing support to Advanced Analytics Support whenever they has issues and required help from us.

Environment: Amazon Management Console, Amazon Redshift, Amazon EC2, Amazon S3, SNS, Amazon Glacier, Amazon Lambda, Amazon Data Pipeline, Aginity workbench for Redshift, SQL Workbench for Redshift, Informatica Cloud, Python, Tableau 10, Linux 6.5

Confidential, Bethpage, NY

BI Redshift Developer

Responsibilities:

  • Gathering project requirements and interacting with Business Users in teh design of technical specification documents.
  • Worked with Business and other teams to resolve all teh bottlenecks and free up teh process of data migration from Netezza to Redshift.
  • Converting teh existing Informatica mappings to PostgreSQL queries for teh data migration from Oracle and Netezza to Amazon Redshift.
  • Experience in creating tables structures with Distribution Keys, Sort Keys and encoding to improve teh performance for data retrieval.
  • Creating new process flow using PostgreSQL scripts to extract transform and load teh data from different databases to Amazon Redshift database.
  • As Redshift does not support Stored Procedures, converted multiple stored procedures from Oracle and Netezza to Redshift understandable SQL queries.
  • Created shell scripts on Linux environment to execute teh PostgreSQL scripts dat were created.
  • Using commands like unload and copy to transfer data between different Redshift Databases and used table, STL LOAD ERRORS to fix teh data copy issues.
  • Working on S3 bowser, which is an interface dat connects to Amazon S3 bucket during teh data transfer between different databases.
  • Used PSQL in teh SQL scripts to execute teh sql queries from teh Linux servers.
  • Re-coded teh existing SQLs in Redshift to speed up teh run time dat helped to meet SLAs.
  • PerformedSQL and PL/SQL tuningand Application tuning using EXPLAIN PLAN.
  • Used DTT scripts on Linux environment to run teh queries and load teh data to Redshift using Amazon Web Services (AWS).
  • Experience in working on Redshift cluster and taking teh snapshots on it and using those snapshots for restoring teh data.
  • Created new scheduling jobs in Pre-Prod environments using teh scheduling tool AUTOMIC (UC4).
  • Creating scripts to create new tables, views, queries for new enhancement in teh application using Aginity Workbench for Redshift.
  • Developing various Mappings, Mapplets and Transformations as per teh business needs.
  • Re-designing ETL mappings to improve data quality when required.
  • Performing testing at different phases in teh life cycle of teh project.
  • Creating Test documents with Test queries to perform teh testing during pre-production and after teh deployment.
  • Worked on PVCS to move teh code from Pre-Prod to Production environment.
  • Working with teh production team in deploying teh code to production and execution of jobs.
  • Providing support during teh production run and whenever needed.
  • Co-coordinating with offshore team all through teh project and making sure teh project is on track.

Environment: Linux 6.5, Informatica Power Center 9.6, Power exchange 9.6, Oracle 12, Netezza, Amazon Redshift Database/Cluster, Amazon S3 bucket, S3 bowser, Amazon Web Services (AWS), Amazon EC2, SQL/PostgreSQL, Toad, Aginity workbench for Redshift, Aginity Workbench for Netezza, Automic (UC4).

Confidential, Chicago, IL

Informatica Developer

Responsibilities:

  • Worked as a Business analyst and Informatica developer, responsible in gathering Requirements and IT review. Interacted with Business Users in teh design of technical specification documents.
  • Involved in creating logical and physical data models using MS Visio based on business requirements.
  • Worked on data warehouses and datamarts, which are built on Kimball Methodology and uses star and snowflake schema.
  • Worked on many Business need requests (BNR) and on many small and large projects.
  • Agile approach with daily scrum meetings is being followed throughput teh development and deployment cycle.
  • Designed and Implemented Tables, Functions, Stored Procedures and Triggers.
  • Created scripts to create new tables, views, queries for new enhancement in teh application using TOAD.
  • Loaded teh tables with data from different Heterogeneous databases.
  • Created indexes on teh tables for faster retrieval of teh data to enhance database performance.
  • Extracted data from Heterogeneous source systems like Oracle, SQL Server and flat files into teh Staging tables and then to teh target databases Oracle and SQL Server.
  • Developed various Mappings, Mapplets and Transformations for datamart and data warehouse.
  • Re-designed ETL mappings to improve data quality.
  • Used Unix Shell Scripting to create scripts for automating teh loading process.
  • Created Stored procedure transformations to populate targets based on business requirements.
  • Responsible for monitoring all teh sessions dat are running, scheduled, completed and failed. Debugged teh mapping of teh failed session.
  • Used Pipeline Partitioning feature in teh sessions to reduce teh load time.
  • Analyzed and Created Facts and Dimension Tables.
  • Used Informatica features to implement Type I, II, and III changes in slowly changing dimension tables.
  • Created Data Breakpoints & Error Breakpoints for debugging teh mappings using Debugger Wizard.
  • Mapplet’s & Reusable Transformations were used to prevent redundancy of transformation usage and modularity.
  • Developed teh pre & post session shell scripts, which will create teh parameter file dynamically.
  • Tuned performance of Informatica session for large data files by increasing block size, data cache size, sequence buffer length and target based commit interval.
  • Created and monitored Database maintenance plans for checking database integrity, data optimization, rebuilding indexes and updating statistics.
  • Extensive use of Persistent cache to reduce session processing time
  • Written documentation to describe program development, logic, coding, testing, changes and corrections.
  • Worked on Control M, by creating teh jobs and executing teh scripts using them
  • Provided support in uploading teh Control M jobs and managing teh scheduling or making changes whenever necessary.
  • Created shell scripts to execute teh SQL scripts created on AIX servers and to automate teh flow of execution or to FTP teh files wherever required.
  • Involved in Unit and Integration testing of Mappings and sessions.
  • Assisted Testing team in creating test plan and test cases.
  • Worked with offshore team, acted as an onsite coordinator for many of teh projects, and was answerable to teh Client.
  • Involved in creating multi-dimensional universe and reports in Business Objects Environment.
  • Designed and developed ad-hoc and customized reports using Crystal reports 2008.

Environment: Informatica Power Center/Exchange 9.1/8.6, Oracle 11g, MS SQL Server 2008, MS Visio, Toad, SQL plus, Business objects XiR3, Crystal Reports 2008, Control-M, BMC Remedy Tool.

Hire Now