We provide IT Staff Augmentation Services!

Senior Data Engineer (cloud) Resume

4.00/5 (Submit Your Rating)

SUMMARY

  • A strategic leader with over 15 years’ experience in business analysis, project management, design, development encompassing wide range of technologies, solutions, roles and industry verticals. Recent experiences focus on building and deploying analytics solutions on the cloud.

TECHNICAL SKILLS

Databases: AWS Snowflake, PostgreSQL, Netezza, Oracle 7.3/8/8i/9i/10g,11g, Teradata, Microsoft SQL Server, DB2, MySQL

Tools: Informatica 4.7.2/5.1/6.2/7.1/8.5/8.6/9.1 ,InformaticaDeveloper, Informatica B2B, SQL SSIS, Cognos, Brio 6.0, Business Objects, Abinitio, Actuate, TOAD, Microsoft DTS 7/2000, Developer 2K/6i,Oxygen XML editor, Mantas, Syfact,Callidus, Microsoft Word, Excel, PowerPoint, Visio, Project .IBM Infosphere Data Stage

Platforms: Windows 2000/NT Server, Windows 95/98/XP/Vista, HP - UX, Sun Solaris, UNIX, IBM AIX, Linux

Languages: Python, SQL, PL/SQL, Servlets, JSP, Shell Scripting, Perl Scripting

Job Schedulers: AutoSys, Control-m

Version Control: MKS, VSS, PVCS, Informatica, GIT

Web/Application Server: IPLANET/Web logic

Tools: in JPMC: ITSM, ITPAM, Peregrine, Database ARM Automation

PROFESSIONAL EXPERIENCE

Senior Data Engineer (Cloud)

Confidential

Responsibilities:

  • Implementing the strategy to migrate Netezza based analytical systems to Snowflake on AWS
  • Responsible for converting all ETL logic into SQL queries, create INFA mapping to load into Netezza and Snowflake database
  • Lead the team in capturing Auto and Umbrella attribute information for the information architect team
  • Worked on CDQ framework to load data into main tables for Auto and Umbrella subject areas
  • Worked with both Maximized and Auto - Scale functionality, used temporary and transient tables on different data sets.
  • Supported the extended team with mentoring, code reviews, design ideas
  • Provided training on Snowflake database to the team.
  • Design batch cycle procedures using scripts + Control M and developed a job monitoring framework by capturing Control M logs

Environment: Informatica Power Center 10.2/9.x, Netezza, Oracle 12 c, PL/SQL, Unix Shell Scripting, Snowflake 3.32.0, Linux 4.5 (64 Bit), Squirrel, SQL Navigator, Python 3, Tableau, Control M, Erwin, Spark, Scala.

Sr. Data Engineer

Confidential

Responsibilities:

  • Key member of the cloud migration team - responsible for the ETL components using a combination of S3 buckets, Python, etc.
  • Responsible for Snowflake deployment including building and configuring the enterprise Snowflake environment for use by the teams
  • Responsible for building data pipelines in Python and created Snowpipes for continuous data load
  • Build ETL Programs to process data from web, xml files, json files and other source files and databases with focus on handling unexpected termination and unexpected actions.
  • Responsible for creation of tables, views, stored procedures, materialized views.
  • Write UNIX shell scripts & leverage PMCMD commands to pull data from AWS S3 buckets and process the data into datamarts.
  • Empower productivity improvements and data sharing using Erwin for effective model management.
  • Ensure data warehouse and data mart designs efficiently support BI and end user.
  • Analyzed datasets using Hive, Map Reduce and Sqoop to recommend business improvements.
  • Develop coding standards, perform code review, enforce industry standards and best practices around data modeling, PL/SQL development, shell/Perl scripting & other ETL efforts.
  • Responsible for design and development of a data ingestion process to consume large unstructured datasets.
  • Responsible for building Dimensional modeling (Star Schema) of the Data warehouse and to use Erwin to design the business process, dimensions and measured facts.

Environment: Informatica Power Center 10.2/9.x,Talend, SSIS,SSAS,SSRS, Talend, Netezza, MS SQL server, Oracle 12 c, PL/SQL, Unix Shell Scripting, Snowflake 3.32.0, Linux 4.5 (64 Bit), TOAD, SQL Navigator, Python 3,AWS S3, EC2, VPC, Kubernetes, Argo, Qlik view, Qlik sense, Tidal, Erwin, Hadoop, Spark, Sqoop, KNN classification, Decision tree, Random Forest.

Application Lead/Architect

Confidential

Responsibilities:

  • Design implement modules in the Data warehouse using Informatica PowerCenter, Informatica Developer and IBM Infosphere Data Stage. Using Informatica developer created mappings, LDO’s, reusable transformation and Mapplets
  • Design, develop DQ Profiles and Scorecards
  • Manage data access pattern and data consumption and partner with Risk group to ensure that right access control procedures are in place.
  • Analyzed datasets using Hive, Map Reduce and Sqoop. Import data from MySQL to HDFS using Sqoop to load data.
  • Designed Data Flow Diagrams (DFD’s), Entity Relationship Diagrams (ERD's)
  • Review BRD and FSD and Prepare Technical specification documents (TSD).
  • Involved in training/mentoring associates in Informatica.
  • Researched created, reviewed and revised requirements documents and use cases.

Environment: Informatica Power Center 8.x/9.x, SSIS, SSAS, SSRS, MS SQL server, Oracle 11, PL/SQL, Unix Shell Scripting, Perl scripting, Linux TOAD, SQL Navigator, Python, Erwin, Control M, Teradata, Hadoop, Spark.

We'd love your feedback!