We provide IT Staff Augmentation Services!

Sr. Etl Developer, Bigdata Developer Resume

3.00/5 (Submit Your Rating)

San Jose, CA

SUMMARY

  • 12+ years of experience in IT, which includes experience in Data warehousing, ETL, Big Data Technologies, Hadoop ecosystem, SQL related technologies in Financial, Capital Markets and Insurance Domains.
  • 10+ years of experience wif Data Warehousing using Informatica Power Center 8.x/9.x/10.x as an ETL tool and Oracle 10g/11g, Teradata, SAP HANA & DB2 as relational databases.
  • Experience on BIG DATA using HADOOP framework and related technologies such as HDFS, HBASE, MapReduce, Yarn, HIVE, SQOOP and SPARK.
  • Experienced in working wif different data sources like Flat files, SQL Server, Oracle, DB2, Teradata and SAP HANA.
  • Experienced in Teradata BTEQ Scripting.
  • Experienced in Python programming language.
  • Experience in Big Data and Analytics field in Storage, Querying, Processing and Analysis for developing E2E Data pipelines. Expertise in designing scalable Big Data solutions, data warehouse models on large - scale distributed data, performing wide range of analytics.
  • Experience in integrating Hadoop wif Informatica and working on teh pipelines for processing data.
  • Experienced in Apache SPARK.
  • Good noledge on Spark and SparkSQL, SparkStreaming, SparkGraphX, SparkMlib.
  • Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
  • Experience in using PL/SQL to write Stored Procedures, Triggers, Functions, Database Performance Tuning, Partitioning and Query optimization techniques.
  • Created and updated Shell scripts for file processing and to execute workflows using UNIX scripts.
  • Experience in developing scripts using Shell and Python for system management and for automating routine tasks.
  • Good Knowledge on Cloud technology - Microsoft Azure Fundamentals.
  • Experience in Data Analysis, Data profiling, Data modelling and Project Estimation.
  • Excellent analytical, problem solving and interpersonal skills. Ability to learn new technologies and methodologies quickly. Consistent team player wif excellent communication skills.
  • Proficient in analyzing and translating business requirements to technical requirements and architecture and data models.
  • Roles played: ETL developer, Bigdata Developer, Teradata developer, offshore team Lead and Data Analyst.

TECHNICAL SKILLS

ETL Tools: Informatica Power Center 8.x/9.x/10.x, Informatica Data Quality 9.x

Bigdata technologies: HDFS, MapReduce, Hive, Sqoop, Apache Spark

RDBMS: Oracle 10g/11g, SQL Server, DB2, Teradata, SAP HANA

Languages: SQL, PLSQL, HiveQL, Python, Pyspark

Schedulers: Autosys, Control-M

Tracking Tools: Quality Center, JIRA, Incidents Management

Operating Systems: Windows, UNIX

PROFESSIONAL EXPERIENCE

Confidential, San Jose, CA

Sr. ETL Developer, Bigdata Developer

Responsibilities:

  • Requirement gathering, preparing impact analysis, functionalandtechnical specification documents. Preparing highanddetail level design, engage in codingandunit testing of large-scale data processing source code.
  • Analyzing data and profiling of teh source data for teh project requirement.
  • Design of data model to support, cleanandtransform data for usage in business intelligence platform.
  • Implement data warehouse solutions using informatica by creating data modelsandtroubleshooting performance issues for customizingandtuning teh ETL jobs.
  • Build back-end ETL componentsandsolutions using Informatica tools. Create database objects like tables, views, indexesandstored Procedures in Oracle, SAP HANA.
  • Develop Bigdata pipelines for new subject areas coming into existing systems using SQL, Hive SQL, UNIXandother Hadoop tools.
  • Create Hadoop Sqoop scripts to read, loadandretrieve data from HadoopandSAP HANA.
  • Extractandanalyze data, patterns,andrelated trends from different sources like Oracle, SAP HANA, Hadoop, flat files as per business requirement.
  • Interact wif businessandtechnical team to understand, investigate,andfix data related issues. Participate in data remediation activities to fix business issues.
  • Create UNIX shell scripts to readandload data from flat files into HANA tablesandsend alerts on completion of ETL loads.
  • Develop data quality checks, data profiling, validationandOLAP pre-processing on big data.
  • Perform code review, coordinate user acceptance testing, project demonstrationandimplementation. Also ensure timely delivery wif best practices of automation.
  • Implement industry standard tools & platforms to reduce maintenance, enhancement,andproduction support efforts in distributed data processing applications.

Environment: Informatica Power Center 9.6/10.2, Oracle 11g, SAP HANA, Teradata SQL Assistant, SQL, PL/SQL, HDFS, Hive, Visio, UNIX, Shell Scripting, Control-M

Confidential, San Jose, CA

Bigdata Developer

Responsibilities:

  • As a Hadoop developer and module lead, worked closely wif teh business system analysts to understand teh requirements and translate teh requirements into development to create teh required EDW data pipeline on teh Hadoop (Horton Works) platform using teh existing Python Framework.
  • Build python wrappers for project execution automation.
  • Contributed teh development of common framework, which is going to be used across all Bigdata applications.
  • Developed Hive scripts as part of teh project using several Hive pre-defined UDAFs - exclusively using collect functions such as posexplode, explode, collect set, etc.
  • Monitor teh progress of teh Hive jobs using teh resource manager web UI.
  • Clean, transform and analyze vast amounts of Raw data from various systems using spark to provide ready-to-use data to teh all developers and business analysts.
  • Used Shell scripting extensively to create teh pipeline handshakes and validation scripts.
  • Developed hive scripts embedded in Shell scripts to do Unit testing and to run system integrated checks/validations for several of teh test cases, developed, written, and as accepted/approved by teh project management team (PMO).
  • Used Sqoop to export data from Hadoop to SAP HANA database (SAP - Bank Analyzer).
  • Developed data load projects (batch jobs - ETL) and monitored teh daily jobs, fix for any issues, and ensure teh job completion to meet SLA.
  • Used UC4 Scheduler to schedule teh daily batch loads.
  • Worked wif offshore team, guide them through teh project/Sprint scope and get teh deliverables done in a timely manner.

Environment: Hadoop, Hive, HDFS, UC4, Unix, Python, Informatica power center 10.6, Red Shift.

Confidential, San Jose, CA

Sr. ETL Developer and Project Lead

Responsibilities:

  • Teh Major objective was to understand teh system landscape both functionally and technically and tan gather teh Requirements and propose an optimal solution which agreeable to teh business and syncs wif teh centralized platform and coordinate wif Development team, testing teams and teh SAP teams to ensure that teh data is loaded as per Business requirements E2E.
  • Interacted wif various business SME’s in terms of requirement gathering and preparation of functional specification document.
  • Created ETL mappings using Informatica Power Center to extract data from multiple sources like Oracle Relational Tables, Teradata tables and Flat Files for based on business requirements and loaded to Data warehouse.
  • Responsible for
  • Worked on Analyzing existing Teradata BTEQ Scripts and got teh Requirements for development of Payment Flow module.
  • Revisited and analyzed all teh BTEQ scripts and got logic for Payment flags and developed ETL’s.
  • Responsible for Analysis, Design, development, implementation and testing of Informatica mappings, sessions & workflows.
  • Worked in Teradata and Oracle Databases, developed Oracle SQL Queries and created Teradata BTEQ scripts. used Informatica PowerCenter components, created new SCD1/SCD2 mappings, sessions and workflows, updated existing mappings, sessions and workflows, created and updated tables, views, indexes SAP HANA & Oracle, created and updated UNIX Scripts.
  • Used Control-M for job monitoring, Jobs execution for user’s data load requests, Used HDFS to browse Hadoop files, Used Hive to query tables backed up tables in Hadoop Environment.
  • Worked on Performance Tuning for optimizing teh ETL objects.
  • Worked in Agile methodology, Created User story, tasks and Attended daily Scrum call.
  • Worked on Error Handling and Performance Tuning for optimizing teh ETL objects.

Environment: Informatica Power Center 9.6/10.2, Oracle 11g, SAP HANA, Teradata SQL Assistant, SQL, PL/SQL, UNIX, Shell Scripting, Control-M

Confidential

Senior ETL Developer and Offshore Lead

Responsibilities:

  • Understanding teh Business Requirement specifications and created teh technical design document.
  • Created ETL mappings using Informatica Power Center to extract data from multiple sources like Oracle Relational Tables, based on business requirements and loaded to Data warehouse.
  • Applied Business logics at various levels of teh Project processes like Staging, Rules, Mastering, Post-Mastering and Distribution.
  • Worked directly wif US clients to get teh business requirements and for Technical and Business-related discussions
  • Responsible for Off-shore development tasks and update onsite and clients through meetings and emails. Closely worked wif QA team for testing and got signoff for all teh Applications developed.
  • Worked for Prod and QA support and Fixed many data and technical issues.
  • Responsible for Design, development, implementation and testing of Informatica mappings, sessions & workflows.
  • Created, validated, and executed sessions of mappings and used other tasks like event wait, event raise, decision, email, command and pre/post SQL.
  • Worked mostly in Oracle Database, developed SQL Queries and used in Informatica transformations and sessions and created and used queries for unit testing.
  • Involved in Unit testing, System integration testing and User Acceptance Testing.

Environment: Informatica power center 9.1, Toad for Oracle, Putty, SQL, PLSQL

Confidential

ETL Developer

Responsibilities:

  • Interacted wif teh Subject matter experts and Domain experts to gather requirements.
  • Created ETL mappings using Informatica Power Center to extract data from multiple sources like DB2 Relational Tables, Fixed-Width and Delimited flat files based on business requirements and loaded to Datawarehouse.
  • Worked directly wif UK and US clients to get teh business requirements and for Technical and domain related discussions.
  • Worked on an End-to-End Application which involves requirement gathering, Technical Design documents, Autosys Design documents, ETL development and testing, Migration to QA and PROD.
  • Created Autosys jobs using Autosys scheduler tool and involved on Prod and QA support and Fixed many data and technical issues.
  • Worked wif business Users and clarified their technical questions on teh Applications I developed.
  • Creating all Project documentations like UTP, Autosys Design document, Migration checklist and Run books for every Production releases.
  • Attended daily Scrum calls wif Clients about teh project tasks and priorities on deliverables. created profiling objects and data quality rules using IDQ, Used Address Doctor in IDQ
  • Migrated objects from DEV to QA and UAT, Raised PROD deployments requests, Analyzed PROD issues, created user requested reports.

Environment: Informatica Powercenter 8.6, IDQ, Autosys Scheduler, DB2, Sybase, Unix, DB2, Quality Center

Confidential

ETL Developer

Responsibilities:

  • Involved in teh design and development for teh single and multisource component like ARC, Incident, Driver, Etc.
  • Working wif frequently changing requirements.
  • Involved in defect fixing in NFTE testing, troubleshooting and closing teh defects on time.
  • Involved in UTP, SIT and Peer reviews and found few bugs.
  • Used Informatica PowerCenter components, created and updated SCD1/SCD2 mappings, sessions and workflows, created and updated tables, views and indexes, created and updated UNIX Scripts
  • Mentored teh fresher’s who joined in teh Project.

Environment: Informatica power center 8.6, Toad for Oracle, Putty, Unix

Confidential

Software Developer

Responsibilities:

  • Understanding teh Business Requirement specifications and created teh technical design document.
  • Responsible for Design, development, implementation and testing of Informatica mappings, sessions & workflows.
  • I have changed 200+ existing PDW mappings to adopt teh new changes and exposed teh same to teh Business.
  • Created, validated, and executed sessions of mappings and used other tasks like event wait, event raise, email, command and pre/post SQL.
  • Worked on UNIX commands, Scripts and Environment.
  • Having experience in Oracle Database, developed SQL Queries and used in Informatica transformations and sessions.
  • Involved in creation of Unit Test cases and executed test cases in Quality Center.
  • Involved in Unit testing, System integration testing and User Acceptance Testing.

Environment: Informatica power center 8.6, Toad for Oracle, Putty, Oracle 9i, SQL

Confidential 

Software Engineer

Responsibilities:

  • We have connected to 300+ applications in Confidential systems which ride on different databases like Oracle, Microsoft SQL Server, Sybase and DB2.
  • Extraction of data from Different Data Sources and Implemented ETL solutions using Informatica and store it in STARS data mart.
  • Created reusable windows DOS scripts for Bulk upload, Threshold logic, Automating FTP scripting and File backup scripts for Flat file sources and feed monitor mechanism.
  • Created reusable UNIX shell scripts for PARAM file validation and Target File validation script which is used for 300+ STARS applications.
  • Created control-M scheduling jobs for STARS Informatica jobs.
  • Responsible for Design, development, implementation and testing of Informatica mappings, sessions & workflows and production support.
  • Understanding teh Business Requirements and Extensively involved in Data Extraction, Transformation and Loading (ETL process) from Source to target systems using Informatica.
  • Have worked on XML sources, Flat file and database sources.
  • Proficient in using Informatica Mapping designer, Workflow manager, Workflow monitor to create, schedule and control workflows, tasks, and sessions.
  • Created scheduling jobs for Informatica using Control-M scheduler.

Environment: Informatica Power Centre 8.1, 0racle 9i, 10g, SQL *plus, Rapid SQL, IBM DB2 client, Control-M

We'd love your feedback!