We provide IT Staff Augmentation Services!

Senior Etl Bigdata Lead Resume

SUMMARY

  • 10+ years of professional IT experience in Data Warehousing, Data Integration, ETL, Big Data Management and Data Quality Management.
  • Currently associated with Confidential as a Senior ETL Big Data Lead.
  • Skilled in working with Big Data products and distribution, analyzing data using HiveQL, Hue, Impala, Spark, Cassandra in HDFS.
  • Proficiency in utilizing ETL Products Informatica BDM, Talend, IBM DataStage.
  • Presently working on a Data warehousing and Analytics Project (using Big Data, Informatica, Python, Spark, Cassandra)
  • Good understanding of Azure Cloud, ADF.
  • Strong decision - making and interpersonal skills having result oriented dedication towards goals with a passion for learning new concepts.

TECHNICAL SKILLS

Applications ETL: Informatica Power center, Informatica BDM, Talend, IBM DataStage

Databases: Hive, Oracle, SQLSever, Cassandra

Programming: Python, Shell Scripting, SQL

PROFESSIONAL EXPERIENCE

Confidential

Senior ETL BigData Lead

Responsibilities:

  • Working on Informatica BDM 10.2.1, Hive, Unix, Autosys, Python, Spark, Cassandra.
  • Implemented multiple projects within strict timelines and also contributed to Deployment automation via UBSDeploy in-house utility.
  • Build ETL pipelines and processing using Bigdata to create feeds and load them in DataLake.
  • Gave KT to entire team on the process & usage of Informatica BDM and Pyspark.
  • Participating in Complete Software development life cycle from gathering and analyzing client's Requirements, Perform software development including: user interface design, prototyping, coding, testing, review, deployment and maintenance to modify existing applications to meet user's needs.
  • Design and Develop data warehousing system and all related extraction, transform and loading of data depending on business needs.

Confidential

ETL Lead

Responsibilities:

  • Handled team of 10 members across multiple projects at client site.
  • Assessed business rules; performed source to target data mapping; designed, reviewed, implemented and optimized Extract, Transform and Load (ETL) processes.
  • Processed multiple sources across domains and created a Bigdata Refinery layer.
  • Worked on full stack Bigdata technologies.
  • Worked on Agile/ Scrum methodology.
  • Performed proof of concepts to find out the most optimized design.
  • Direct connect with the client and SME’s for code implementation and data provisioning.
  • Collaborated with several client teams to address product related problems.

Confidential

Senior Software Engineer

Responsibilities:

  • Reviewed design documents and coordinated for timely completion of the project; technical & functional documentation specification.
  • Prepared Data Mapping Documents and Design the ETL jobs
  • Develop processes for extracting, cleansing, transforming, integrating and loading data into Data Warehouse Databases.
  • Developed jobs using various stages like Lookup, Join, Transformer, Dataset, Row Generator, Column Generator, Datasets, Sequential File, Aggregator and Modify Stages.
  • Extracted & Processed data of all types (Structured / Semi-structured / Unstructured) from different systems.
  • Created ETL jobs with history tracking & logging functionalities.
  • Ensured the development of applications to be in accordance with the client’s specifications by organizing internal team meetings for peer-review and unit-testing.

Hire Now