- 10+ years of professional IT experience in Data Warehousing, Data Integration, ETL, Big Data Management and Data Quality Management.
- Currently associated with Confidential as a Senior ETL Big Data Lead.
- Skilled in working with Big Data products and distribution, analyzing data using HiveQL, Hue, Impala, Spark, Cassandra in HDFS.
- Proficiency in utilizing ETL Products Informatica BDM, Talend, IBM DataStage.
- Presently working on a Data warehousing and Analytics Project (using Big Data, Informatica, Python, Spark, Cassandra)
- Good understanding of Azure Cloud, ADF.
- Strong decision - making and interpersonal skills having result oriented dedication towards goals with a passion for learning new concepts.
Applications ETL: Informatica Power center, Informatica BDM, Talend, IBM DataStage
Databases: Hive, Oracle, SQLSever, Cassandra
Programming: Python, Shell Scripting, SQL
Senior ETL BigData Lead
- Working on Informatica BDM 10.2.1, Hive, Unix, Autosys, Python, Spark, Cassandra.
- Implemented multiple projects within strict timelines and also contributed to Deployment automation via UBSDeploy in-house utility.
- Build ETL pipelines and processing using Bigdata to create feeds and load them in DataLake.
- Gave KT to entire team on the process & usage of Informatica BDM and Pyspark.
- Participating in Complete Software development life cycle from gathering and analyzing client's Requirements, Perform software development including: user interface design, prototyping, coding, testing, review, deployment and maintenance to modify existing applications to meet user's needs.
- Design and Develop data warehousing system and all related extraction, transform and loading of data depending on business needs.
- Handled team of 10 members across multiple projects at client site.
- Assessed business rules; performed source to target data mapping; designed, reviewed, implemented and optimized Extract, Transform and Load (ETL) processes.
- Processed multiple sources across domains and created a Bigdata Refinery layer.
- Worked on full stack Bigdata technologies.
- Worked on Agile/ Scrum methodology.
- Performed proof of concepts to find out the most optimized design.
- Direct connect with the client and SME’s for code implementation and data provisioning.
- Collaborated with several client teams to address product related problems.
Senior Software Engineer
- Reviewed design documents and coordinated for timely completion of the project; technical & functional documentation specification.
- Prepared Data Mapping Documents and Design the ETL jobs
- Develop processes for extracting, cleansing, transforming, integrating and loading data into Data Warehouse Databases.
- Developed jobs using various stages like Lookup, Join, Transformer, Dataset, Row Generator, Column Generator, Datasets, Sequential File, Aggregator and Modify Stages.
- Extracted & Processed data of all types (Structured / Semi-structured / Unstructured) from different systems.
- Created ETL jobs with history tracking & logging functionalities.
- Ensured the development of applications to be in accordance with the client’s specifications by organizing internal team meetings for peer-review and unit-testing.