- 12 years of experience in software industry with good knowledge in Agile development environment. Highly experience in DWH projects and having good knowledge on Data Warehousing Architecture , Test - driven development ( TDD ), Guidewire ETL integration & Bigdata ETL integration.
- An enthusiastic and project-oriented team player with sound leadership and problem solving skills to develop creative solution for challenging customer needs.
- Designed and coded application components in an Agile environment utilizing a test driven development approach.
- Hands on experience in end to end warehouse implementation which includes gathering client requirements, Analysis, Design, Coding and Testing.
- Expert in Informatica tool like designer, workflow manger, workflow monitor, repository manger etc.
- Integrated Hadoop with traditional ETL systems and performed extraction, transformation, and loading of massive structured and unstructured data.
- Optimized existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames.
- Exploring with the Spark improving the performance and optimization of the existing algorithms in Hadoop.
- Converted text files into AVRO and loading into Hive tables.
- Experience in Understanding project issues and ability to work independently or as a part of a team with excellent communication skills and analytical, presentation skills.
- Provided the solution for the design changes, build the informatica data quality mapplets and powercenter mappings for the Customer.
- Have hands on experience in Designing ETL & Data Integration Solutions, Development and Support using Informatica PowerCenter.
- Worked on Data profiling for Data quality process.
- Implemented best practices in DWH projects like naming standards, performance oriented and efficient mapping design.
- Responsible for development and implementation of PL/SQL Procedures, Functions as per client requirement.
- Have experience in adherence to coding standards & best practices using Informatica Power Center.
- Imported and exported data into HDFS/HBase using scoop.
- Responsible for managing tasks and deadlines for onsite and offshore ETL teams
- Worked on Business Objects designer, Webi reports, report testing, report requirement analysis.
- Experience in on impact analysis and map & gas analysis.
- Worked in different role models like Software Quality Analyst, Team Lead and managerial roles.
Hadoop/Big Data: Spark1.6, HDFS, Mapreduce, HBase, Hive,Sqoop, Oozie .
ETL:: Informatica PowerCenter 9.x/8.x/7.x/6.x
Data Quality Tool:: Informatica Developer,IDQ 9.6.1
Virtualization Tool:: Denodo platform 5.5
Languages/Utilities:: PL/SQL, SQL, PERL, Ruby, Python 2.5, Scala 2.11
Reporting Tool:: Business Objects XIR2
Other Software Tools:: TOAD, RubyMine, Harvest, GitHub
Industry Knowledge:: Finance, Aero, Telecommunications, Insurance, Pharma.
Big Data Engineer
- Loaded files from LexisNexis to Hive tables using spark
- Created Sqoop tasks to transfer data from DWH to Hive tables.
- Scheduled Hive tasks using Ooze workflow tasks.
- Designed Hive tables for the various subject areas like Customer driving habits, Driving time etc.
- Importing and exporting data into HDFS/HBase using scoop.
- Designed and created Spark data frames and RDD for the quick data analysis.
- Worked on Hadoop POC in comparison with existing ETL jobs.
- Flat legacy data in to Hive databases .
- Responsible for loading data from UNIX file systems to HDFS. Installed and configured Hive and written Hive UDFs
- Optimizing of existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames.
- Worked with NoSQL databases like HBase in creating HBase tables to load large sets of semi structured data coming from various sources.
- Scheduled Hive jobs and spark jobs using control M
- Prepared technical specifications to develop Informatica ETL mappings to load data into various tables confirming to the business rules.
ETL Architect and ETL Informatica Lead
- Creation and maintenance of Logical and Physical Data Model diagrams.
- Understand business and technical concepts to create optimal data design solutions
- Working in designing, development and data modeling in business intelligence, analytics and data warehousing environment.
- Extensively used Informatica Power Center to load data from Flat Files to DB2, Flat Files to SQL Server, DB2 to XML files, Flat Files to Oracle.
- Involved and develop several complex mappings in Informatica a variety of PowerCenter transformations, Mapping Parameters, Mapping Variables, Mapplets & Parameter files in Mapping Designer using Informatica PowerCenter.
- Provide the post production implementation support and work with Operations team to provide them the KT for continuous Application Support & Maintenance.
- Facilitate any performance tuning activity for Informatica Workflows.
- Work closely with cross-functional teams to design and build data stores, data flows, ETLs and related data management application/systems that serve both internal and external business.
- Hands on interaction, technical evaluation, project support, and the establishment of patterns and standards with the various and multiple emerging data products and technologies.
- Mentor junior associates.
- Created Informatica mappings as per technical specifications.
- Analyzing defects and incidents and document it.
- Worked on database environment setting for System test and Performance test etc.
- Responsible for on call and on call rotation.
- Coordinating Production deployment.
- Working with project teams to implement processes which result in a high degree of data quality and integrity, and assists in the resolution of data quality issues from source systems.