SUMMARY:

• Big Data/Informatica developer with 5+ years of work experience in Data Analytics based Software applications in a leading financial institution
• Technical work experience in Hadoop framework, Informatica Powercenter9.x, SQL, Spark, and Datameer
• Experience in dealing with Apache Hadoop components like HDFS, Map Reduce, HIVE, HBase, PIG, SQOOP, OOZIE, and Spark QL.
• Expertise in Creating Hive Internal/External Tables/Views using shared Meta store, writing scripts in HiveQL also data transformation & file processing using Pig Latin Scripts.
• Optimized Hive queries with 20% more performance index
• Experience in MR streaming in Python
• Proficient in NoSql databases like Mongo DB,Cassandra
• Experience in HBase bulk load process to load data to HBase tables using complete bulk load tool
• Experience in Scheduling jobs in tools like Oozie, Control - M and Windows scheduler
• Developed Automation process which can switch between Hive and Spark execution
• Strong Data Warehousing ETL experience of using Informatica 9.1/8.6 Power Center Client tools Mapping Designer, Repository manager, Workflow Manager/Monitor
• Strong understanding of Data warehouse concepts, ETL, Star Schema, Snowflake, data modelling experience using Normalization, Business Process Analysis, Dimensional Data modelling, FACT& dimensions tables, physical & logical data modelling
• Experience in resolving on-going maintenance issues, bug fixes and performance tuning of mappings and sessions.
• Experience in writing Korn shell scripts and configuring SFTP
• Work exposure in software development as Team Leader, Team Member and Onsite Coordinator.
• Experience in configuring SFT tools like STClient, CuteFTP
• Analyzed the Report and prepared presentation for top-level management.
• Experience in Service now - Incident management, Problem management and Change management

TECHNICAL SKILLS:

Operating System: AIX, Windows Server 2008

Software Development methodologies: Waterfall Software Development Methodology, Agile Methodology

ETL tools: Informatica Power Center 9.1/8.6, SSIS

Big data: Hadoop, HDFS, Pig, MR Streaming, Hive,Spark, Oozie

Database: DB2, SQL server, Netezza, Hbase, Mongo DB

Scheduler: Control-M, Windows Scheduler, Oozie

Client Software for DB invocation: Squirrel SQL, TOAD, Aginity Workbench

Languages/ API/ Technologies: SQL, PL/SQL, Korn shell, Pig Latin, HQL, Python

PROFESSIONAL EXPERIENCE:

Confidential

Tech Lead

Responsibilities:

• Analysed data and worked with data scientist on the logistic model to derive coefficients.
• Involved in ETL, Data Integration and Migration from Heterogeneous source platform to data lake(cornerstone)
• Used Sqoop to load data from SQLserver/DB2 to HDFS on regular basis.
• Developed complex Oozie XMLs to schedule HQL batches for co-efficient calculations
• Created/implemented Hive tables with Batch mode execution with incremental logic
• Developed programs to parse the raw data, populate staging tables and store the refined data in partitioned tables
• Written Hive and PIG UDFs
• Loaded the final flagged data to Hbase tables to expose it via a Java API
• Prepared Ad-hoc Hive queries for data analysis to meet the business requirements.
• Created a POC to run the HQL batches in the SparkQL
• Implemented test scripts to support test driven development and continuous integration.
• Loaded the coefficient calculations into Datameer representations
• Documented all the solution architecture, Security interaction and detailed design documents.
• Managed ETL packages and ensured availability of the application to the Business users
• Support the application migration to production and provide support to teams during the post warranty period.
• Working on a POC in SPLUNK for real-time status reporting

Solution Environment: Hadoop 1.2, Hive 0.2, Pig, Oozie, Sqoop

Confidential

Tech Lead

Responsibilities:

• Analysed data and proposed the Business model with a threshold value
• Involved in ETL, Data Integration and Migration from Heterogeneous source platform to staging and then to Hadoop environment
• Used Sqoop to load data from SQLserver to HDFS on regular basis.
• Developed Oozie XMLs to schedule HQL/PIG batches
• Created/implemented Hive tables with Batch mode execution
• Developed programs to parse the raw data, populate staging tables and store the refined data in partitioned tables
• Prepared Ad-hoc Hive queries for data analysis to meet the business requirements.
• Implemented test scripts to support test driven development and continuous integration.
• Experience in managing and reviewing Hadoop log files.
• Interaction with onsite team which includes the client’s technical representatives for project related activities
• Managed ETL packages and ensured availability of the application to the Business users
• Deployed SFT client migrations incompliance with the AXP polices
• Support the application migration to production and provide support to teams during the post warranty period.

Solution Environment: Hadoop 1.2, Hive 0.2, Pig, Oozie

Confidential

Sr. Programmer

Responsibilities:

• Customization and enhancement of Existing packages and flows
• Understanding business requirement and providing solution to the client.
• Involvement in Production assurance for WSDW Projects and SLA Maintenance
• Optimizing the data flows and reporting
• Handling escalations from client.
• Maintaining Quality Standards as per the Company & Clients policy.
• Documenting as per the standards to ensure in compliance with the Internal and External Audits.
• Maintaining the Production data repository in NAS
• Providing access control to the business requests
• Provide technical guidance as needed for the following: Tech Inquiries, Test issues, Production issues.
• Provide consultation support for Impact Requests and analysis
• Provide Knowledge transfer to team members

Solution Environment: SSIS, Webfocus, Windows server 2008, Task scheduler

Confidential

Programmer

Responsibilities:

• Worked closely with the Business Analysts to understand the Data requirements and business rules to be implemented.
• Prepared Technical design document and mapping design document based on the functionality requirement
• Created and maintained Mappings using Mapping Designer to load the data from various sources using different transformations like Aggregator, Expression, Filter, Joiner, Lookup, Router, Sequence Generator, Source Qualifier, and Update Strategy transformations.
• Worked extensively with all the tools in Informatica Powercenter like Repository Manager, Designer, Workflow Manager and Workflow Monitor.
• Experience in writing UNIX scripts to validate and preprocess the files
• Responsible for Unit testing ETL code both for Quality and Quantity and verify data loaded in target by writing SQL queries and document the unit test results.
• Responsible for creating Technical design document (TDD) as part of code handoff to production support team.
• Provided / worked in Production support 24 X 7
• Worked with Sequential files, Flat Files and XML.
• Extensively used SQL tools TOAD, MS SQL Server Management Studio to execute SQL queries in DB2 and Netezza.
• Monitored troubleshoot workflows and sessions for the daily jobs
• Provided the Production support for the production issues by troubleshooting the session logs, bad files, error log tables and debugging the mappings.
• Worked on Informatica object migration with Labels, queries and Deployment groups

Solution Environment: AIX, Informatica Powercenter 9.1, DB2, Netezza, Microstrategy, Webfocus