Senior Etl Developer/bigdata Consultant Resume
San Jose, CA
CAREER SUMMARY:
- Resourceful 8 years of IT experience in system design, development and maintenance with technical and functional experience in Data Warehousing, ETL tools and Hadoop frameworks.
- Extensively worked on data extraction, transformation, loading with various databases using Informatica, Talend and Big Data.
- Proficient in development of mapping using Talend 5.x and Informatica Power Center 8.x, 9.x
- Highly proficient in creating transformations, mappings, mapplets, tasks, sessions and workflows.
- High experience processing the large sets of unstructured and structured data using MapReduce, Hive, Pig and HDFS.
- Extensive experience with multiple MapReduce jobs using Java for data cleansing and processing.
- Proficient in RDBMS databases including Teradata, Oracle, MySQL, SQL Server and Hive. Experience in Oracle database development using SQL Tuning, PL/SQL and Stored Procedures.
- Expertise knowledge in Performance Tuning to resolve the bottleneck and optimize session performance with sound understanding of data warehousing concepts, dimensional modeling, Star and Snowflake schemas.
- Working experience in UNIX shell scripting to manage and schedule workflows.
- Experience with different types of SSH client like Putty and WinSCP for UNIX platforms.
- Working experience in Reporting tools such as Crystal Reports, Business Objects, Qlikview and Tableau.
- Working experience in UNIX environment, good in scripting languages like Shell script, Python and Perl.
- Working experience on JIRA to monitor, track and fix bugs and defects on the workflows.
CAREER OBJECTIVE:
I aspire for the role of Senior Developer in Data Warehousing and ETL Development. I am highly experienced in providing ETL solutions coupled with effective development strategies to meet the business needs.TECHNICAL SKILLS:
Category Software/Tool/Technology Proficiency:
ETL Tools: Informatica Power Center 8/9.1, IDQ, Power Exchange, Talend Proficient
Databases: Oracle 9i/10g/11g, SQL Server 2000/2005/2008 , Teradata Proficient
Big Data Ecosystems: Hadoop, MapReduce, HDFS, Hive, Pig, Oozie, Hue, Sqoop Proficient
Operating Systems/Servers: Windows, UNIX (Solaris) - WinSCP, Putty Proficient
Scripting: UNIX Shell Scripting, Perl Scripting, PL/SQL scripts Proficient
RDBMS: Load Tools SQL Developer, SQL * Plus (Oracle), Queryman, Teradata SQL Assistant Proficient
Scheduling Tools: Autosys, Tivoli Workflow Scheduler, Oozie Proficient
Reporting Tools: Tableau, Business Objects, Crystal Reports Proficient
Data Modeling Tools: TOAD 7.6/8/9.5/9.7 , SQL Developer Proficient
Office Applications: MS Office Suites including Excel, Word, Outlook Proficient
PROFESSIONAL PROJECTS HANDLED:
Confidential, San Jose, CA
Senior ETL Developer/BigData Consultant
Software / Tools / Technology Used: Informatica, Talend for Big Data, Hadoop, Hive, Pig, HDFS, MySQL, UNIX Shell Scripts
Responsibilities:
- Responsible for assessing the requirements and provide the estimate on the turnaround time to effectively deliver the expected output.
- Proficient in Informatica & Talend ETL tools to extract, transform and load data and generate reports in different formats.
- Analyzing source data for potential data quality issues and addressing these issues in ETL procedures.
- Used Informatica BigData components to transfer the file into HDFS and load the data into Hive.
- Responsible for coding and modifications to enforce business rules and to improve performance of the system.
- Implemented Talend ETL/BigData processes with Hadoop, MapReduce, Pig and Hive
- Developed Talend jobs to load the data into Hadoop and Hive and used most of the Talend BigData components like tHdfsInput, tHdfsOutput, tHiveLoad, tSqoopImport etc
- Used different components in Talend like tMap, tmysqlinput, tmysqloutput, tFiledelimited, tfileoutputdelimited, tmysqloutputbulkexec, tUnique, tLogcatcher, tFilelist, tAggregate, tSort, tFilterRow, tFTP and tJava components.
- Used Talend & Informatica BigData components to transfer the file into HDFS and load the data into Hive.
- Migrated Informatica jobs to Talend and developed complex jobs in Talend to pull the data from HDFS, load the data into hive, load data into stage and datamart.
- Extensively used MySQL, Hue, HiveQL, Pig and HDFS methodologies to process and load data.
- Extensively used Hadoop related Java components in Talend ETL tool to process the BigData.
- Using Talend ETL tool, Hive, MySQL and UNIX shell scripting have automated many projects which basically increased the performance of the jobs.
- Developed MapReduce programs to parse raw data by writing Hive queries.
- Used Oozie scheduler to automate data loading into Hadoop Distributed File Systems to pre-process data.
- Served as Senior Business Analyst to gather requirements by directly interacting with clients and submit the deliverables on time.
- Worked on JIRA to track and fix tickets related to improvements and bugs.
- Have good knowledge on Agile methodology.
Confidential, Silver Spring, MD
Senior ETL Developer
Software / Tools / Technology Used: Informatica Power Center 9.1, Oracle, Teradata, SQL Server 2005, PL/SQL, Business Objects, UNIX Shell Scripting, SQL*Loader, Business Objects XI and Crystal Reports.
Responsibilities:
- Responsible for requirement gathering and conversion of requirements into technical provisions
- Participated in the detailed requirement analysis and designed data marts and star schemas
- Worked intensively in various types of transformations like Source Qualifier, Expression, Lookup, Update Strategy, Stored Procedure, Joiner, Filter, Aggregation, Rank, Router, Sorter, Sequence Generator and Joiner etc.
- Extensively used Informatica, Teradata and Oracle to load real time data
- Implemented many mappings with slowly changing dimensions to insert and update data
- Responsible for incremental loading using Mapping Variables and Mapping Parameters
- Worked on Performance Tuning at target, source, mapping, session, and system levels
- Worked with connected and unconnected lookup transformation with different types of caches
- Worked with SQL Developer to access database and do the analysis on data validity
- Designing, Scheduling & Monitoring of workflows. Optimizing/Tuning mappings for better performance and maintenance
- Have implemented the business requirements and created ESC report using Crystal Reports functionalities of Grouping, Dynamic Parameters, Section Expert, Select Expert, Sort Expert, Group Expert, Formulas, Running Total, Summary, Cross-tabs etc.
- Supporting the BO reports built using BO XI reporting tool which total involves around 45 reports, these reports gets data from abinitio projects MASGL, MMGT and SAM.
Confidential
ETL Lead
Software / Tools / Technology Used: Informatica Power Center 8.6 /9.1, Oracle 10g, Teradata, PL/SQL, WinSCP, Putty, JIRA and UNIX shell scripting
Responsibilities:
- Worked with team of Developers to plan, design, and implement DWH for the end client
- Extracted historical data from different legacy systems to load data into target databases
- Designed various maps which has complex transformations to manipulate and load the data
- Extensive usage of Lookup Transformation to match data from multiple tables to retrieve data
- Contributed as Business Analyst to analyze, write and design requirements for implementation
- Contributed to help Quality Analysts understand the design and development of the ETL logic
- Worked with other ETL leads and contributed to conclude on the development of the project
- Designed complex mappings involving target load order and constraint based loading
- Created Tasks, Worklets and Workflows using Informatica Workflow Manager
- Produced Unit Test results to validate the mapping development and data consistency
- Extensive experience in creating database objects using Oracle, SQL Server and Flat files.
- Used WinSCP, Putty to schedule workflows though shell scripts and command line
- Worked on LLD and HLD to document the requirements, logic, analysis involved in the project
- Worked on JIRA to track and fix bugs in the code by analyzing the logs in Informatica.
Confidential
ETL LeadSoftware / Tools / Technology Used: Unix Shell Scripting, C, Oracle
Responsibilities:
- Handled Incident management effectively.
- Jobs have to be supported such that if there are issues, Incidents has to be resolved within the given SLA time frame.
- Tough timelines has to be maintained, since the project involves highly sensitive financial data.