Senior Etl Data Engineer Resume
San Bruno, Ca
SUMMARY:
- Experienced, action oriented IT Professional with expertise in Data Warehouse Engineering/Analyst and Developing Applications that produce solutions rooted in business needs.
- Well honed analytical skills that consistently produce results to meet and exceed both internal and external client expectations.
- Well armed with extensive business savvy that translates to a high level of comfort working with any level of an organization.
- Over 15 years of experience in Information Technology. Extensive knowledge of systems analysis, numerous programming and scripting languages, multi dimensional relational databases, and operating systems
- Expertise in RDBMS covering data gathering and analysis, query optimization, data mapping, data validation, OLAP, ETL tools (IBM InfoSphere DataStage, IBM InfoSphere FastTrack, Teradata Utilities - TPT, FastLoad, MultiLoad, Beq, FastExport, Tpump, TPT) and SQL performance tuning
- Over 5 years using ETL integration tool (IBM Inforsphere DataStage V8.5) to load data into ODS or EDW
- Solid experience in integration of various data sources with Multiple Databases like Teradata, Netezza,Oracle 11g, MS SQL Server and COBOL/DB2/VSAM, JCL, PROCS, ENDEVOR package, CA7 scheduler tool, IBM Cognos, Tableau
- Work closely and effectively with all levels of management, end users and cross functional team to satisfy project/productivity requirements
- Write complex Teradata/Oracle/MS SQL Server queries for data analysis and cross data checking
- Created SSRS report writing in-line SQLs, Store Procedures or common table expressions (CTE)
- Analytical and persistent problem solver, focused on meeting customer’s requirements and expectations
- Experience in analyzing/transferring business specifications into functional/technical designs and developing, coding and testing SQL databases to manage information
- Quickly understands the business issues and data challenges of client's organization and industry
- Excellent interpersonal skills for credibility in working with senior executives, as well as managing client relationships and external partnerships
- Optimized existing code by identifying and rewriting inefficient subroutines or codes, reducing processing time
- Proven abilities in organizing management workflow and utilizing a proactive approach to problem solving
TECHNICAL SKILLS:
Databases: Relational databases (Teradata, MySql, Oracle, MS SQL Server, DB2)
Operating System: Unix, Linux, MVS Mainframe z/OS, Windows, AS400, OS/2
Languages: Teradata SQL, SQLs, Unix Korn Shellscript, Perl, Cobol, Easytrieve Plus, JCL, Spufi & File-Aid
ETL Tools: IBM Inforsphere DataStage V8.5, Teradata Database 14 (Utilities - FastLoad, MultiLoad, Bteq, Teradata Parallel Transporter, FastExport, SQL Assistant), Toad, T-SQL, Microsoft SQL Server Management Studio
Reporting Tools: IBM Cognos v8, v10.2, SSRS - SQL Server Reporting Services (2005, 2008), Tableau 9.2
PROFESSIONAL EXPERIENCE:
Senior ETL Data Engineer
Confidential, San Bruno, CA
Responsibilities:
- Interact with clients, SME's, Business Owners, Technology team to analyze existing Netezza processes and systems for data requirement gathering and to convert processes from Netezza to Teradata which required system/data analysis, data design, data validation, data matching, data profiling between current and new Teradata system, create system flow and technical documents to capture the new processes in Teradata, Hive and MySql
- Support/Maintain Customer Care Services data (Incidents, Contacts, Service Agents, Sales from various Call Centers and provide meaningful insights around customer satisfaction, agent performance and various key metrics (contact per order, sales returns, cancels and adjustments)
- Extract and merged Teradata, Hive data and populating target tables in MySql, Teradata databases
- Utilized Nexidia voice mining application to research issue areas then deep dive into calls to determine what issues/reasons from customer contact calls.
- Run Search/Discovery functions in Nexidia application to provide insights that might be the problems.
- Using Linux wrapper script to invoke Terdadata, MySql, Hive processes in the Linux integrated environment
- Utilize shellscript to automate batch processes and reformat data
- Extract/loading data from/to Hive and merge data in Teradata for data analysis
- Utilized Teradata Utilities (Bteq, FastExport, FastLoad) for ETL processes
- Created source to target column data mapping/transformation documents in Teradata from Netezza
- Conduct data mapping review sessions with the ETL development team on various subject areas
- Act as a subject matter expert in the Teradata Customer Care application and gather user requirements and translate business requirements into functional and technical specifications and provider data solutions
- Converted Netezza Sql which is in normalized form To Teradata Sql which is in third normal form, do analysis on data by writing sqls to validate between two systems to ensure data correctness/completeness and called out if there are any discrepancies and work with ETL development to fix the data if needed
- Read Teradata and MySql data to create reports/dashboards in Tableau in the Customer Care Data area.
- Converted Tableau reports to repoint source from Netezza to Teradata custom sqls and build/validated Tableau Worksheets/Dashboards
- Handle multiple requirements/projects to meet the aggressive timeline
- Build reconciliation processes to implement data quality check around completeness of data including certain data checks (data type, numbers, in/out row counts, aggregations)
- Tunes large and long running Teradata queries
Senior ETL/BI Developer
Confidential, Redwood Shores, CA
Responsibilities:
- Technical Team Lead for all aspects of the Data Warehouse Projects and/or application enhancement. responsible for the Design, Construction, Testing, and Implementation of projects using Software development life cycle
- Acted as a SME in the data warehouse team and interpret business requirements into various of documentations (interface functional, system flows, interface high level and level technical designs, source and target data mappings)
- Created detail/execute test scripts/plans and documented test results for completeness of data and pass the format and data quality check pattern.
- Manage Datastage Administration and setup users, securities, monitor the performance of the system
- Manage Cognos Administration, build complex Aggregate tables or write Reporting Views for Cognos Reporting and managed Cognos upgrade to v10.2
- Troubleshooting for data abnormities and run Sql and check impacted code for data discrepancies
- Responsible for the facilitation of project meetings between project and operational teams
- Used IBM WebSphere DataStage v8.5 as primary ETL tool to extract/transform/load data
- Publishing data in XML format using IBM WebSphere DataStage v8.5
- Subscribing data from XML to Oracle using IBM WebSphere DataStage v8.5
- Creating crontab process in Unix to kick off Datastage Sequencer Jobs and shell scripts for each process
- Using born shell scripting for formatting/cleansing data
- Writing Sql queries to analyze and identify data discrepancies between source and target system in Oracle 11g
- Implement code and utilize version control software Perforce
- Design, document and deploy systems pertaining to Enterprise Data Warehouse standards and best practices
- Manage Longview Forecasting Administration for FP&A/Finance team
- Provide 24x7 support on IBM Datastage, IBM Cognos, Longview loading processes and Jobs
Teradata ETL Engineer/Analyst
Confidential, Concord, CA
Responsibilities:
- Design, develop, implement and support various Banking data applications under Wholesale domain Enterprise Capital Management Group.
- Create low level technical designs based on high level/functional specifications
- Creating credit risk /Basel II related data by writing queries for business users to run the simulation credit risk model on various calculation logic (product mix, risk drivers and data discrepancies).
- Prepared shell script to execute ETL processes to populate Enterprise Data Warehouse by using Teradata utilities (Fastload, Fastexport, Multiload, and Bteq) on Unix & Linux box
- Writing complex SQL queries using Teradata SQL Assistant for answering adhoc questions from business user’s requests
- Performance Tuning and Trouble shooting and queries to improve data load times
Teradata ETL Consultant
Confidential, Seal Beach, CA
Responsibilities:
- Worked as a Integration Data ETL Lead in the COE group use DataStage (v8.5) ETL tool and Teradata Utilities FastLoad/Multiload/Bteq to convert/transform/load from Oracle into Teradata platform
- Created DataStage ETL coding/naming standard and designed DataStage Parallel jobs as is or forklifted them
- Created DataStage Sequencers, Audited Control tables to capture row counts and Unix Shell scripts for each process
- Writing complex Teradata/Oracle Sqls for data analysis and data validation checking when needed
- Reviewed Cognos Decision Stream ETL codes and prepared source to target data mapping document for DataStage development
- Tuned the DataStage jobs and queries to improve data load times when needed
- Participated, run, and validated the ETL interfaces in System testing, executed positive/negative/minus testing cases and documented the results
- Acted as a Teradata expert to provided support for any related Teradata Utilities/Sqls to Boeing Team members
Senior Data Analyst
Confidential - San Francisco, CA
Responsibilities:
- Work With Data Modeler for BI 2010 Project to understand the business, logical, physical models to create source to target data mappings
- Construct sqls for data analysis and crossing data checking
- Use IDQ in Informatica software for data analysis and data profiling, also use Oracle sqls to do data check, data validation, data format
- Conduct data mapping review sessions with the ETL development team to ensure smooth transaction to begin code designs
- Handle multiple requirements/projects to meet the aggressive timeline
DataStage ETL Developer
Confidential, Sunnyvale, CA
Responsibilities:
- Designed Datastage v8.1 jobs using Filter stage, Sort stage, Change Capture stage, Aggregator stage, Surrogate Key Generator stage, Transformer stage, Lookup stage, Modify stage, Merge stage and etc …
- Worked with Deliverable Manager and Business Owner to understand business process and user requirement to prepare the functional specifications and then curve them into Technical specifications
- Act as a Technical Team Lead for Partner Analytics project for nightly call to offshore team members to review deliverables, project statues and provided technical guidance when needed
- Analyzing both source and target systems to identify dataflow and improving the existing process and creation of new process as and when required
- Writing Oracle sql queries to analyze and identify data discrepancies/analysis between source and target system
- Creating sequencers and Unix shell scripts for each process
- Modified Datastage jobs to fix Production data issue or missing data in the NetApp EDWNI Product Dimensions, Organization Party, Partner Analytics subject areas.
- Worked for improve the performance for Datastage jobs by tuning them, changing the job designs, changing the SQL queries
- Worked and extracted data from various database sources such as Oracle 10g/9i/8i/ and Flat files
- Prepared Unit Test plan, cases, scripts documents and captured the results
- Supporting QA testing team to run and validate the ETL interfaces
- Prepared Implementation Plan and reviewed them with the Platform team and BI Operations team
- Knowledge of the Quality Management, Change Management, Release Management, Testing Processes
Senior Data Analyst
Confidential, San Jose, CA
Responsibilities:
- Interact with clients, SME's, Business Owners and Sr. Process Analysts, Technology team to analyze existing business processes and systems for data requirement gathering
- Generate reports (Customer data, Credit data) in csv files, delimiter flat files formats using in combinations of Teradata Utilities (FastLoad, MultiLoad, Bteq, FastExport), Teradata SQL, Oracle SQL called by Shell script in Unix/Linux environment
- Creating a Mapping, Session, Workflow using Informatica Powercenter 8.6 for loading to UK credit Staging table
- Work with Marketing email team to design campaign specifications, convert requirements into source/target rules using plain English language, prepare for building by writing Teradata queries and execution of campaign, run final campaign using IBM Enterprise Marketing Management Tool Unica general outbound email file and troubleshooting any issues with corresponding parties (internal or external)
- Handle multiple requirements/projects to meet the aggressive timeline
- Gathered high level business requirements related to Product Credit in PayPal services from Business Owners and Technology Managers to understand business needs and prepared High level Business Requirements
- Providing ongoing tracking and reporting on marketplace developments to support Marketing team
- Participated in Business and Technical Review sessions to gather Detailed Business Requirements as well as System & Process requirements
- Actively participated in Project Review sessions with Business Owners and Technology Team
- Developed data driven tests to easily maintain the scripts and to test with various data including positive and negative test data and ensure the test pass the data quality checks routine (data type, date format and counts)
- Role involved requirements gathering & analysis, data analysis, data design, data validation, development, and quality analysis
- Worked with PM's and SME's to gather requirements and prepare system requirements, functional specs, and detail design documentation
