Data Engineer Resume
Sunnyvale, CA
SUMMARY:
- Developer with wide knowledge and proven experience in development and maintenance
- A self - starter and quick learner capable of working independently to achieve milestones and deadlines
- Around 7 years of IT experience with technical proficiency in the field of Data Warehousing and Business intelligence, with Business Requirements Analysis, Application Design, Development, Analysis and Support skills
- Experience in using Teradata (V2R5 thru 14.10) and Teradata ETL tools/utilities
- Experienced in Design and Setup of ETL solutions to meet Business needs using Informatica Power Centre platforms .
- With strong understanding of Teradata Architecture, RDBMS and Data Modeling concepts, a specialist in SQL tuning and Performance optimization techniques to achieve target SLAs in OLTP & OLAP environments
- Strong programming skills in Teradata SQL, PL SQL and hands on experience with SQL Server and DB2 databases
- Experience in automating batch jobs using tools like Tivoli Workload Scheduler (TWS/Conman) and Crontab
- Experience in creating reporting solutions using Teradata Views, Macros, Excel functions, Excel graphs and Pivot tables
- Created SQL scripts and written Procedures using PL/SQL to load data from flat files into new tables using both UTL and SQL Loader.
- Experience in scripting using Unix Shell (ksh/bash), PowerShell
- Ability to deliver quality improvements and streamlining processes in a cross functional environment
- Experienced in leading teams of various sizes with offshore and onsite combinations
- Results-oriented attitude and willingness to learn new technologies on own, if required
- Very good understanding of Reporting tools like Cognos, Business Objects, Micro strategy, Tableau.
- Involved in production support activities 24/7 during on call and resolved database issues.
- Strong problem solving, analytical, interpersonal skills, communication skills and have the ability to work both independently and as a team.
- Highly enthusiastic, self-motivated and rapidly assimilate new concepts and technologies.
TECHNICAL SKILLS:
Operating Systems: Windows 7/Vista/XP/NT/2000, HP-UNIX 10.2, IBM AIX 6.1,MAC OSX
Languages: TD SQL, PL SQL, C, C++, HTML
DBMS: Teradata 15.10/14.10 , 13/12/V2R5, Oracle 9i, PostgreSQL, MySQL, SQL server 2000, DB2
BI Tools: Tableau, Excel Pivots,Thoughtspot
SQL/ETL Tools: BTEQ, FastLoad, MultiLoad, FastExport, Tpump, Teradata SQL Assistant, SQL Developer, TOAD, Tivoli Workload Scheduler, Crontab, Conman, UC4 and GitHub.
ETL Platforms: Informatica 7.x, 8.x (Workflow Manager, workflow Monitor, Source Analyser, Mapping Designer, Mapplet Designer, Transformation developer)
Scripting: UNIX Scripting, PowerShell.
Others: Putty, WinSCP, Attachmate, MS Office Suite, MS Excel, Forklift
PROFESSIONAL EXPERIENCE:
Confidential, Sunnyvale CA
Data Engineer
Responsibilities:
- Loaded unstructured data into Hadoop File System (HDFS).
- Created ETL jobs to load JSON data and server data into HDFS and transported HDFS into the Teradata Data Warehouse.
- Created reports and dashboards using structured and unstructured data.
- Data Architecture and Data Modeling were enhanced extensively as required, involved in the analysis and design of the system.
- Actively managed the planning, organization of activities for JIRA tickets.
- Followed the scrum timelines and updated the version stories as needed.
- Organizing meeting with the SME’s of the dependent systems when changes are done for the existing system.
- Supporting global risk analytics (GRA) in their back-model creations and testing.
- Lead the project of craving out the Quantification process from saber/saber2 project and made it as a standalone application which got recognitions from high-level management.
- Experience with Cloud providers GCP(Google cloud Platform).
- Part of multiple teams to support the needs of the work stream as needed based on priorities.
- Worked on loading of data from several flat files sources using Teradata TPT, FastLoad and MultiLoad.
- Fine-tuned the existing mappings and achieved increased performance and reduced load times for faster user query performance.
Environment: Teradata 14, Hive, Hadoop Streaming, KAFKA Streaming, Talend, Oracle 11g, XML, Atomic (UC4) scheduler, Tableau, ThoughtSpot.
Confidential, San Francisco/ San Leandro, CA
Teradata ETL Developer/Analyst
Responsibilities:
- BBG is a Line of Business in Wholesale group Confidential Wells, main objective of this project is to prepare “Customer/ Entity” data for consumption by ECM (Enterprise content management).
- Analyze the systems; meet with end users and business units in order to define the requirements.
- Worked with business analysts and users to understand about Wholesale Business Banking group.
- Extensively worked with Teradata utilities like TPT, Fast Load, Fast Export and BTEQ to export and load data from and to different systems with source systems.
- Worked with Business analyst team and documented BRD, FSD, TDD, Mapping and Test cases which gained immediate signoff by client.
- Worked extensively on SQL, PowerShell scripting. Performed Data profiling for data quality purposes.
- Created and Automated the process using PowerShell to call and build TPT/BTEQ scripts, generate logs, upload data and logs to Share Point.
Environment: Teradata SQL assistant, Teradata Database, Flat Files, CSV files, MS Excel, Windows 8, SQL, Stored procedures, PowerShell Scripting, Scrum, Rally, SVN.
Confidential, San Jose, CA
Sr. Teradata ETL Developer
Responsibilities:
- Supporting the Teradata Enterprise Data and Analytics platform including mentoring, implementing best practice recommendations, and participating in the day-to-day activities
- Analyze complex SQL queries and create mapping sheets for Big Data migration project
- Teradata to Hadoop migration of a financial application using HIVE and SPARKSQL
- Usage of different Talend Hadoop Component like Hive, Spark.
- Load and transform data into HDFS from large set of structured data using Talend .
- Performance tuning of existing Teradata Applications (TFC and TLD)
- Developed Complex SQL queries and Inline views, global temporary tables for reporting purposes.
- Created SQL scripts and written Procedures using PL/SQL to load data from flat files into new tables.
- Developed strict SLA based regulatory reports using Teradata SQL and reported to end users by using ROOT
- Coordination with clients, Requirements Gathering and Impact Analysis.
- Enhancement of existing applications using Teradata SQL scripts.
- Job designing and scheduling batch jobs in UC4, and commit code using GitHub.
- Support and test new functionalities with Teradata 15.10 upgrade.
- Develop Informatica mappings for getting source data to target Teradata database.
- Scheduling Teradata scripts through internal frame and CRON for development purposes.
- Use DBC Tables for the performance measurement, Space calculations.
- Has Worked on Query grid tools for Improving the performance.
- Work on complex adhoc queries to support user help requests within a short time frame.
- Involved heavily in writing complex SQL queries based on the given requirements.
- Extensively worked with Teradata utilities like TPT, Fast Load, Fast Export and BTEQ to export and load data from and to different systems with source systems
- Used BTEQ and SQL Assistant (Query man) front-end tools to issue SQL commands matching the business requirements to Teradata RDBMS
Environment: Teradata 14, 15.10 (BTEQ, Fast Export, MLOAD, FLOAD, TPump), MySQL, TPT, PL/SQL HIVE, SPARK SQL, UNIX, UC4, Informatica, GitHub, Task Scheduler, Eclipse.
Confidential, Alpharetta, GA
Teradata ETL Developer
Responsibilities:
- Work with Business to gather requirements, translate them to technical requirements and involve in the design process of applications
- Review and optimize code developed by Teradata team and as well create FastExport, FastLoad, MultiLoad, BTEQ, TPT and UNIX scripts for loading/unloading files and database transfers on Teradata database taking advantage of the potency of Teradata technology
- Create Standard Operating Procedures for team members and approve/disapprove new application coding standards.
- Troubleshoot high severity issues within SLAs and take leadership in the war room process
- Build huge tables, views making sure there is appropriate indexing and Partitioning on tables with frequent inserts, deletes and updates to reduce the contention.
- Write code on peripheral databases like Oracle, SQL Server and DB2
- Create UNIX shell wrappers that call the BTEQs/PL SQL code and automate the batch jobs using Tivoli Workload Scheduler.
- Work with DBAs for transition from Development to Testing and Testing to Production.
- Created an archive process that archives the data files and FTP to the remote server
- Created a cleanup process for removing all the Intermediate temporary files that were used prior to the loading process.
- Worked on ETL Tool Informatica for massively parallel processing
- Created new tables and packages in the schema to support the web application.
- Used SQL Analytics functions for various reporting needs.
- Developed Complex SQL queries and Inline views, global temporary tables for reporting purposes.
- Used Cognos the reporting tool to extract corporate data, analyze it and assemble reports
- Created a shell script that checks the corruption of data file prior to the load
- Created unit test plans to unit test the code prior to the handover process to QA
- Involved in troubleshooting the production issues and providing production support
- Streamlined the Teradata scripts and Perl scripts migration process on the UNIX box
- Involved in analysis of end user requirements and business rules based on given documentation and working closely with tech leads and analysts in understanding the current system
- Collected statistics every week on the tables to improve performance
- Developed unit test plans and involved in system testing.
Environment: Teradata 13, (BTEQ, FastExport, MLOAD, FLOAD, TPump), TPT, Oracle 10g,9i, DB2,, 2003 MS Excel, Tivoli Workload Scheduler 8.3, Perl, Putty, IBM AIX 6.2, Erwin, Windows XP, Win 7.