Data Engineer Resume
Charlotte, NC
PROFESSIONAL SUMMARY:
- 7 Years of total IT experience and Technical proficiency in the Data Warehousing involving Business Requirements Analysis, Application Design, Data Modelling, Development, testing and documentation in in Teradata (MPP), Informatica Power center and Hadoop Big data Environments.
- Teradata 12 Certified Professional, Teradata 12 SQL Certified.
- Certified Scrum Master (CSM).
- Strong understanding of the principles of Data warehousing, slowly changing Dimension techniques, Fact Tables, Dimension Tables, snowflake schema, star schema, Conceptual, Physical and Logical Data modelling.
- Involved in full lifecycle of various projects, including requirement gathering, system designing, application development, enhancement, deployment, maintenance and support.
- Extensive experience in the analysis, design, development, implementation and testing of Data warehouse applications in Banking, Retail and E - Commerce Domains.
- Extensive Experience in Teradata Database design, implementation and maintenance mainly in large scale Data Warehouse environments, experience in Teradata RDBMS using FastLoad, MultiLoad, TPump, FastExport, Teradata SQL Assistance, Teradata Parallel Transporter and BTEQ Teradata utilities.
- Certified Teradata consultant with experience in Teradata Physical implementation and Database Tuning, technical and functional applications of RDBMS, Data Mapping, Data management, Data transportation and Data Staging.
- Analyzed, Designed and documented requirements for data migration projects between numerous source Legacy/Oracle application feeds and the new Teradata platform.
- Expert in Coding Teradata SQL, Teradata Stored Procedures, Views, Macros and Triggers.
- Expertise in Complex Query Analyzing, performance tuning and testing.
- Extensive database experience and highly skilled in SQL in Oracle, MS SQL Server, Teradata, Mainframe Files, Flat Files, MS Access.
- Expertise in OLTP/OLAP System Study, Analysis and E-R modeling, developing Database Schemas like Star schema and Snowflake schema (Fact Tables, Dimension Tables) used in relational, dimensional and multidimensional modeling.
- Technical expertise in ETL methodologies, Informatica 9.1 - Power Center, Client tools - Mapping Designer, Workflow Manager/Monitor and Server tools - Repository Server Manager.
- Strong Experience in Performance optimization of Teradata Queries.
- Experience in Developing Extraction jobs to Import data into Hadoop file system from Mysql traditional systems using Sqoop Import tasks.
- Good working Experience on Hive scripts and HDFS commands.
- Implemented real time data streaming using Flume for online banking and mobile banking.
- Have worked semi-structured data (XML and JSON) parsing using hive.
- Performance optimization of XML parsing using JAX-T parser.
- Working experience in UNIX environment and Preparation wrapper scripts to call ETL jobs and setting up alerts about the key load tables and space/memory issues in Systems.
- Good at Preparation of Technical responses, High-level, Low level design, Unit Test Case and Release documentation.
- Good Working Knowledge on OBIEE, Cognos BI tools.
- Extensive experience in Onsite-Offshore model.
- ETL Team leading experience to distribute/gather tasks and daily status update to client on weekly/daily scrum calls.
- Excellent Client handling ability with good communication and Presentation skills.
- Experienced working in versatile software development methodologies including Agile/Scrum, Iterative and Waterfall SDLC methodologies.
TECHNICAL SKILLS:
Databases: Teradata TD12,TD13,TD14,TD 15.10 Oracle 11g, SQL Server,
ETL: Informatica 8.6,9.1
Bigdata: HDFS, Hive and Sqoop
BI Tools: Cognos, OBIEE
Programming: Unix Shell scripting
OS: Windows, UNIX,MAC
Office Tools: MS Office, MS Access, MS Visio and MS Project
PROFESSIONAL EXPERIENCE:
Confidential, Charlotte, NC
Data Engineer
Environment: Teradata TD 15, UNIX, Hadoop, MySQL, Flume
Responsibilities:
- Designing and developing new ETL Jobs/ Workflows as per the Business requests.
- Creating ETL mappings using Sqoop/Hive/Teradata Loaders queries and SQL scripts in UNIX environment through scheduler.
- Extraction of Data from MySQL server to Hadoop platform using sqoop Import tasks.
- Transforming data using Hive Queries and loading into Temporary hive tables.
- Daily monitoring the jobs in Opswise monitoring tool.
- Doing hotfixes for the failed jobs and complete the data loading with SLA time.
- Creating Perm fix for Code issues and move them into production as pat of weekly Releases.
- Data analysis and clarifying the audit reports misleads to the users.
- Handling Adhoc requests from the BI team to reload or performance related issues.
- Performance Optimizations of the Teradata/Hive Queries, which are impacting the system.
- Preparing and maintaining operational excellence document to present every week meeting with BI director.
- Scheduling changes of Jobs to reduce the heavy load the system based on the criticality.
Confidential
ETL Lead Engineer
Environment: Teradata TD 14, UNIX, Hadoop, MySQL, Flume
Responsibilities:
- Designing and developing new ETL Jobs/ Workflows as per the Business requests.
- Creating ETL mappings using Sqoop/Hive/Teradata Loaders queries and SQL scripts in UNIX environment through scheduler.
- Extraction of Data from MySQL server to Hadoop platform using sqoop Import tasks.
- Transforming data using Hive Queries and loading into Temporary hive tables.
- Daily monitoring the jobs in Opswise monitoring tool.
- Doing hotfixes for the failed jobs and complete the data loading with SLA time.
- Creating Perm fix for Code issues and move them into production as pat of weekly Releases.
- Data analysis and clarifying the audit reports misleads to the users.
- Handling Adhoc requests from the BI team to reload or performance related issues.
- Performance Optimizations of the Teradata/Hive Queries, which are impacting the system.
- Preparing and maintaining operational excellence document to present every week meeting with BI director.
- Scheduling changes of Jobs to reduce the heavy load the system based on the criticality.
Confidential
Analyst 1 Apps Prog
Environment: Teradata TD12, TD 13, Oracle 11g, UNIX, Informatica 9.1
Responsibilities:
- Analyzing the business requirements specified by the client.
- Preparation of Technical requirement, High level and Low level design documents based on the business process document.
- Involved in loading data from flat files to ware house landing zone tables using Informatica loaders.
- Writing BTEQ Scripts to load data from landing zone to Work tables in staging area based on the ETL mapping documentation.
- Writing BTEQ Scripts for all the Static and Dynamic data loading using BTEQ.
- Unit testing in UNIX environment and performance turning based on explain plain.
- Involved while Creation of Informatica mappings based on the ETL mapping document and including the BTEQ scripts in the workflows.
- Creation of views in Target database to calculate measures from different schemas.
- Performance tuning of BTEQ scripts and Informatica mappings.
Confidential
Associate
Environment: Teradata TD 14, Informatica, UNIX
Responsibilities:
- Analysing the business requirements specified by the Business users.
- Preparation of mapping document and specifying the data lineage specifications to the developers.
- Loading data from Different source system like Oracle into Stage and JRNL areas.
- Populating data into Sematic layers after applying multiple business transformations.
- Writing BTEQ Scripts to load data from landing zone to Work tables in staging area based on the ETL mapping documentation.
- TPT scripts to load data from various source systems.
- Writing BTEQ Scripts to load Initial and Incremental for all the Static and Dynamic data loading using BTEQ.
- Unit testing in UNIX environment and performance turning based on explain plain.
- Creation of Informatica mappings based on the ETL mapping document and including the BTEQ scripts in the workflows.
- Creation of views in Semantic layers.
- Performance tuning of BTEQ scripts and Informatica mappings.
- Files upload into SFTP servers to third party vendors with Encryption and decryption technique.
- Performance optimization of costly SQL.
- Team leading and assigning/gathering task status and update to the client manager in daily scrum calls.
- Co-ordinating with offshore team on daily team meetings.
Confidential
Application Engineer
Environment: Teradata, BTeq, Fastload, Multiload and Fexp utilities, UNIX
Responsibilities:
- Design, develop and test process for validation and conditioning data prior to loading into the DW.
- Preparation of loading scripts using Teradata BTEQ, Fastload, Multiload utilities.
- Debugging jobs using Queryman to meet the business requirements.
- EXPORT/IMPORT to extract/load data from/into tables in Teradata data warehousing applications.
- Prepare the ETL Transformation Specifications for business logics. Analyse the existing mappings and prepare Business Documents for existing mappings.
- Involved in Database migration work and assisted in resolving performance issues and also migrated Different type of tables from Source (OLTP) to Teradata.
Confidential
Systems Engineer
Environment: Teradata TD14, Mainframe - JCL, Changeman, CA7
Responsibilities:
- Monitoring the jobs in WLM tool.
- Pre run checklist preparation prior to every month end loads and communicating with WLM operations team.
- Worked on the issues with Teradata BTEQ, Fastexport, Multiload utilities, and batch script.
- Debugging SQLs using Queryman to resolve the loading issues and run the jobs from test environment.
- Identifying the COBOL related issues using Netexpress debugging tool.
- Worked on the User Queries on the Reports.
- Bug fixing and code promotion to the production sever.
- Extensive Experience in HP Service centre usage for Change ticket and Incident management.
- Preparations of Post month end documents like Run status report, Solution documents.
