Sr. Etl Lead/admin Resume
SUMMARY
- 12+ Years of IT experience designing, developing and implementing data warehousing solutions.
- Extensive experience in Analysis, Design, Development, Implementation, Testing and Support of Data Warehousing and Data Integration solutions using Teradata, HP Vertica, DB2 and Informatica Power Center
- Overall 6+ years of experience in Teradata for loading from various data sources and legacy systems into Teradata using BTEQ, FAST EXPORT, MULTI LOAD, FAST LOAD and TPT
- Technical expertise in performing ETL transformations using Informatica PowerCenter 9.x
- Creating Source to Target Mappings by working with various downstreams and business users
- Strong knowledge ofEntity - Relationship concept, Facts and dimensions tables, slowly changing dimensionsandDimensional Modeling (Star Schema and Snow Flake Schema).
- Well versed inOLTP Data Modeling, Data warehousing concepts.
- Expertise in database programming in writing of the SQL, Stored Procedures, Views in Teradata
- Worked on Teradata to Big data migration projects using HP-Vertica
- Proficient in performance analysis, monitoring and SQL, query tuning using EXPLAIN Plan, enforcing PI, PPI, SI, analyzing data skewness and Collect statistics in Teradata
- In-depth knowledge on HP Vertica, got direct training from HP Vertica team
- Have excellent knowledge of Software Development Life Cycle and Agile - SCRUM Methodologies and skilled in using version control systems.
- Can efficiently handle enterprise level projects of massive magnitude and pressure under critical deadlines.
- Highly inclined towards technology and updates with industry advancements. Enthusiastic in taking up challenging and technology oriented tasks with high scaling up capability.
- Experienced in LINUX/Unix work environment, shell scripting, file transfers, job scheduling and error handling, understanding of OS level kernel parameters
- Experienced in Job scheduling using Autosys and CA7.
- Experience working with moving data between Hadoop and Vertica and Kafka integration
- Experience in leading a team and providing technical leadership.
- Performing extensive Data Analysis and reporting data issues.
- Involved inUnit testing, System Integration Testing and User Acceptance Testing
- Performing Data analysis by writing extensive SQL to compare the final data over Vertica vs. Teradata.
- Very good debugging, analytical and logical skills.
- Excellent presentation, documentation and communication skills.
- Highly motivated and can work independently or part of a team
- Experience working in Onsite and Offshore co-ordination model
- Exposure to Apache Hadoop eco systems
- Exposure to IBM Cognos Report Studio, Tableau Visualization tool, Microsoft Power BI and Micro Strategy
TECHNICAL SKILLS
Core Technology: Teradata, HP- Vertica Appliance for Big Data, Big Data, SQL, Informatica 9.x
Scheduler: Autosys, CA-7 scheduler
Version Control: SVN, CHANGEMAN
Programming: Linux Shell Scripting, Mainframes- COBOL,JCL,PROC,PARM
Life Cycle Mgt: HP Quality Centre
Operating System: Linux, Windows 7, Windows XP, Windows 2000 / NT
Data Modeling Tool: Erwin
Reporting Tools: Tableau, Microsoft Power BI, Micro Strategy
Tools: Teradata SQL Assistant, Teradata Studio
PROFESSIONAL EXPERIENCE
Confidential
Sr. ETL Lead/Admin
Responsibilities:
- Responsible for migrating the data over from Legacy Teradata PRIDE 1.0 to PRIDE 2.0 running on Vertica using Teradata and Vertica Utilities
- Business Analysis and Requirement Collection.
- Preparing High Level and Low Level documents, and mapping specs
- Developing Linux/Unix Shell scripts for moving the data over from Teradata and to load to Vertica using Teradata Utilities such as MLOAD, FASTLOAD, FASTEXPORT.
- Used Teradata SQL Assistant and Teradata Studio for ad-hoc queries the performance of queries
- Working as Lead for PRIDE 2 Application managing cluster workloads and monitoring the performance
- Worked on defining and loading the data from Upstream
- Creating a BTEQ scripts to extract the data from Source and MLOAD to load high volumes of data from flat file to populated tables
- Optimizing Query Performance, Session Performance and Reliability, defining PPI and fine tuning the queries
- High interaction with Business Users, ETL/data base team to know their Business views in developing and modifying Universe accordingly. Established user groups, users and assigned privileges
- Worked on low latency query requirements and to fine tune the queries
- Setting up partitions, copy partitions, drop partitions, swap partitions, tuning for partition pruning
- Performance improvement on long running, high memory utilization jobs on a daily basis
- Working with Development team on resolving day-to-day issues, design suggestions and improvements, setting up Erwin Data Models for project requirement
- Working with Production Support team on Trouble shooting Production incidents or query performance issues
- Understanding the existing architecture, identifying problem areas and making improvements
- Writing extensive SQL queries on system tables
- Working with Vertica Support team to resolve any issues with into Nodes going down, refresh issues, cluster performance, running Scrutinize, debugging by looking into Vertica logs
- Rolling out pro-active performance improvements, running DBD and to consulting team on use of best practices
- Partition cleanup and enforcing retention on tables using drop partitions, move partitions.
- Generating ad-hoc reports as required
- Monitoring clusters using MC, Microsoft Power BI and MicroStartegy
Technologies / Environment: Teradata, HP-Vertica, Linux Shell ScriptingXIB and Athena
Confidential
Vertica Consultant
Responsibilities:
- Understanding the existing architecture and consulting performance improvement techniques
- Running SQL’s on system tables to identify system issues
- Performing Health checks
- Understanding the resource pools and suggesting improvements
- Walk thru with the development teams to make them understand how Vertica works and its architecture
- Working with Vertica Support team to resolve any issues with Nodes going down
- Consulting on Analyze Statistics run times and Tuple Mover tuning.
- Cluster expansion and rebooting
Technologies / Environment: HP-Vertica, Linux, Autosys
Confidential
Technology Lead/Sr. Teradata Developer
Responsibilities:
- Analyze and provide solution for integrating Customer Data Mart with Customer Relationships and Accounting system.
- Working with various Loading utilities of Teradata such as TPT, Fast Load, Multiload, T-Pump.
- Teradata performance tuning via Explain, PPI, Indices, collect statistics or rewriting of the code.
- Performing data quality and integrity end to end testing and reverse engineering and documented existing ETL program codes.
- Design solution to create interface to extract all necessary data across independent line of business, perform necessary transformations and load into Vertica.
- Development of solutions to migrate business scenarios implemented across multiple systems.
- Prepare ETL mapping documents to capture the source to target transformations
- Communication with multiple stakeholders involved in the migration to synchronize and analyze streamlined data movement
- Develop Join Indexes and Secondary indexes for better data access techniques.
- Work with multiple SORs (System of Records) across Hadoop, Mainframes, Linux, Windows and UNIX operating systems to receive the files using SFTP/NDM transmission mechanisms.
- Work with Data Modeler to build the data model and table structures
- Performing Unit, System Integration Testing and User Acceptance Testing by writing the test plan and test cases
- Use of Query Profiling techniques using Workload Analyzer and Query profiles
- Writing SQL queries and fine tuning them
- Validating data in Teradata vs. Vertica after the migration
- Proficient in writing Linux shell scripting and embed VSQL statements
- Status reporting on the progress and working with LOB to get the required approvals
- Communication with multiple upstream and downstream stakeholders, working with transmission protocols such as NDM/FTP/SFTP.
Technologies / Environment: HP-Vertica, Teradata, Linux, Autosys
Confidential
Technical Lead/Senior Teradata Developer
Responsibilities:
- Analyze and Design solutions for projects related to PCHH along with Domiciling (DOM) and IHH
- Preparing HLD and LLD documents as per the requirements
- Working on using Teradata Utilities such as BTEQ, FAST LOAD, FAST EXPORT, TPT, MLOAD to load the Teradata tables
- Developed highly efficient logic to retrieve data from multiple SORs to the warehouse and provide to downstream users.
- Performing Unit, System Integration Testing and User Acceptance Testing by writing the test plan and test cases
- Performance tuning for legacy applications to improve efficiency and provide faster query response times for Analytical data users.
- Impact Analysis for all legacy applications in case of any data model changes.
- Communication with multiple upstream and downstream stakeholders, working with transmission protocols such as NDM/FTP/SFTP.
- Weekly status updates to clients/higher management
Technologies / Environment: Teradata- BTEQ, FASTEXPORT, MULTI LOAD, FASTLOAD, TPT, Unix, Stored Procedure
Confidential
Technical Analyst/Teradata Developer
Responsibilities:
- Analyze and Design solutions for projects related to Mass Affluent such as Mass Affluent GL Segmentation, Mass Affluent Outbound
- Developed highly efficient logic to retrieve data from multiple SORS to the warehouse and provide to downstream users.
- Working on using Teradata Utilities such as BTEQ, FAST LOAD, FAST EXPORT, MLOAD to load the Teradata tables
- Developed highly efficient logic to retrieve data from multiple SORs to the warehouse and provide to downstream users.
- Performing Unit, System Integration Testing and User Acceptance Testing by writing the test plan and test cases
- Performance tuning for legacy applications to improve efficiency and provide faster query response times for Analytical data users.
- Impact Analysis for all legacy applications in case of any data model changes.
- Communication with multiple upstream and downstream stakeholders.
- Weekly status updates to clients/higher management
Technologies / Environment: Teradata- BTEQ, FASTEXPORT, MLOAD, FASTLOAD, Mainframe, Quality Center
