PROFESSIONAL SUMMARY:

• Over 6.5 years of Total IT professional experience in Big Data and Data warehousing (ETL/ELT) technologies includes requirements gathering, data analysis, design, development, system integration testing, deployments and documentation.
• Hands on experience in solutions for Big data using Hadoop, HDFS, Map Reduce, Spark, PIG, Hive, Kafka, Sqoop, Zoo keeper, Flume, Oozie.
• Excellent knowledge and hands on experience of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm and monitoring systems.
• Hands on experience in installing, configuring, and using Hadoop ecosystem components and management.
• Experience in importing and exporting data using Sqoop from HDFS/Hive to Relational Database Systems and vice - versa.
• Experienced and well versed in writing and using UDFs in both Hive and PIG using Java.
• Excellent understanding with different storage concepts like block storage, object storage, column storage, compression storage.
• Extensive experience in Extraction, Transformation <<>> Loading (ETL and ELT) data from various sources into Data Warehouses and Data marts with industry best practices.
• Experience with Informatica ETL for data movement, applying data transformations and data loads.
• Good working experience with different Relational DB systems.
• Very good understanding with implementations in building data warehousing and data marts with OLTP vs OLAP, star vs snow flake schema, normalization vs de-normalization methods.
• Hands on experience in building wrapper shell scripts and analysis shell commands in practice.
• Supported various reporting teams and experience with data visualization tool Tableau.
• Very good at SQL, data analysis, unit testing, debugging data quality issues.
• Excellent communication, creative, technically competent, problem solving and leadership skills.
• Focus on customer satisfaction and drive results by being team player and individual contributor with good collaboration skills as well.

SKILL:

Languages: Hive, Sqoop, SQL, PL/SQL, UNIX shell scripting.

DB Utilities: BTEQ, Viewpoint, Fast Load, Multi Load, Fast Export, T Pump, SQL*Loader, Exp/Imp, TD Administrator, TD Manager, TSET, SQL Assistant, Visual Explain, TASM.

Scheduling Tool: Multiload, AutoSys, and Version Control.

Tools: CSV, VSS, Arcmain, Teradata Administrator, Visual Explain, SQL Assistant, Toad, Putty, WINSCP, CYGWIN, Oracle Developer 2000, SQL*Plus.

PROFESSIONAL EXPERIENCE:

Sr. Teradata Developer and Hadoop Developer

Confidential, Sanjose, CA

Responsibilities:

• Evaluated business requirements and prepared detailed specifications that follow project guidelines required to develop written programs.
• Responsible for building scalable distributed data solutions using Hadoop.
• Importing the data from the SQLServer to HIVE and HDFS using SQOOP for One time and daily solution.
• Worked on Big Data Hadoop environment on multiple Nodes
• Implemented Hive tables and HQL Queries for the reports.
• Experience in performing data validation using HIVE dynamic partitioning and bucketing.
• Involved in Extracting, loading Data from Hive to Load an RDBMS using Sqoop.
• Extensive data validation using HIVE and also written Hive UDFs
• Involved in creating Hive tables loading with data and writing hive queries which will run internally in map reduce way.
• Worked on exporting the same to SQLServer, which further will be used for generating business reports.
• Worked in tuning HiveQL to improve performance
• Good experience in troubleshooting performance issues and tuning Hadoop cluster.
• Load flat files from Hdfs file path to local informatica ( ETL ) file system directories. From here loaded as Source file into Informatica Power Center for transformations and loading processed data into final destination database for further decision making process.
• Used Teadata utilities: FastLoad, Multiload, Tpump, Fast EXPORT, BTEQ,TPT
• Used EXPLAIN, COLLECT STATISTICS for TERADATA performance tuning.
• Used Informatica PDO( push down optimization) and SESSION PARTITION for better performance .
• Created procedures, macros in Teradata
• Used Bteq for sql scripts and batch scripts and created batch programs using Shell scripts.
• Developed the sqoop scripts in order to make the interaction between Pig and MYSQL Database.
• Developed the script files for processing data and loading to HDFS. Written CLI commands using HDFS. Developed the UNIX shell scripts for creating the reports from Hive data.
• Ran cron jobs to delete hadoop logs/local old job files/cluster temp files, Setup Hive with MySql as a Remote Metastore.
• Moved all log/text files generated by various products into HDFS location. Created External Hive Table on top of parsed data.
• Worked on different phases of Data Warehouse development lifecycle from Mappings to extracting data from various sources to tables and flat files. Created Re-Usable objects like Maplets & Re-usable transformations for business logic.
• Worked on Transformations, such as Rank transformations, Expressions, Aggregator and Sequence
• Experienced in working with complex mappings using expressions, routers, lookups, aggregators, filters. Worked on updates and joiners in Informatica, session partition, cache memory, connected lookups and unconnected lookups.
• Used TERADATA 13/Oracle databases for informatica DW tool to load source data.
• Created Teradata schemas with constraints, Created Macros in Teradata. Loaded the data using Fast load utility. Created functions and procedures in Teradata.
• Experienced in writing SQL queries, PL/SQL programming and Query Level Performance tuning.
• Developed and Tested database sub-programs (Packages, Stored Procedures, Functions) according to business and technical requirements

Sr. Teradata Developer

Confidential, Sanjose, CA

Responsibilities:

• Data Analysis and issue identification
• Propose Architectural design changes to improve data warehouse performance
• Visualize a data architecture design from high level to low level, and design performance objects for each level
• Troubleshooting database issues related to performance, queries, stored procedure
• Create ER diagram and conceptual, logical, physical data model
• Fine-tune the existing scripts and process to achieve increased performance and reduced load times for faster user query performance
• Accountable for Architect related deliverables to ensure all project goals are met within the project time lines
• Performs mapping between source and target data, as well as performing logical model to physical model mapping and mapping from third normal form to dimensional (presentation layer).
• Creates, validates and updates the data dictionary and analysing documentation to make sure that the information captured is correct
• Design logical and physical data model using Erwin data modelling tool and visio
• Architecture and design support to provide solution for business initiated requests/ projects
• Writing teradata sql queries to join or any modifications in the table
• Creation of customized Mload scripts on UNIX platform for Teradata loads
• Provide design for CDC implementation for real time data solutions
• Interact with business to collect critical business metrics and provide solution to certify data for business use
• Analyse and recommend solutions for data issues
• Writing Teradata BTEQ scripts to implement the business logic.

Teradata Developer

Confidential

Responsibilities:

• Provided data warehousing architecture by writing SQL query code based on detail requirements provided by marketing group.
• Proper analysis of data included identification and enforcement of standards to ensure consistent and maintainable queries and reporting SQL data through Excel.
• Performed all work within the Teradata RDBMS environment using Queryman 6.0.
• Used ERWIN for metadata and table associations and ascential data stage.
• Provided proper documentation using technical writing tools.
• Followed department SDLC methodology and development procedures.
• Developed modules to extract, process & transfer the customer data using Teradata utilities.
• Created Fast Export scripts for extracting and formatting customer data from data warehouse in a mainframe file.