We provide IT Staff Augmentation Services!

Teradata/bigdata Developer Resume

NJ

PROFESSIONAL SUMMARY:

  • Over 5+ Years of experience in IT, most recently focused on building ETL, data warehouse and business intelligence applications using Teradata and 6 months experience in Data lake (Hadoop)
  • Extensively worked with TERADATA utilities (BTEQ, Fast Load, MultiLoad, FastExport, TPT)
  • Excellent understanding of Hadoop Architecture and underlying frame work including storage management.
  • Hands of experience in GCP, Big Query, GCS bucket, cloud dataflow, GSUTIL, BQ command line utilities, Data Proc.
  • Well knowledge and experience in Cloudera ecosystem (HDFS, YARN, Hive, SQOOP, HBASE,) Data pipeline, data analysis and processing with hive SQL.
  • Involved in creating MIS documents to get create web roles, HDFS paths, Edge node folders and HIVE Databases for both onshore and offshore teams in DEV,TEST and PROD environments.
  • Involved in Importing and Exporting data between HDFS and Relational Database Management System using Sqoop.
  • Good experience in the implementation of Data Warehousing and Database business systems.
  • Hands on experience in extraction of data from various Heterogeneous sources (Relational database, Flat Files) to load into data warehouse/data mart targets.
  • Hands on experience in Performance Tuning of sources, targets, mappings, transformations, and sessions.
  • Proficient in performance analysis, monitoring and SQL query tuning using EXPLAIN PLAN, Collect Statistics, Hints and SQL Trace both in Teradata as well as Oracle.
  • Hands on experience in UNIX shell scripts to write wrapper scripts to call Mload scripts, Stored Procedures...etc.
  • Hands on experience in developing BTEQ, Fast Load, MLOAD scripts to extract data from multiple sources.
  • Hands on experience in scheduling tools Zena and Tivoli.
  • Experienced in Git, Jenkins and SVN tools.
  • Involved in review meetings with Project Managers, Developers and Business Associates for Project.
  • Experience on Shell scripting to automate various activities/Jobs.
  • Involved in preparing production Install packets which contains details about all objects that are getting deployed into production.
  • Have extensively worked in developing ETL program for supporting Data Extraction, transformations and loading using Informatica Power Center.

TECHNICAL SKILLS:

Operating Systems: Windows XP, 2000, UNIX/Linux

Teradata tools & Utilities query facilities: SQL Assistant, BTEQ

ETL Tools: Informatica

Scripting Languages: UNIX Shell, XML, HTML, JavaScript

Databases: Teradata16.20, Oracle (8i/9i/10g), MS Access DB,DB2

Load & Export: FastLoad, MultiLoad, FastExport, BTEQ, Oracle SQL Loader

Scheduler: Tivoli, Zena

Hadoop, Cloud: Sqoop, Hive, HBase, Hdfs,Map reduce, GCP,BigQuery

PROFESSIONAL WORK EXPERIENCE:

Confidential, NJ

Teradata/Bigdata Developer

Responsibilities:

  • Developed SQOOP script and SQOOP job to ingest RDBMS data to HDFS, HIVE .
  • Experienced moving HDFS files to Google DataProc, GCS bucket. And to Big query.
  • Loading RDBMS Data to BIGQUERY using Python, Gsutil And Shell Script.
  • Using rest API with Python to ingest Data from Google Cloud to BIGQUERY.
  • Using g - cloud function with Python to load Data in to Big query for on arrival csv files in GCS bucket.
  • Using YARN in CLOUDERA manager to monitor job processing.
  • Co-ordinated with the Business Analysts, Data Architects, DM’s and users to understand business rules.
  • Developed Technical design documents and getting approvals from business team.
  • Created MLOAD and Fastload scripts to load data related to Commission Junction orders into staging tables.
  • Created stored procedures to load staging data into corresponding target tables.
  • Involved in error handling, performance tuning of SQLs, testing of Stored Procedures.
  • Involved in creating Automated jobs to send sales and returned order reports to Business.
  • Involved in scheduling Teradata and UNIX objects to run the jobs (ECOM,POS) on daily/weekly basis depending on business requirement.

Environment: Teradata 16.20, Mload, FastLoad, Big query, Gcs Bucket, Cloud Shell, Gsutil, Dataproc,Cloud Sql, Python, Spark, Hive.

Confidential, Richardson, TX

Teradata/Hadoop Developer

Responsibilities:

  • Worked with DBA team to reduce the skew factor with proper primary index (PI)
  • Updated Teradata Development System utilizing Teradata Utilities like FastExport, Fastload and Restore in order to test and develop the applications.
  • User access management, space management, session management, Index management (Partition Primary Index, Join Index, Secondary Index, etc.).
  • Addressed system issues by notifying and directing application users for temporary workarounds (turning on diagnostics or instructing onsite Teradata personnel to modify dBs control flags).
  • Create and modify Indexes to improve performance of Batch and reporting applications.
  • Applying multi-value compression on all tables & Block level compression (BLC) on cold tables helped in saving a large amount of space.
  • Loaded data using FastLoad, BTEQ, Fast Export, MultiLoad and shell scripts.
  • Archived cold tables into data lake using sqoop automation scripts.
  • Identifying the scheduling dry run processes, analyzing the flow of data, gathering the associated tables & their usage reports, having a backup on data lake and decommission the processes and tables.
  • Removing the duplication of sources by identifying the related sources.
  • Involved in Importing and exporting data into HDFS and Hive using Sqoop.
  • Knowledge in performance troubleshooting and tuning Hadoop clusters.
  • Participated in development/implementation of ClouderaHadoop environment.
  • Implemented Partitioning, Dynamic Partitions and Buckets in HIVE for efficient data access.
  • Involved in working with various kinds of data sources such as Teradata. Successfully loaded files to HDFS from Teradata, and load loaded from HDFS to HIVE and exported to Teradata based on business requirement.
  • Experienced in managing and reviewing Hadoop log files.
  • Supported Map Reduce Programs those are running on the cluster. Involved in loading data from UNIX file system to HDFS.
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Involved in loading business data (gateway to Td and Teradata to Data lake) on by weekly basis.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Created an e-mail notification service upon completion of job for the particular team which requested for the data.

Environment: Teradata 16.20 and 14, HIVE, UNIX Shell Scripting, Zena, Mload, FastLoad.

Confidential, Union, NJ

Teradata ETL Developer

Responsibilities:

  • Understand the business rules and changes required per business requirements and working with Business Analyst.
  • Analyze the current data movement process and procedures.
  • Developed MLOAD scripts to load into staging data base.
  • Analyze and loading STORE CREDIT data into Data warehouse which is part of new initiation in the company.
  • Good analytical and logical programming skills with a good understanding at the conceptual level
  • Analyze and Fix Production bugs in POS surfaced during daily loads.
  • Involved in complete SDLC (System Development Life Cycle)
  • Created Teradata Stored Procs for Staging to Target Transformations
  • Analyzing and coding for new enhancements coming from BI team in POS
  • Involved in analyzing and loading near real time data that is coming from POS terminals from stores.
  • Used Teradata utilities fastload, multiload, tpump to load data
  • Involved in preparing validation scripts to validate data loaded in to Data warehouse.
  • Involved in masking the credit card numbers coming as part of point of sale transactions
  • Currently involved in Analyzing the effect on data warehouse due to change in terminal software at Point of Sale.

Environment: Teradata 14, DB2, UNIX Shell Scripting, Tivoli. Mload, FastLoad.

Confidential, Union, NJ

Teradata ETL Developer

Responsibilities:

  • Co-ordinated with the Business Analysts, Data Architects, DM’s and users to understand business rules.
  • Developed Technical design documents and getting approvals from business team.
  • Created MLOAD scripts to load data related to ECOM orders into staging tables
  • Created stored procedures to load staging data into corresponding target tables.
  • Involved in error handling, performance tuning of SQLs, testing of Stored Procedures.
  • Involved in scheduling Teradata and UNIX objects to run the jobs on daily/weekly basis depending on business requirement.
  • Prepared Validation scripts to Validate data in target tables with source data.
  • Prepared data cleansing scripts to clean source data as per business rules
  • Handling exception data into exception table for further analysis.
  • Prepared scripts to load look tables based on new data coming from source tables.
  • Responsible for migrations of the code from Development environment to QA and QA to
  • Production.
  • Provided production support on various issues during daily loads.
  • Involved in analyzing various production issues and necessary enhancements required.
  • Participated in knowledge transfer sessions to Production support team on business rules, Teradata objects and on scheduling jobs.
  • Involved in Defect Analysis and fixing of bugs raised by users UAT.

Environment: Teradata 14.0, SQL Assistant, Teradata BTEQ, Teradata Fast Load, Teradata Multi Load, Teradata Fast Export, TPT, Shell Scripts, Linux

Confidential

ETL Informatica developer

Responsibilities:

  • Developed ETL programs using Informatica to implement the business requirements.
  • Communicated with business customers to discuss the issues and requirements.
  • Created shell scripts to fine tune the ETL flow of the Informatica workflows.
  • Used Informatica file watch events to pole the FTP sites for the external mainframe files.
  • Production Support has been done to resolve the ongoing issues and troubleshoot the problems.
  • Performance tuning was done at the functional level and map level. Used relational SQL wherever possible to minimize the data transfer over the network.
  • Effectively used Informatica parameter files for defining mapping variables, workflow variables, FTP connections and relational connections.
  • Involved in enhancements and maintenance activities of the data warehouse including tuning, modifying of stored procedures for code enhancements.
  • Effectively worked in Informatica version-based environment and used deployment groups to migrate the objects.
  • Used debugger in identifying bugs in existing mappings by analyzing data flow, evaluating transformations.
  • Effectively worked on Onsite and Offshore work model.
  • Pre and post session assignment variables were used to pass the variable values from one session to other.
  • Designed workflows with many sessions with decision, assignment task, event wait, and event raise tasks, used informatica scheduler to schedule jobs.
  • Reviewed and analyzed functional requirements, mapping documents, problem solving and trouble shooting.
  • Created SSIS packages for File Transfer from one location to the other using FTP task
  • Performed unit testing at various levels of the ETL and actively involved in team code reviews.
  • Identified problems in existing production data and developed one-time scripts to correct them.
  • Fixed the invalid mappings and troubleshoot the technical problems of the database.

Environment: Informatica 8.6.1, SQL Server 2008 R2, HP-UX.

Hire Now