We provide IT Staff Augmentation Services!

Sr. Technical Consultant Resume

Reno, NV

SUMMARY:

  • Around 10 Years of experience in IT, this includes Implementation of Data Warehousing projects like Teradata/Netezza/Redshift Database design, migration, implementation/maintenance mainly in Data Warehouse/Datalake environments.
  • SNOWPRO Certified and experienced in implementing data migration projects from on - prem legacy systems to Cloud Datalake. Design & built ELT (for bulk & CDC loads) & ETL data pipelines in Snowflake environment (snowsql, snowpipe, streams, tasks etc.).
  • Experience in Hadoop based environments to implement data movement, data cleansing, data integrations using HDFS, Sqoop (for import/export), Hive (analytical purposes), APACHE SPARK (data processing), Airflow (scheduling & monitoring). Worked on Message queuing tools like Apache Kafka, cloud pubsub to process data streams using spark streaming module.
  • Trained and well versed in Data Preprocessing, exploration analysis &Visualization Techniques using Data Science pipelines (handled Supervised, Unsupervised & Time series Algorithms) using python libraries (scikit, sklearn, pandas, seaborn, matplotlib etc.)
  • Proficiency in programming languages like SQL, Python, Scala.
  • Expertise in handling services on AWS cloud like S3 (storage management), creating/configuring/integrating IAM roles, EC2, Cloud Formation etc. knowledge on handling streaming data using AWS kinesis.
  • Experience in handling services on Google Cloud Platform like GCS, Transfer services (for data movement), Big Query (for storage & processing engine), Cloud composer (for scheduling workflows), Cloud dataflow & Apache Beam (to build pipelines for Transformations).
  • Around 3 + years of experience in working on Data integration & Business intelligence tools (Tableau, Business objects) and ETL tool as Informatica.
  • Experience on working with OLTP systems such as MS SQLServer & MySQL relational databases and well versed in implementing WEB API’s thru HTTP POST method to post files for data processing.
  • Strong Data Modeling experience in ODS, Dimensional Data Modeling Methodologies likes Star Schema, Snowflake Schema. Design and development of OLAP models consisting of multi-dimensional cubes and drill through functionalities for data analysis.
  • Well versed in writing UNIX shell scripting.

TECHNICAL SKILLS:

Languages: SQL, PL/SQL, SCALA, Python, COBOL, JCL.

Operating Systems: Windows, Unix/Linux.

Database/DWH/Datalake: Snowflake (SnowSQL & snowpipe), MySQL Database, DB 2 7.0/8.0/9.0 , Oracle 11g/10g/9i/8i/8, Teradata 13.10/14, SQL Server 2008, Netezza & RedShift.

File System: HDFS, JSON, XML, CSV, Flat Files.

ETL/BI Tools: SPARK, Informatica PowerCenter 9.X, IDQ 10.1 Analyst & Developer, Business Objects, Tableau.

Cloud Services/Other: AWS S3, AWS EC2, SNS, IAM, GCS (GSUTIL & BQ command Utility), DataBricks, ERWIN.

PROFESSIONAL EXPERIENCE:

Confidential, Reno, NV

Sr. Technical Consultant

Responsibilities:

  • Work across multiple functional projects to understand data usage and implications for data migration.
  • Assist in designing, planning and managing the data migration process.
  • Work with subject matter experts and project team to identify, define, collate, document and communicate the data migration requirements.
  • Prepare data migration plans including migration risk, milestones, quality and business sign-off details.
  • Written dataflow jobs using compute fit (python function) to push records to BigQuery, scheduled federated queries, copy/move data using bq command utility. using GSUTIL command utility, copied data from AWS to GCP (GCS buckets).
  • Created Snowflake roles (RBAC), databases, warehouse & schemas and grants permissions to the corresponding objects. Implemented storage integration (external stages) to access S3 files and load data into S3 using COPY command & relevant options as per file formats.
  • Implemented CDC logic using snowpipe, stream and task and well versed in creating external tables & materialized views to improve the performance. Created snowflake external table which hold entire data including history data, also created a view on top of external table. This view displays only current state of data by eliminating history data.
  • Worked on importing and exporting data using Sqoop and PySpark from Relational Database system to HDFS and vice-versa. Developed data ingestion script to ingest data from SQL & Oracle to HDFS/HIVE using PySpark.
  • Involved in handling streaming Data from Apache Kafka and consuming data thru Spark jobs. Responsible for Optimizing Spark SQL queries that helped in saving Cost to the project.
  • Well versed in implementing spark code using Python/SCALA (like using scala collections, functions (higher order functions, block expressions, relevant classes & packages etc.)
  • Handled streaming data using data bricks platform (troubleshooting & debugging the spark jobs to verify STDOUT & STDERR logs to detect & diagnose data patterns.
  • Handling partitioning and spark optimizing techniques like caching, compression and avoiding to declare functions in advance.
  • Developed ELT data pipelines using combination of Python and Snowflake.
  • Utilized Python Libraries like BOTO3, PANDAS, NUMPY (arthematic operations) for AWS.
  • Created AWS DMS Replication Instances to pick-up data from source endpoint and place it in target endpoint.
  • Created Snowflake roles, databases, warehouse & schemas and grants permission for the same.
  • Implemented storage integration to access S3 files and load data into S3.
  • Implemented CDC logic using snowpipe, stream and task.
  • Created snowflake external table which hold entire data including history data, also created a view on top of external table. This view displays only current state of data by eliminating history data.
  • Scheduled the jobs using AWS CloudWatch service to monitor the hourly, daily & weekly frequencies .
  • Created AWS SNS service for reporting error messages and error handling.
  • Loading the data into the warehouse from different flat files.
  • This Project deals with Migration of on-premises oracle data to Cloud Datalake (i.e. AWS S3). Also, it deals with implementation of ELT pipeline using snowflake and python.

Technologies Used: SQL Server, Oracle, Snowflake environment, GCP, AWS, Apache Spark, DataBricks, Apache Kafka, Python scripting, SnowSQL, Flat files, SQL, AWS, Erwin, JIRA.

Confidential, Miami, FL

Sr. Technical Consultant

Responsibilities:

  • Analyzing the Business requirements and System specifications to understand the Application.
  • Converted Hive/SQL queries into RDD transformations using PySpark. Used Hive Sql, PySpark, Sparksql scripts to handle the different use cases as part of Extract, Transform, Load phases.
  • Built real time steaming applications using Spark Streaming API, Kafka, Zoo Keeper and Scala.
  • Consumed JSON messages from Kafka in batch intervals and process using Spark data frames.
  • Responsible for Tuning Report Queries and ADHOC Queries.
  • Extracting files from Teradata tables and placed in Hadoop distributed file in system by java using Sqoop.
  • Experienced in Tuning SQL Statements and Procedures for enhancing the load performance in various schemas across databases. Tuning the queries to improve the performance of the report refresh time.
  • Worked on different file formats like AVRO, PARQUET, ORC, CSV, JSON, XML and different Compression Codecs (GZIP, BZIP2, SNAPPY, LZO). Experienced in writing Spark Applications in Scala and Python (Pyspark).
  • Experience with AWS installing Cloudera on EC2 instances, configured SNS on few buckets in S3. Hands on experience working on Data loads to S3 using Spark.
  • Created Snowflake roles (RBAC), databases, warehouse & schemas and grants permissions to the corresponding objects. Implemented storage integration (external stages) to access S3 files and load data into S3 using COPY command & relevant options as per file formats.
  • Implemented CDC logic using snowpipe, stream and task and well versed in creating external tables & materialized views to improve the performance. Created snowflake external table which hold entire data including history data, also created a view on top of external table. This view displays only current state of data by eliminating history data.
  • Developed SQL scripts and shell scripts to move data from source systems to staging and from staging to Data warehouse in batch processing mode and understanding of Google cloud platforms by AWS to retrieve the data.
  • Involved in building and deploying cloud-based data pipelines and BI applications using AWS and GCP services.
  • Build Data Pipelines to ingest/store/analyze & visualize the structured data.
  • Exploring and performing POC on Google Cloud Platform (including Cloud Machine Learning, Cloud Data Store, Bigtable, Big Query, DataLab, and DataStudio).
  • Used UNIX scripts to run Teradata DDL in BTEQ and write to a log table.
  • Creating, loading and materializing views to extend the usability of data.
  • Loading the data into the warehouse from different flat files.

Technologies Used: DWH pipelines, Hadoop, Apache Spark, Python, Oracle 10G, MySQL, SQL Server, SQL, AWS, Erwin, Linux Shell Scripting.

Confidential, Las Vegas, NV

Sr. Data Engineer/Analyst

Responsibilities:

  • Involved in designing the tableau ETL Servers process to Extract transform and load data from OLAP to Teradata data warehouse.
  • Installed and configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Experienced in defining job flows and importing & exporting data into HDFS and Hive using Sqoop.
  • Experienced in managing and reviewing Hadoop log files and running Hadoop streaming jobs to process terabytes of xml format data.
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Was responsible to manage data coming from different sources and supported MapReduce Programs those are running on the cluster. Involved in loading data from UNIX file system to HDFS.
  • Installed and configured Hive and written Hive UDFs.
  • Generated different Space Reports in Teradata Manager to analyze different kind of issues
  • Provided ongoing support by developing processes and executing object migrations, security and access privilege setup and active performance monitoring.
  • Involved in working with SSA requestor responsibilities which will be assigned for both project and support requests.
  • Worked on different data stores and file formats in web services.
  • Used Fast Export utility to extract large volumes of data at high speed from Teradata warehouse.
  • Performance tuning for Teradata SQL statements using huge volume of data.
  • Created Fast Load, Fast Export, Multi Load, TPump, and BTEQ to load data from Oracle database and Flat files to primary data warehouse.
  • Responsible for trouble shooting, identifying and resolving data problems, worked with analysts to determine data requirements and identify data sources, provide estimates for task duration.
  • Gather information from different data warehouse systems and loaded into warehouse using Fast Load, Fast Export, Xml import, Multi Load, BTEQ, Teradata parallel transporter (TPT) and UNIX shell scripts.
  • Generated the Business Objects reports involving complex queries, sub queries, Unions and Intersection.
  • Involved in unit testing, systems testing, integrated testing, Data validation and user acceptance testing.
  • Involved in 24x7 production support.

Technologies Used: Teradata V13.0, Cloudera-Hadoop, HDFS, MapReduce, Sqoop, Hive, Teradata SQL Assistant, SQL Server, Flat files, SQL, Erwin, Windows RDP servers, Linux, Shell Scripting

Confidential, Atlanta, GA

Hadoop/DWH Developer

Responsibilities:

  • Involved in the performance tuning of ETL code review and analyze the target-based commit interval for optimum session performance.
  • Developing Data Extraction, Transformation and Loading jobs from flat files, Oracle, SAP and Teradata Sources into Teradata using BTEQ, Fast Load, MultiLoad and stored procedure.
  • Load data into DWH tables using FASTLOAD & MULTILOAD Utilities in staging & core environment for downstream applications.
  • Export and import data in to HDFS environment using Sqoop with applying some filters as per business requirements.
  • Applied data transformations & data cleaning activities using Apache Hive & PIG tools in cloudera platform to implement Hadoop ecosystem.
  • Performance tuning at application, database & code level and implemented SQL optimization techniques.
  • Well versed in handling advanced SQL queries for reporting & analytical purposes.
  • Involved in unit testing, systems testing, integrated testing, Data validation and user acceptance testing.
  • Involved in 24x7 production support.

Confidential, Atlanta, GA

Data Analyst/Teradata Consultant

Responsibilities:

  • Involved in Designing the ETL process to Extract transform and load data from OLAP to Teradata data warehouse.
  • Used BTEQ and SQL Assistant front-end tools to issue SQL commands matching the business requirements to Teradata RDBMS.
  • Involved in the performance tuning of ETL code review and analyze the target-based commit interval for optimum session performance.
  • Was responsible for Tuning Report Queries and ADHOC Queries.
  • Wrote transformations for data conversions into required form based on the client requirement using Teradata ETL processes.
  • Experienced in Tuning SQL Statements and Procedures for enhancing the load performance in various schemas across databases. Tuning the queries to improve the performance of the report refresh time.
  • Generated different Space Reports in Teradata Manager to analyze different kind of issues
  • Provided ongoing support by developing processes and executing object migrations, security and access privilege setup and active performance monitoring.
  • Expertise in using Visual Explain, Index wizard, Statistics wizard to tune the bad queries and analyze the plans implement the recommendations to gain the performance.
  • Developed scripts to load high volume data into empty tables using Fast Load utility.
  • Wrote SQL queries and matching the data with database and reports.
  • Tuned and Enhanced Universes with SQL Queries for the Report Performance.
  • Created complicated reports including sub-reports, graphical reports, formula base and well-formatted reports according user requirements.
  • Involved in working with SSA requestor responsibilities which will be assigned for both project and support requests.
  • Worked on different data stores and file formats in web services.
  • Used Fast Export utility to extract large volumes of data at high speed from Teradata warehouse.
  • Performance tuning for Teradata SQL statements using huge volume of data.
  • Created Fast Load, Fast Export, Multi Load, TPump, and BTEQ to load data from Oracle database and Flat files to primary data warehouse.
  • Developed several Informatica Mappings, Mapplets and Transformations to load data from relational and flat file sources into the data mart.
  • Created UNIX scripts for various purposes like FTP, Archive files and creating parameter files.
  • Scripts were run through UNIX shell programs in Batch scheduling.
  • Created procedures to delete duplicate records from warehouse tables.
  • Used Informatica debugging techniques to debug the mappings and used session log files and bad files to trace errors occurred while loading.
  • Was responsible for trouble shooting, identifying and resolving data problems, worked with analysts to determine data requirements and identify data sources, provide estimates for task duration.
  • Involved in unit testing, systems testing, integrated testing, Data validation and user acceptance testing.
  • Involved in 24x7 production support.

Technologies Used: Teradata V13.0, CDC, Business Objects XIR3.1, Mainframes, Oracle 10G, DB2, Teradata SQL Assistant, SQL Server, Flat files, SQL, Erwin, Linux, Shell Scripting

Confidential, CA

Data Analyst/ Teradata Consultant

Responsibilities:

  • Used Stored Procedures created Database Automation Script to create databases in different Environments.
  • Generated different Space Reports in Teradata Manager to analyze different kind of issues
  • Provided ongoing support by developing processes and executing object migrations, security and access privilege setup and active performance monitoring.
  • Expertise in using Visual Explain, Index wizard, Statistics wizard to tune the bad queries and analyze the plans implement the recommendations to gain the performance.
  • Teradata performance tuning via Explain, PPI, AJI, Indices, collect statistics or rewriting of the code.
  • Developed BTEQ scripts to load data from Teradata Staging area to Teradata data mart
  • Developed scripts to load high volume data into empty tables using Fast Load utility.
  • Created Reports using all the BO functions like Drill Down, Prompts, Dimensional and Measure variables to show accurate Results.
  • Wrote SQL queries and matching the data with database and reports.
  • Tuned and Enhanced Universes with SQL Queries for the Report Performance.
  • Created complicated reports including sub-reports, graphical reports, formula base and well-formatted reports according user requirements.
  • Involved in working with SSA requestor responsibilities which will be assigned for both project and support requests.
  • Worked on different data stores and file formats in web services.
  • Used Fast Export utility to extract large volumes of data at high speed from Teradata warehouse.
  • Performance tuning for Teradata SQL statements using huge volume of data.
  • Created Fast Load, Fast Export, Multi Load, TPump, and BTEQ to load data from Oracle database and Flat files to primary data warehouse. Created procedures to delete duplicate records from warehouse tables.
  • Developed several Informatica Mappings, Mapplets and Transformations to load data from relational and flat file sources into the data mart. Scripts were run through UNIX shell programs in Batch scheduling.
  • Created UNIX scripts for various purposes like FTP, Archive files and creating parameter files.
  • Used Informatica debugging techniques to debug the mappings and used session log files and bad files to trace errors occurred while loading.
  • Responsible for trouble shooting, identifying and resolving data problems, worked with analysts to determine data requirements and identify data sources, provide estimates for task duration.
  • Gather information from different data warehouse systems and loaded into warehouse using Fast Load, Fast Export, Xml import, Multi Load, BTEQ, Teradata parallel transporter (TPT) and UNIX shell scripts.
  • Generated the Business Objects reports involving complex queries, sub queries, Unions and Intersection.
  • Involved in unit testing, systems testing, integrated testing, Data validation and user acceptance testing.
  • Involved in 24x7 production support.

Technologies Used: Teradata V13.0, SAP, Informatica 8.1.1, MicroStrategy, Business Objects XIR3.1, Crystal reports, Oracle 10G, DB2, Teradata SQL Assistant, SQL Server, Flat files, TOAD 9.x, SQL, Erwin, Linux, Shell Scripting

Confidential, Richmond, VA

Data Analyst/ Teradata Developer

Responsibilities:

  • Experienced in Tuning SQL Statements and Procedures for enhancing the load performance in various schemas across databases. Tuning the queries to improve the performance of the report refresh time.
  • Created customized Web Intelligence reports from various sources of data.
  • Involved in performance tuning on the source and target database for querying and data loading.
  • Developed MLoad scripts and shell scripts to move data from source systems to staging and from staging to Data warehouse in batch processing mode.
  • Developed BTEQ scripts to load data from Teradata Staging area to Teradata data mart
  • Developed scripts to load high volume data into empty tables using Fast Load utility.
  • Created Reports using all the BO functions like Drill Down, Prompts, Dimensional and Measure variables to show accurate Results. Writing SQL queries and matching the data with database and reports.
  • Created complicated reports including sub-reports, graphical reports, formula base and well-formatted reports according user requirements. Tuned and Enhanced Universes with SQL Queries for the Report Performance.
  • Developed Data Extraction, Transformation and Loading jobs from flat files, Oracle, SAP, Teradata Sources into Teradata using BTEQ, Fast Load, MultiLoad and stored procedure.
  • Designed of process-oriented UNIX script and ETL processes for loading data into data warehouse.
  • Used Stored Procedures created Database Automation Script to create databases in different Environments
  • Analyzed the Business requirements and System specifications to understand the Application.
  • Designed Informatica mappings to propagate data from various legacy source systems to Oracle. The interfaces were staged in Oracle before loading to the Data warehouse. Exported data from Teradata database using Teradata Fast Export.
  • Performed Data transformations using various Informatica Transformations like Union, Joiner, Expression, Lookup, Aggregate, Filter, Router, Normalizer, Update Strategy, etc.
  • Experienced in Tuning SQL Statements and Procedures for enhancing the load performance in various schemas across databases. Tuning the queries to improve the performance of the report refresh time.
  • Created customized Web Intelligence reports from various sources of data.
  • Involved in performance tuning on the source and target database for querying and data loading.
  • Created, loading and materializing views to extend the usability of data.
  • Automated Unix shell scripts to verify the count of records added everyday due to incremental data load for few of the base tables in order to check for the consistency. Loading the data into the warehouse from different flat files.

Technologies Used: Teradata V12.0, Informatica, Business Objects XIR3.1, Crystal reports, Teradata Utilities (Multiload, FastLoad, Fast Export, BTEQ, TPump), SQL Server 2000, Sybase, DB2, Oracle, FTP, CVS, Windows XP, UNIX, Pentium Server.

Hire Now