SUMMARY

• 9+ years of professional experience in IT, which includes work experience in Big Data, Hadoop ecosystem for Data Processing, Data Warehousing and Data Pipeline design and implementation.
• Expert knowledge in SDLC (Software Development Life Cycle) and participated in all phases in projects.
• Expertise in using Cloud based managed services for data warehousing in Confidential Azure (Azure Data Lake Storage, Azure Data Factory).
• Strong experience with big data processing using big data tools like Map Reduce, Apache Spark, Apache Hive and Pig.
• Good understanding of cloud configuration in Amazon web services (AWS).
• Have experience in Dimensional Modeling using Star and Snowflake schema methodologies of Data Warehouse and Integration projects.
• Excellent proficiency in Agile/Scrum and Waterfall methodologies.
• Experience in setting up the enterprise infrastructure on Amazon Web Services (AWS) like EC2 instance and S3.
• Experience in performing data analysis on various IDE's like PyCharm, Jupyter notebook, Spyder.
• Experience in using various packages in R and python - like ggplot2, NumPy, Pandas, Matplotlib, SciPy for Data Analysis, and Model Building.
• Good experience in creating visualizations, interactive dashboards, reports, using Tableau, Power BI.
• Expertise in ETL (Data Extraction, Transformation & Loading), Data Modeling, Report designing, and Performance Tuning using QlikView.
• Experience in using ER modeling tools such as Erwin and ER/Studio.
• Experience in integration of various data sources with multiple Relational Databases like SQL Server, Teradata, and Oracle.
• Experience in Data Ingestion projects to inject data into Data Lake using multiple sources systems using Talend BigData.
• Excellent knowledge in Data Analysis, Data Validation, Data Cleansing, Data Verification and identifying data mismatch.
• Experienced with cloud platforms like AWS and Azure.
• Proficient in data governance, data quality, metadata management, master data management.
• Experience in working with creating ETL specification documents, & creating flowcharts, process workflows and data flow diagrams.
• Experience in execution of Batch jobs through the data streams to SPARK Streaming.
• Good knowledge in streaming applications using Apache Kafka.
• Hands on experience in working with Tableau Desktop, Tableau Server, and Tableau Reader in various versions.
• Extending HIVE and PIG core functionality by using custom UDF's.
• Experience in designing both time driven, and data driven automated workflows using Oozie.
• Expertise in SQL Server Analysis Services (SSAS), SQL Server Reporting Services (SSRS) and SQL Server Integration Services (SSIS).
• Strong experience in using MS Excel and MS Access to dump and analyze the data based on business needs.
• Good communication skills, work ethics and the ability to work in a team efficiently with good leadership skills.

TECHNICAL SKILLS

BI Tools: Tableau 10, Tableau Server, SAP BusinessObjects, Microsoft Power BI

Languages: Java, PL-SQL, C, SQL, Python, UNIX Shell Scripting, Scala

Big Data Technologies: Apache Spark, Hadoop, Apache Kafka, Apache Sqoop, Apache ZooKeeper, Hive, Apache Oozie, Apache HBase

Data Visualization: AWS QuickSight, Tableau, Cognos, Microsoft Excel, Power Point, Informatica, Spotfire

Methodologies: JAD, System Development Life Cycle (SDLC), Agile, Waterfall Model

ETL Tools: Informatica 9.6/9.1, Tableau Prep, Pentaho

Data Analysis: Web Scraping, Data Visualization, Statistical Analysis, Data Mining, Data Warehouse, Data Migration, Database Management

Data Modeling Tools: Erwin Data Modeler 9.8, ER/Studio v17, PowerDesigner 16.6

Web Technologies: JavaScript, XML, XSL, HTML, XHTML, AJAX, ASP.NET, CSS

Cloud Architecture: Amazon AWS, EC2, EMR, Basic MS Azure, Google Cloud Platform (GCP)

Deployment Tools: Force.com IDE, Force.com Migration Tool (ANT)

PROFESSIONAL EXPERIENCE

Confidential - Eden Prairie, Minnesota

Data Engineer

Responsibilities:

• Developed complete end to end Big-data processing in Hadoop eco-system.
• Provided application support during the build and test phases of the SDLC for their product.
• Used Oozie for automating the end-to-end data pipelines and Oozie coordinators for scheduling the workflows.
• Recreated existing application logic and functionality in the Azure Data Lake, Data Factory, Data Bricks, SQL Database and SQL Data warehouse environment.
• Performed data profiling and transformation on the raw data using Pig, Python, and Oracle.
• Developed predictive analytics using Apache Spark Scala APIs.
• Performed end-to-end delivery of PySpark ETL pipelines on Azure Databricks to perform the transformation of data orchestrated via Azure Data Factory (ADF) scheduled through azure automation accounts and trigger them using Tidal Schedular.
• Created dimensional model for the reporting system by identifying required dimensions and facts using Erwin.
• Developed and implemented a data pipeline using Kafka and Strom to store data into HDFS.
• Created automated Python scripts to convert the data from various sources and to generate the ETL pipelines.
• Worked with Snowflake SaaS for cost effective data warehouse implementation on cloud.
• Designed and implemented database solutions in Azure SQL Data Warehouse.
• Developed customer cleanse functions, cleanse lists and mappings for MDM Hub.
• Worked extensively on Oracle PL/SQL, and SQL Performance Tuning.
• Extensively worked on Python Open Stack API's.
• Involved in modeling (Star Schema methodologies) in building and designing the logical data models into Dimensional Models.
• Created shared dimension tables, measures, hierarchies, levels, cubes, and aggregations on MS OLAP/ OLTP/Analysis Server (SSAS).
• Created indexes, both non-clustered and clustered indexes to maximize the query performance in the T-SQL.
• Created Hive External Tables and loaded the data into tables and query data using HQL.
• Generated multiple enterprise reports like SSRS and Crystal report, worked on Tableau.
• Managed Azure Data Lakes (ADLS) and Data Lake Analytics and got an understanding of how to integrate with other Azure Services.
• Wrote MapReduce jobs to generate reports for the number of activities created on a day, dumped from the multiple sources and the output was written back to HDFS.
• Used Sqoop to efficiently transfer data between databases and HDFS.
• Developed Spark code using Scala and Spark-SQL for faster testing and data processing.
• Used Pig as ETL tool to do transformations, joins and some pre-aggregations before storing the data into HDFS.
• Used Hive to analyze data ingested into HBase by using Hive-HBase integration and compute various metrics for reporting on the dashboard.
• Worked on creating Ad hoc reports, Database Imports and Exports using SSIS.

Environment: Erwin, SQL, Oracle 12c, PL/SQL, Big data, Azure Data Lake, Spark, APIs, Pig, Python, Kafka, HDFS, ETL, MDM, OLAP, OLTP, SSAS, T-SQL, Hive, Oozie, SSRS, Tableau, MapReduce, Sqoop, Scala, HBase, SSIS.

Confidential - Plano, TX

Data Engineer

Responsibilities:

• Followed Test driven development of Agile Methodology to produce high quality software.
• Designed and developed a horizontally scalable APIs using Python Flask.
• Conducted JAD sessions, wrote meeting minutes and documented the requirements.
• Worked with cloud providers and APIs for Amazon (AWS) EC2, S3, VPC with GFS2 storage.
• Worked with Data ingestion, querying, processing, and analysis of big data.
• Performed tuning and optimized various complex SQL queries.
• Developed normalized Logical and Physical database models to design OLTP system.
• Extensively involved in creating PL/SQL objects i.e., Procedures, Functions, and Packages.
• Performed bug verification, release testing and provided support for Oracle based applications.
• Used Model Mart of Erwin for effective model management of sharing, dividing and reusing model information and design for productivity improvement.
• Extensively used Hive optimization techniques like partitioning, bucketing, MapJoin and parallel execution.
• Worked with Real-time Streaming using Kafka and HDFS.
• Worked with Alteryx, a data Analytical tool to develop workflows for the ETL jobs.
• Designed the data marts in dimensional data modeling using star and snowflake schemas.
• Wrote, tested, and implemented Teradata FastLoad, Multiload, DML and DDL.
• Used various OLAP operations like slice/dice, drill down and roll up as per business requirements.
• Wrote SQL queries, stored procedures, views, triggers, T-SQL and DTS/SSIS.
• Handled importing of data from various data sources, performed data control checks using Spark and loaded data into HDFS.
• Designed SSRS reports with sub reports, dynamic sorting, defining data source and subtotals for the reports.
• Designed and implemented importing data to HDFS using Sqoop from different RDBMS servers.
• Gathered SSRS reports requirements and created the reports in Tableau.
• Designed and developed MapReduce jobs to process data coming in different file formats like XML.

Environment: Erwin, SQL, PL/SQL, Kafka1.1, AWS, API's, Agile, ETL, HDFS, OLAP, T-SQL, SSIS, Teradata, Hive, SSRS, Sqoop, Tableau, MapReduce, XML.

Confidential

Data Engineer

Responsibilities:

• Spearheaded analysis, design, and development for building a common architecture and data repository for Apparel and Footwear sourcing data across the Geo's for data management and reporting needs.
• Work cross functionally with analysts, business insights, report developers, and stakeholders to ensure a tight Development/QA/UAT process and quality deployments.
• Evaluate, cleanse, extract/transform data for analytic purpose within the context of big data environment and structure large data sets by applying standard data modeling methods.
• Designed and developed efficient PySpark programs using cloud-based data platforms (EMR) to extract/transform/load data in between various data warehouse applications.
• Worked on Cloud Platforms like AWS and possess good knowledge on different types of instances for optimal usage of clusters based on the requirement.
• Engineered a solution to optimize the ETL process of Alteryx to Snowflake ingestion process which takes around 7+ hours into a more reliable process by using PySpark which simplified and drastically reduced the data ingestion time to 7 minutes.
• Designed Hive tables over the parquet and csv files for loading and analysing data.
• Involved in developing DAGS using Airflow orchestration tool and monitored the weekly processes.
• Identify gaps in data processes and drive improvements via continuous improvement loop by ensuring good data flow between databases and backend systems.

Environment: Python, Alteryx, AWS S3, AWS EMR, Spark, Hive, Airflow, Sqoop, Teradata SQL assistant, Snowflake, MS SQL Server 2016, Tableau 2018.3, Snowflake database, R - Programming, SharePoint, SSIS.

Confidential

Data Analyst /Data Modeler

Responsibilities:

• As a Data Analyst /Data Modeler I was responsible for all data related aspects of the project.
• Created reports on Cloud based environment using Amazon Redshift and published on Tableau.
• Developed Python APIs to dump the array structures in the Processor at the failure point for debugging.
• Worked extensively on ER/Studio in several projects in both OLAP and OLTP applications.
• Created SQL tables with referential integrity and developed queries using SQL, SQL*PLUS and PL/SQL.
• Performed Data Analysis and Data Profiling using complex SQL on various sources systems including Oracle.
• Developed the required data warehouse model using Star schema for the generalized model.
• Implemented Visualized BI Reports with Tableau.
• Worked on stored procedures for processing business logic in the database.
• Extensively worked on Teradata Viewpoint to look at performance monitoring and performance tuning.
• Performed Extract, Transform and Load (ETL) solutions to move legacy and ERP data into Oracle data warehouse.
• Developed and maintained data dictionary to create metadata reports for technical and business purpose.
• Managed database design and implemented a comprehensive Snowflake-Schema with shared dimensions.
• Worked on Normalization and De-normalization concepts and design methodologies.
• Worked on the reporting requirements for the data warehouse.
• Worked on SQL Server concepts like SSIS (SQL Server Integration Services), SSAS (Analysis Services) and SSRS (Reporting Services).
• Developed complex T-SQL and PL/SQL codes for the business application to validate data and testing processes.

Environment: ER/Studio, SQL, Python, APIs, OLAP, OLTP, PL/SQL, BI, Tableau, ETL, SSIS, SSAS, SSRS, T-SQL, Redshift, Oracle, Teradata.