SUMMARY

• Around 8 years of IT experience with Microsoft SQL Server 2000/2005/2008/2012/2017 , SSIS/SSRS in OLTP and OLAP (Data warehousing) environments on Azure Cloud. Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and controlling and granting database access and Migrating On premise databases to Azure Data Lake store using Azure Data factory.
• Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
• Experience with MS SQL Server Integration Services (SSIS), T-SQL skills, stored procedures, triggers.
• Design and develop Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Big Data Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-processing).
• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL
• Experience in Database Design and development with Business Intelligence using SQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS), DAX, OLAP Cubes, Star Schema and Snowflake Schema.
• Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion.
• Worked in mixed role DevOps: Azure Architect/System Engineering, network operations and data engineering.
• Excellent communication skills with excellent work ethics and a proactive team player with a positive attitude.
• Created Azure SQL database, performed monitoring and restoring of Azure SQL database. Performed migration of Microsoft SQL server to Azure SQL database.
• Have designed and developed ETL mapping for data collection from various data feeds using REST API.
• Experience in analyzing data using Python, R, SQL, Microsoft Excel,Hive, PySpark, Spark SQL for Data Mining, Data Cleansing, Data Munging and Machine Learning.
• Expertise in various phases of project life cycles (Design, Analysis, Implementation and testing).
• Implemented Unit testing and involved in different SDLC models like Agile, waterfall and Knowledge in scrum model.
• Experience in Performance Tuning, optimization, data integrity and statistics by using SQL Profiler.
• Have worked in multiple projects as BI Developer in Designing ETL Packages and reports in development and implementation stages of the Project according to client requirements.
• Experience in SQL server performance tuning in Stored Procedures and creating required indexes on tables, understand the execution plan to improve the Sql queries, joins, views, functions.
• Involved in MSBI projects with extensive usage of Data Warehouse, ETL & Reporting tools like SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), SQL Server Reporting Service (SSRS), Data Warehousing, ETL/ELT, Data Migration, Production Support, Parallel Data Warehouse (PDW/MPP).

TECHNICAL SKILLS

• Azure Data Lake
• Data factory
• Azure Databricks
• Azure SQL database
• Azure SQL Datawarehouse
• SQL Server programming
• Programming Scala
• Python
• Spark SQL
• MSBI (SSIS
• SSAS
• SSRS)
• Data Visualization
• Data Migration
• SQL Server programming
• Confidential Power BI
• Analytic Problem-Solving

PROFESSIONAL EXPERIENCE

Confidential, SACRAMENTO CA

Azure Data Engineer

Responsibilities:

• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.
• Developed Spark applications usingPysparkandSpark-SQLfor data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Responsible for estimating the cluster size, monitoring and troubleshooting of the Spark databricks cluster.
• Loaded CSV files containing user event information into Hive External tables on daily basis.
• Created Spark applications and used Data frames and Spark-SQL API primarily for performing event enrichment and performing lookups with other enterprise data sources.
• Perform ETL on different formats of data like JSON, CSV files and converted them to parquet while loading to final tables.
• Developed a generic spark framework using Java to improve the ease of development of new applications.
• Converted Hive queries into Spark transformations using Spark data frames to enhance the performance reduce the memory used by the cluster.
• Developed batch jobs to read and transform data from SQL and Oracle database to hive external tables making use of Spark SQL.
• Written transformations and actions on data frames used SparkSQL on data frames to access hive tables into sparkfor faster processing of data.

Environment: Azure Data factory, Azure Databricks, Azure Data warehouse, Sql server 2016, Snowflake and Power BI, Python, Azure Data Lake, MSBI (SSIS, SSRS, SSAS).

Confidential, NORFOLK VA

Data Engineer

Responsibilities:

• Implemented POC on DataBricks to migrate Data science python libraries into DataBricks cluster.
• Working on importing and exporting data from SQL into HIVE using for visualization, analysis and to generate reports.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics.
• Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.
• Strongabilitytointerpretbusinessneedandtranslateitintotechnicaloutput.
• GathermappingandreportingrequirementsastheyrelatetoETLprocess.
• Translatecomplexbusinessrequirementsintocomplexdatamodels.
• Created stored procedures to automate the SQL DB backups and implement statistics on Data Base to Table level.
• Worked with offshore team and make them understand the requirements and make sure that on time deliverables.

Environment: Azure Data factory, Azure Databricks, Python, Sql Server 2018

Confidential

SQL BI Developer (SQL/SSIS/SSRS)

Responsibilities:

• Worked with Business analysts and users to gather business requirements, understand and implement them.
• Developing T-SQL queries, triggers, functions, cursors, and stored procedures.
• Responsible for logical and physical design of SQL Server databases.
• Designed and Implemented data extract from multiple sources into Target system using SSIS and SQL Stored producers, scripts.
• Used DDL and DML for writing triggers, stored procedures, and data manipulation.
• Created views to restrict access to data in a table for security.
• Expert in extracting data using SSIS from OLTP to OLAP.
• Performed Bulk data migration using DTS and SSIS from flat files using Data warehouse concepts.
• Designed and optimized all the SSIS packages.
• Created SSIS packages to export data from text file to SQL Server Database.
• Designed and implemented stylish report layouts.
• Identified the Data Source and defining them to build the Data Source Views.
• Identified and defined the Datasets for report generation and included report parameters.
• Generated Reports using Global Variables, Expressions and Functions for the reports.
• Implemented full and differential backups and created and managed DB objects.
• Tuning Queries, Stored producers and Database performance using tools Database Tuning Advisor, SQL Server Profiler.
• Developed reports using SSRS
• Scheduling backups and automating weekend maintenance tasks.
• Administering MS-SQL Server, performing day-to-day tasks such as Database tuning by checking indexes.
• Provided Training and provided enough documentation to them for understanding the system.
• Involved in All phases of software Development life cycle, Development, Testing, Regression Testing, UAT, and Finally Production validation.

Environment: MSSQL Server, SSIS, SSRS, SSAS