SUMMARY

• Around 7+ years of IT experience as on Azure Cloud. Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and controlling and granting database access and Migrating On premise databases to Azure Data Lake store using Azure Data factory.
• Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming teh data to uncover insights into teh customer usage patterns.
• Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
• Experience wif MS SQL Server Integration Services (SSIS), T-SQL skills, stored procedures, triggers.
• Design and develop Spark applications using Pyspark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming teh data to uncover insights into teh customer usage patterns.
• Good understanding of Big Data Hadoop and Yarn architecture along wif various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-processing).
• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL
• Experience in Database Design and development wif Business Intelligence using SQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS), DAX, OLAP Cubes, Star Schema and Snowflake Schema.
• Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion.
• Worked in mixed role DevOps: Azure Architect/System Engineering, network operations and data engineering.
• Excellent communication skills wif excellent work ethics and a proactive team player wif a positive attitude.
• Created Azure SQL database, performed monitoring and restoring of Azure SQL database. Performed migration of Microsoft SQL server to Azure SQL database.
• Have designed and developed ETL mapping for data collection from various data feeds using REST API. Teh data sources include feeds from mobile, social Facebook Query Language, You tube, Twitter, web, and other partner feeds.
• Expertise in various phases of project life cycles (Design, Analysis, Implementation, and testing).
• Proficiency in handling software development sprints, test, and delivery cycle for teh development teams.
• Excellent communication, team membership and relationship management skills.

TECHNICAL SKILLS

Big Data Technologies: SQL Azure, Azure Storage

Microsoft Azure: Azure Stack, AKS, ARM, NSG, ASR, Azure DevOps project, IoT Hub, On-Premises migration to Azure, ALM, Service Bus, Event Hub, CDN, Azure Active Directory, Storage, Backup-Restore, Release management Web App, Web jobs, Web Role, Worker Role, Cloud Service, Traffic manager, OMS, API Management, Data Lake, Data factory, Virtual Network, VSTS, Azure Automation, and Many others.

Languages/Script: Shell Scripts, PowerShell, Ant,Python, .Net & C#

Database: MS SQL Azure, MS SQL Server, DB2, Oracle

Operating Systems: Windows 8, Windows 10, Windows Server 10, Linux

Domain Expertise: Banking, Financial services and Insurance

PROFESSIONAL EXPERIENCE

Confidential, Dallas,Texas

Azure Data Engineer

Responsibilities:

• Analyze, design, and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine teh impact of new implementation on existing business processes.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics.
• Used Liquibase for database migration and JDBC for database connections.
• Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing teh data in InAzure Databricks.
• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.
• Completed online data transfer from AWS S3 to Azure Blob by using Azure Data Factory.
• Used Azure Migrate to get started migrating your AWS EC2 instances over to Azure.
• Developed Spark applicationsusingPysparkandSpark-SQLfor data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming teh data to uncover insights into teh customer usage patterns.
• Responsible for estimating teh cluster size, monitoring, and troubleshooting of teh Spark databricks cluster.
• Migration of on-premises data (SQL Server / MongoDB) to Azure Data Lake Store (ADLS) using Azure Data Factory (ADF V1/V2).
• Exposures wif Azure Active Directory compatibility. Extensive experience in deployment,migration, patching and troubleshooting of windows 2008 and 2012 R2 Domain Controllers in Active Directory.
• Experienced in performance tuning of Spark Applications for setting right Batch Interval time, correct level of Parallelism and memory tuning.
• To meet specific business requirements wrote UDF’s inScalaandPyspark.
• Developed JSON Scripts for deploying teh Pipeline in Azure Data Factory (ADF) that process teh data using teh Sql Activity.
• Hands-on experience on developing SQL Scripts for automation purpose.
• Created Build and Release for multiple projects (modules) in production environment using Visual Studio Team Services (VSTS).

Confidential, Dayton, OH

Azure Data Engineer

Responsibilities:

• Design and implement end-to-end data solutions (storage, integration, processing, visualization) in Azure.
• Worked on migration of data from On-prem SQL server to Cloud databases(Azure Synapse Analytics (DW) & Azure SQL DB).
• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.
• Migrate data from traditional database systems to Azure databases.
• Build Complex distributed systems involving huge amount data handling, collecting metrics building data pipeline, and Analytics.
• Experience managing Azure Data Lakes (ADLS) and Data Lake Analytics and an understanding of how to integrate wif other Azure Services like Synapse and Azure Data Factory.
• Developed and created pipelines, jobs, scheduling triggers, Mapping data flows using Azure Data Factory(V2) and using Key Vaults to store credentials.
• Worked on Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics (DW).
• Had used T-SQL to write stored procedures, triggers, functions, tables, views, indexes and relational database models.
• SSIS performance tuning using counters, error handling, event handling, re-running of failed SSIS packages using checkpoints and scripting wif Active-X and VB.NET in SSIS.
• Development using SSIS script task, look up transformations and data flow tasks using T- SQL and Visual Basic (VB) scripts.
• Transferred teh data (ETL) to data warehouse by SSIS and processed SSAS cubes to store data to OLAP databases.
• Performance Monitoring wif SQL Profiler Windows System Monitor.
• Contributed to teh project's overall understanding of Indiana Medicaid Management Information System Core MMIS.

Environment: Azure Data Lake & BLOB, Azure SQL, Azure data factory,MS SQL Server 2018,SQL Profiler, BIDS, Management Studio (SSMS), ETL, Integration Services (SSIS),DevOps.

Confidential, Denver, CO

Azure Data Engineer

Responsibilities:

• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.
• Architect & implement medium to large scale BI solutions on Azure using Azure Data Platform services (Azure Data Lake, Data Factory, Data Lake Analytics, Stream Analytics, Azure SQL DW, HDInsight/Databricks, NoSQL DB).
• Design & implement migration strategies for traditional systems on Azure (Lift and shift/Azure Migrate, other third-party tools.
• Engage wif business users to gather requirements, design visualizations and provide training to use self-service BI tools.
• Used various sources to pull data into Power BI such as SQL Server, Excel, Oracle, SQL Azure etc.
• Propose architectures considering cost/spend in Azure and develop recommendations to right-size data infrastructure.
• Develop conceptual solutions & create proof-of-concepts to demonstrate viability of solutions.
• Technically guide projects through to completion wifin target timeframes.
• Collaborate wif application architects and DevOps.
• Identify and implement best practices, tools and standards.
• Design Setup maintain Administrator teh Azure SQL Database, Azure Analysis Service, Azure SQL Data warehouse, Azure Data Factory, Azure SQL Data warehouse.
• Build Complex distributed systems involving huge amount data handling, collecting metrics building data pipeline, and Analytics.

Confidential

Data Engineer

Responsibilities:

• Extract Transform and Load data from Sources system to Data Warehouse using a combination of SSIS, T-SQL, Spark SQL.
• Worked on migration and conversion of data using Pyspark and Spark SQL for data extraction, transformation and aggregation from multiple file formats for analyzing and transforming using Python.
• Ability to apply Data FrameAPI to complete Data Manipulation wifin spark session.
• Created Data Quality Scripts to compare data built from spark data frame API.
• Design and develop ETL Integration patterns using python on spark.
• Analyzed SQL scripts and design it by using PySpark SQL for faster performance.
• Performed ETL Transformation activities in SSIS and built several packages and loaded data to
• Data warehouse
• Involved in writing stored procedures in T-SQL do teh transformations of teh data
• Involved in Data Modeling
• Engage wif business users to gather requirements, design visualizations and provide training to use self-service BI tools.

Environment: MS SQL Server 2018,SQL Profiler, BIDS, Management Studio (SSMS), ETL, Integration Services (SSIS),PowerBI,Pyspark,