SUMMARY

• Around 6+ years of IT experience as on Azure Cloud . Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and Controlling and granting database access and Migrating On premise databases to Azure Data lake store using Azure Data factory.
• Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
• Experience with MS SQL Server Integration Services (SSIS), T-SQL skills, stored procedures, triggers.
• Design and develop Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Big Data Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-processing).
• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL
• Experience in Database Design and development with Business Intelligence using SQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS), DAX, OLAP Cubes, Star Schema and Snowflake Schema.
• Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion .
• Worked in mixed role DevOps: Azure Arcitect/System Engineering, network operations and data engineering.
• Excellent communication skills with excellent work ethics and a proactive team player with a positive attitude.
• Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and Controlling and granting database access and Migrating On premise databases to Azure Data lake store using Azure Data factory.
• Created Azure SQL database, performed monitoring and restoring of Azure SQL database. Performed migration of Microsoft SQL server to Azure SQL database.
• Having the ability to perform preliminary investigation of interface-related issues identified during the implementation phase and has the ability to search for, and edit HL7 messages through Interface Admin.
• Have designed and developed ETL mapping for data collection from various data feeds using REST API. The data sources include feeds from mobile, social Facebook Query Language, You tube, Twitter, web and other partner feeds.
• Expertise in various phases of project life cycles (Design, Analysis, Implementation and testing).

TECHNICAL SKILLS

• Azure Data lake, Data factory
• Azure Databricks, Azure SQL database, Azure SQL Datawarehouse
• SQL Server 2017, SQL Server 2016, SQL Server 2014
• Programming Scala, Python, Spark SQL
• MSBI (SSIS, SSAS, SSRS)
• Data Visualization
• Data Migration
• SQL Server programming
• Confidential Power BI
• Analytic Problem-Solving

PROFESSIONAL EXPERIENCE

Confidential

Azure Data Engineer

Responsibilities:

• Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.
• Implemented Proof of concepts for SOAP & REST APIs
• REST APIs to retrieve analytics data from different data feeds
• Collaborate with data architect / engineers to establish data governance / cataloging for MDM/ security (key vault, network security schema level & row level), resource groups, integration runtime setting, integration patterns, aggregated functions for data bricks development.
• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.
• Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Expertise in Creating, Debugging, Scheduling and Monitoring jobs using Airflow and Oozie.
• Created data bricks notebooks using Python(PySpark),Scala and Spark SQL for transforming the data that is stored in Azure Data Lake stored Gen2 from Raw to Stage and Curated zones.
• Built numerous technology demonstrators using Confidential Edison Arduino shield using Azure EventHub and Stream Analytics, integrated with PowerBI and Azure ML to demonstrate the capabilities of Azure Stream Analytics.
• Responsible for estimating the cluster size, monitoring and troubleshooting of the Spark data bricks cluster.
• Experienced in performance tuning of Spark Applications for setting right Batch Interval time, correct level of Parallelism and memory tuning.
• Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the Sql Activity.
• Hands-on experience on developing SQL Scripts for automation purpose.
• Created Build and Release for multiple projects (modules) in production environment using Visual Studio Team Services (VSTS).

Confidential

Data Engineer

Responsibilities:

• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.
• Architect & implement medium to large scale BI solutions on Azure using Azure Data Platform services (Azure Data Lake, Data Factory, Data Lake Analytics, Stream Analytics, Azure SQL DW, HDInsight/Databricks, NoSQL DB).
• Design & implement migration strategies for traditional systems on Azure (Lift and shift/Azure Migrate, other third-party tools.
• Engage with business users to gather requirements, design visualizations and provide training to use self-service BI tools.
• Used various sources to pull data into Power BI such as SQL Server, Excel, Oracle, SQL Azure etc.
• Propose architectures considering cost/spend in Azure and develop recommendations to right-size data infrastructure.
• Created Airflow Scheduling scripts in Python
• Develop conceptual solutions & create proof-of-concepts to demonstrate viability of solutions.
• Technically guide projects through to completion within target timeframes.
• Collaborate with application architects and DevOps.
• Identify and implement best practices, tools and standards.
• Design Setup maintain Administrator the Azure SQL Database, Azure Analysis Service, Azure SQL Data warehouse, Azure Data Factory, Azure SQL Data warehouse.
• Build Complex distributed systems involving huge amount data handling, collecting metrics building data pipeline, and Analytics.

Confidential

SQL Server Developer

Responsibilities:

• Manage a centralized MS SQL Server 2014, SQL 2012 and 2008R2 Platform consisting of thousands of mission critical databases.
• Expertise in High Availability and Recoverability of databases using standard MS SQL technologies including Always On.
• Migrate SQL Server and Oracle database to Microsoft Azure Cloud.
• Migrate the Data using Azure database Migration Service (AMS).
• Provide on Call Support for Prod mission-critical (24x7) or problematic databases, and own problem resolution from end-to-end.
• Worked Azure SQL Database Environment.
• Manage SQL Server databases through multiple project lifecycle environments, from development to mission-critical production systems.
• Design, setup administrator AlwaysOn high-availability (HA) and Disaster recovery (DR) solution in SQL Server 2014.
• Worked in roadmap development migration to all lower version of SQL Databases to Microsoft SQL Server 2014.
• Responsible for capturing various database, server, and application metrics, observing trends that will help longer-term capacity planning and analysis efforts.
• Design, Implement and maintain of Disaster Recovery (DR) and High Availability (HA) solutions with Log Shipping, Replication Database Mirroring, etc.
• Extensive worked on Migrating Databases residing on servers in one data center to another and Upgrading Database platforms to newer versions.
• Experience working with Windows Hyper-V Server, Azure, Windows Clustering including.
• Participate in software design reviews with developers to ensure that the architecture is scalable to meet long-term performance goals.