SUMMARY

• Around 10+ years of IT experience as on Azure Cloud. Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and controlling and granting database access and Migrating On premise databases to Azure Data Lake store using Azure Data factory.
• Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
• Experience with MS SQL Server Integration Services (SSIS), T-SQL skills, stored procedures, triggers.
• Strong experience in migrating other databases to Snowflake.
• Participates in the development improvement and maintenance of snowflake database applications
• Design and develop Spark applications using Pyspark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Good understanding of Big Data Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-processing).
• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL
• Experience in Database Design and development with Business Intelligence using SQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS), DAX, OLAP Cubes, Star Schema and Snowflake Schema.
• Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion.
• Worked in mixed role DevOps: Azure Architect/System Engineering, network operations and data engineering.
• Automated the build & deployments for many applications from end-to-end using CICD strategies with various DevOps tools and technologies.
• Excellent communication skills with excellent work ethics and a proactive team player with a positive attitude.
• Created Azure SQL database, performed monitoring and restoring of Azure SQL database. Performed migration of Microsoft SQL server to Azure SQL database.
• Have designed and developed ETL mapping for data collection from various data feeds using REST API.
• The data sources include feeds from mobile, social Facebook Query Language, You tube, Twitter, web, and other partner feeds.
• Expertise in various phases of project life cycles (Design, Analysis, Implementation, and testing).
• Proficiency in handling software development sprints, test, and delivery cycle for the development teams.
• Excellent communication, team membership and relationship management skills.

TECHNICAL SKILLS

Advance Analytics: Azure Machine Learning, Power BI

Big Data Technologies: SQL Azure, Azure Storage

Microsoft Azure: Azure Stack, AKS, ARM, NSG, ASR, Azure DevOps project, IoT Hub, On-Premises migration to Azure, ALM, Service Bus, Event Hub, CDN, Azure Active Directory, Storage, Backup-Restore, Release management Web App, Web jobs, Web Role, Worker Role, Cloud Service, Traffic manager, OMS, API Management, Data Lake, Data factory, Virtual Network, VSTS, Azure Automation, and Many others.

Languages/Script: Shell Scripts, PowerShell, Ant, .Net & C#

Database: MS SQL Azure, MS SQL Server, DB2, Oracle

Operating Systems: Windows 8, Windows 10, Windows Server 10, Linux

Domain Expertise: Banking, Financial services and Insurance

PROFESSIONAL EXPERIENCE

Confidential

Azure Data Engineer

Responsibilities:

• Analyze, design, and build Modern data solutions using Azure PaaS service to support visualization of data.
• Understand current Production state of application and determine the impact of new implementation on existing business processes.
• Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics.
• Used Liquibase for database migration and JDBC for database connections.
• Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks.
• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.
• Have good experience working with Azure BLOB and Data Lake Storage and loading data into Azure SQL Synapse analytics (DW)
• Completed online data transfer from AWS S3 to Azure Blob by using Azure Data Factory.
• Used Azure Migrate to get started migrating your AWS EC2 instances over to Azure.
• Developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Responsible for estimating the cluster size, monitoring, and troubleshooting of the Spark data bricks cluster.
• Develop stored procedures/views in Snowflake and use in Talend for loading Dimensions and Facts.
• Experience with Snowflake cloud data warehouse and Azure Data Lake for integrating data from multiple source system which include loading nested JSON formatted data into snowflake table.
• Created Snowpipe for continuous data load and Redesigned the Views in snowflake to increase the performance.
• Migration of on-premises data (SQL Server / MongoDB) to Azure Data Lake Store (ADLS) using Azure Data Factory (ADF V1/V2).
• Exposures with Azure Active Directory compatibility. Extensive experience in deployment, migration, patching and troubleshooting of windows 2008 and 2012 R2 Domain Controllers in Active Directory.
• Experienced in performance tuning of Spark Applications for setting right Batch Interval time, correct level of Parallelism and memory tuning.
• To meet specific business requirements wrote UDF’s in Scala and Pyspark.
• Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity.
• Hands-on experience on developing SQL Scripts for automation purpose.
• Created Build and Release for multiple projects (modules) in production environment using Visual Studio Team Services (VSTS).
• Performing requirements gathering and analysis of various tools which are used for End to-End CICD pipeline, design, documentation, design reviews and development, testing and deployment of application enhancements.
• Implemented CI/CD process on Apache Airflow binary and DAG&s by building Airflow Docker images and uploading to JFrog antifactory.
• Implemented a workflow for multiple teams within the organization, which includes CI/CD tools, integrating with bamboo, build architecture, deployment using bamboo provided tools and plugins, inspecting the code from time to time, with SonarQube, Maven, MS Build and J Frog Artifactory in Linux/Windows environment for Java and .net based application.

Confidential

Azure Data Engineer

Responsibilities:

• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.
• Architect & implement medium to large scale BI solutions on Azure using Azure Data Platform services (Azure Data Lake, Data Factory, Data Lake Analytics, Stream Analytics, Azure SQL DW, HDInsight/Databricks, NoSQL DB).
• Design & implement migration strategies for traditional systems on Azure (Lift and shift/Azure Migrate, other third-party tools.
• Engage with business users to gather requirements, design visualizations and provide training to use self-service BI tools.
• Used various sources to pull data into Power BI such as SQL Server, Excel, Oracle, SQL Azure etc.
• Propose architectures considering cost/spend in Azure and develop recommendations to right-size data infrastructure.
• Develop conceptual solutions & create proof-of-concepts to demonstrate viability of solutions.
• Technically guide projects through to completion within target timeframes. Collaborate with application architects and DevOps.
• Identify and implement best practices, tools and standards.
• Design Setup maintain Administrator the Azure SQL Database, Azure Analysis Service, Azure SQL Data warehouse, Azure Data Factory, Azure SQL Data warehouse.

Confidential

Azure AD Expert

Responsibilities:

• Experience in using various Key Azure services (Blob, VM, Vnet, Subnet, Storage accounts).
• Configure VM IN optimum cost, Check Performance and Security.
• Backing up VMs and Potentially providing failover and Recovery.
• Experience in working with Azure active directory, AAD connect, RBAC Roles.
• Experience in Backup in Azure VMs.
• Experience in Managing and Monitoring Azure Kubernetes Cluster.
• Azure AD Connect and Sync Mechanism APP Registration.
• User account and Group Creation and management.
• SAML Configuration and issue troubleshoot.
• RBAC Permission, B2B Sync, Hybrid Join MFA Issue