Azure Data Engineer Resume
Alpharetta, GA
SUMMARY
- Over10+ years of experiencein Software Design, Development, Testing, Maintenance, Enhancement and Production/Application Support.
- Having 4+ years of relevant experience in designing and deploying solutions on Microsoft Azure Cloud platform using Azure Datafactory (ADF), Azure Data Lake store (ADLS), Azure Data Bricks (ADB), Azure Keyvaults, Azure Synapse Analytics (SQL Data warehouse) and integration solutions.
- Experienced in Developing prototypes using Azure services to test the business functionality and to seek feedback from business stakeholders.
- Design and Develop Azure data factory Ingestion pipelines to load the data from different source systems into ADLS.
- Developed Azure data bricks notebooks using Pyspark and Spark - SQL for data transformations and aggregations from multiple file formats to uncover insights into the customer usage patterns.
- Hands on experience on developing SQL scripts for automation purpose.
- Experienced in developing SQL Stored procedures to implement and simplify the complex business transformations in Azure Synapse Analytics (SQL Data warehouse).
- Experienced in using various Hadoop infrastructures such as MapReduce, Pig, Hive, Zookeeper, HBase, Sqoop, Oozie and Flume data storage and analysis.
- Developed custom alerts using Azure Data factory, SQLDB and Logic apps.
- Developed Databricks ETL Pipelines using notebooks, Spark Dataframes, SPARK SQL, and python scripting.
- Developed complex SQL queries using common table expressions (CTEs) to analyze the data.
- Experience working with various relational databases such as MS SQL server, Oracle, Teradata, and Sybase.
- Technical expertise in Microsoft technologies, such as .NET...ASP.NET, ASP.NET MVC 3.0/4.5, Web services, Web API, LINQ, SOAP, XML, HTML, Java Script, Visual Studio, AJAX, Ado.Net.
- Experience in Restful services is implemented using ASP.NET web API to support light weight data exchange in ASP.NET MVC.
- Experience in ADO.NET controls such as Grid View, Data List, Form view and Repeater.
- Extensive experience of .NET security features such as Authentication & Authorization, windows-based authentication, Forms-based authentication, Authorizing users and roles.
- Experienced in Configuration management, setting up version policies, build schedules using Team Foundation Server (TFS).
- Experience in UNIT testing (UIT), Integration testing and preparing test cases.
- Expertise in working with and supportingCA Technologies Workload Automation AEin with bothUNIXandWindowsenvironments.
- Experience in writingJob Information Language(JIL) scripts that comply with internal standards and CA best practices.
- Experience with tools likeSQL Server management studio and SQL Server 2005/2008 integration (SSIS) and Crystal Reports XI.
- Good experience in creating ASP.net web application UI.
- Excellent problem solving, decision-making and analytical skills. Willingness to learn and master the latest cutting-edge technologies.
- Involved in Agile SCRUM Methodology and extensively participated in Sprint Planning, daily SCRUM updates Demo and Retrospective meetings.
- Daily interactions with business and technical product owners to understand business requirements and convert them into technical user stories.
TECHNICAL SKILLS
Technologies stack: Azure Data Factory, Azure Datalake, Apache Spark2.4.0, Azure Logic Apps, Hive, Hue, Azure Databricks, MuleSoft
Databases: Oracle 11g/10g/9i, MS SQL Server 2008/2005/2000 , Teradata 15x, Azure SQL DB, Azure Synapse Analytics, Sybase
Languages: Python, ASP.NET, SQL, UNIX Shell Scripting, HTML, XML, C
Web Technologies: ADO.NET, MVC, AJAX, SOAP, Web Services, HTML, DHTML, Master Pages, CSS, jQuery, JavaScript
Scheduling: Autosys, Control-M, Smart Batch Server
Tools: Toad, SQL Developer, Erwin, Visio, JIRA, HPQC, Focal Point, WinSCP, Microsoft Azure Storage explorer, Jenkins, Bitbucket, GitHub, Git.
Deployment Tool: Azure DevOps, Team Foundation Server (TFS)
Operating systems: Windows, UNIX
PROFESSIONAL EXPERIENCE
Confidential, Alpharetta, GA
Azure Data Engineer
Responsibilities:
- Architect and Implement medium to large scale BI solutions on Azure using Azure Data platform services such as Azure Data Lake, Azure Data factory, Azure SQL Datawarehouse.
- Design and Develop Azure data factory Ingestion pipelines to load the data from different source systems into ADLS.
- Extracted data from various relational and non-relational sources using Azure Datafactory ingestion pipelines.
- Involved in design and development of metadata driven Azure Datafactory ingestion pipelines for optimal performance.
- Developed Databricks ETL Pipelines using notebooks, Spark Dataframes, SPARK SQL, and python scripting to apply business transformations.
- Have good experience working with Azure Blob and Data Lake storage and loading the data int SQL DWH.
- Developed External tables in Azure SQL DWH using polybase for reporting teams to consume the data.
- Converted the existing Azure Datafactory dataflows into Azure SQL DWH stored procedures for optimal performance.
Confidential, Alpharetta, GA
Data Engineer
Responsibilities:
- Design and Develop Azure data factory Ingestion pipelines to load the data from different source systems into ADLS.
- Performed data factory pipeline optimizations.
- Performed Data validation checks after trigger activity on every pipeline.
- Worked on migrating a very large scale on-prem applications to Cloud using Azure Cloud based services.
- Developed stored procedures in Azure SQL DB for business transformations.
- Supporting the existing Reporting Databases built on Teradata and MS SQL server Databases.
- Monitoring the business-critical applications in production and provide timely fixes in case of any failures.
- Develop complex MS SQL server procedures to provide data for Ad-hoc reporting.
- Worked with System Integration team and addressed SIT defects.
- Performed data reconsolidations between the existing and new systems as part of system modernization.
Confidential
Sr Software Engineer
Responsibilities:
- Architect and Implement medium to large scale BI solutions on Azure using Azure Data platform services such as Azure Data Lake, Azure Data factory, Azure SQL Datawarehouse.
- Design and Develop Azure data factory Ingestion pipelines to load the data from different source systems into ADLS (Azure Data Lake Storage).
- Extracted data from various relational and non-relational sources using Azure Datafactory ingestion pipelines.
- Involved in design and development of metadata driven Azure Datafactory ingestion pipelines for optimal performance.
- Have good experience working with Azure Blob and Data Lake storage and loading the data int SQL DWH.
- Developed web applications using ASP.NET and C#.
- Experience in designing the User Interface (UI) using HTML5.
- Used various data controls such as grid view, multi view and detail views for creating user controls.
- Writing and modifying the SQL Stored procedures and User defined functions to carryout database operations.
- Managed all development tasks and source code modifications via TFS.
- Responsible for Monitoring Production jobs of US Trust applications to ensure successful execution of batch and continued use of the application.
- Analyze production issues to determine root cause and provides fix recommendations to the development team
- Extensive use of Autosys batch monitoring tool and SQL server.
- Developed Job Information Language scripts to create jobs in Autosys, experience in using the best of Autosys resources, calendars, and other components supported by CA Autosys.
- Performed migration of jobs from lower-level environments to Production environments.
Confidential
Associate
Responsibilities:
- Designed, developed and trouble-shoot the application using advanced Microsoft technologies such as ASP.NET with MVC 3 Entity framework, C#.NET, CSS, jQuery, Java script
- Worked with ASP.NET web forms, State management, caching features, for optimization and configuration of the web application using C#.NET.
- Developed the data layer class using C# which is reusable for data transactions of entire project.
- Planned and performed migration of jobs from Smart Batch server to Autosys in Production and lower environments.
- Providing post deployment support for upgrades/changes/enhancements done on production (Live) Environment. - Changes/Upgrades in SQL SERVER 2005/2008
- Assisted with implementation of migration to Autosys, troubleshoot errors and documentation procedures.
- Monitoring Production and other lower-level environment Jobs to ensure successful execution.
- Finding issues, researching, and implementing the solutions.
- Performed daily Production Support recurring tasks and monitoring of the Jobs, resolve incidents, update root cause and solution.
- Written shell scripts for batch implementation and processing in Autosys & Smart Batch server.
- Developed various Crystal Reports XI per business requirements.
- Involved in complete SSIS life cycle in creating SSIS packages, building, deploying, and executing the packages in Development and Production environments.
- Developed Stored procedures, User defined functions and other database objects.
Confidential
Programmer Analyst
Responsibilities:
- Worked as .NET developer in the team for designing and developing the applications.
- Developed a fully object oriented, 3-tier web application using C# and ASP.NET.
- Developed web pages for various work streams using ASP.NET, AJAX, JavaScript, jQuery, CSS and connected them with business logic layer.
- Designed and implemented the web pages using ASP.NET and C#
- Involved in designing the database tables.
- Responsible for designing and developing the ASP.NET application authentication using .NET framework, MVC and security model.
- Understanding and analyzing the business requirements.
- Worked as a part of a team in design, develop, manage, and supports web-based application
- Developed various SSIS packages to extract, transform and load data coming from various sources such as flat file, excel file, web service and database.
- Developed, configured, and maintained jobs in SQL Server Agent that executes SSIS packages and stored procedures on daily/weekly/Monthly basis.
- Developed Crystal Reports and SSRS to generate SQL server reports.
- Used Team Foundation Server for version and source code control.
- Involved in code reviews and bug fixing.
- Performed Unit testing to identify and resolve functional issues.