We provide IT Staff Augmentation Services!

Senior Data Engineer Resume

5.00/5 (Submit Your Rating)

AtlantA

SUMMARY:

  • Over 8 years of experience in design and development of scalable Business Intelligence and database solutions using MS Azure and MS on - premises technologies with expertise in database and data warehouse.
  • Around 3 years of experience in Azure Cloud, Azure Data Factory, Azure Data Lake Storage Big Data Technologies (Apache Hadoop and Apache Spark), Data bricks, Cosmos DB and Azure Functions.
  • Expert in implementation of Azure Data solutions, ETL, data migration solutions using Azure Databricks, Azure Data Factory, Azure Data Lake, and SQL databases
  • Expertise in implementing Business Intelligence Solutions like ETL & Reporting using SSIS, SSAS, SSRS and T-SQL programming using DDL, DML, DCL commands for various business applications
  • Proficient in creating data pipelines using Azure Cloud ADFv2 components such as Move & Transform, copy, filter, for each, data bricks etc.
  • Experience in working with NoSQL databases and building apps on NoSQL databases like COSMOS
  • Well versed with Relational and Dimensional Modeling techniques like Star, Snowflake Schema, OLTP, OLAP, Normalization, Fact and Dimensional Tables
  • Experienced in performing various SSIS data transformation tasks like Lookups, Fuzzy Lookups, Conditional Splits and Event Handlers, Error Handlers etc.
  • Experienced in Performance Tuning in SSIS packages by using Row Transformations, Block and Unblock Transformations.
  • Experienced in conversion of packages from legacy systems and migrating, deploying the packages across production environment
  • Ability to work in large, complex development BI projects including the proactive identification of issues and coordination of resolutions.
  • Experienced in creating different visualizations using Bars, Lines and Pies, Maps, Scatter plots, Gantts, Bubbles, Histograms, Bullets, Heat maps and Highlight tables
  • Experienced in the migration of data from Excel, Flat file and XML to MS SQL Server by using conversions and DTS utility.
  • Experienced on Tableau activities, Multidimensional database and writing SQL queries
  • Created Tableau scorecards, dashboards using stack bars, bar graphs, scattered plots, geographical maps, Gantt charts using the functionality
  • Involved in creating dashboards and reports in Tableau and maintaining user and server activities
  • Defined best practices for Tableau report development and effectively used data blending feature in tableau
  • Have good understanding of Tableau architecture, design, development and end user experience

PROFESSIONAL EXPERIENCE:

SQL Server Tools : MS SQL Server 2014/2012/2008 R 2/2008/2005/2000/7.0, MSSQL Server Reporting Services (SSRS)

ETL Tools : Azure Data Factory (ADF), Azure Database migration Service (DMS), Azure Data Bricks, SSIS, Informatica

Reporting Tool : Power BI, SQL Server Reporting Services (SSRS), Tableau

Azure Cloud Platform : ADFv2, BLOB Storage, ADLS, Azure SQL DB, SQL server, Data bricks, Mapping Dataflow (ADF), Azure Cosmos DB, Logic Apps, Cosmos DB

Data Modeling Tools : Erwin 7.2/4.5/4.0, MS Visio 2003/2007, Rational Rose

Languages : T-SQL, VBA

Operating Systems: Windows Variants

PROFESSIONAL EXPERIENCE:

Senior Data Engineer

Confidential, Atlanta

Responsibilities:

  • Developed solutions in Databricks for Data Extraction, transformation and aggregation from multiple data sources
  • Designed and implemented highly performant data ingestion pipelines from multiple sources using Azure Data Factory and Azure Databricks.
  • Built SCD Type 2 Dimensions and Facts using Delta Lake and Databricks.
  • Built ETL solutions using Databricks by executing code in Notebooks against data in Data Lake and Delta Lake and loading data into Azure DW following the Bronze, Silver and Gold layer architecture.
  • Integrated the end-to-end data pipeline to take data from source systems to target data repositories.
  • Created complex ETL Azure Data Factory pipelines using mapping data flows with multiple Input/output transformations, Schema Modifier transformations, row modifier transformations using Scala Expressions.
  • Developed dynamic Data Factory pipelines using parameters and trigger them as desired using events like file availability on Blob Storage, based on schedule and via Logic Apps.
  • Developed Custom Email notification in Azure Data Factory pipelines using Logic Apps and standard notifications using Azure Monitor.
  • Wrote Dax using functions like Calculate, All, All Except, Filter, Divide, Count rows, Iterator functions like SumX, Maxx.
  • Developed JSON Scripts for deploying the Pipeline in Azure Data Factory (ADF) that process the data using the SQL Activity.
  • Performed operations like Data reconciliation, validation and error handling after Extracting data into SQL Server.
  • Worked on SSIS Package, DTS Import/Export for transferring data from Database (Oracle and Text format data) to SQL Server.
  • Generated and formatted Reports using Global Variables, Expressions and Functions for the reports.
  • Used various sources to pull data into Power BI such as SQL Server Used Power BI, Power Pivot to develop data analysis prototype, and used Power View and Power Map to visualize reports
  • Wrote calculated columns, measures query in Power BI desktop for data analysis techniques.
  • Published Power BI Reports in the required originations and Made Power BI Dashboards available in Web clients and mobile apps

Azure/SSIS Developer

Confidential - Tampa FL

Responsibilities:

  • Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.
  • Migration of on-premises data (Oracle/ SQL Server/ DB2/ MongoDB) to Azure Data Lake Store (ADLS) using Azure Data Factory (ADF V1/V2).
  • Designed and deployed many ETL workflows via Azure Data Factory (ADF) and SSIS packages to extract, transform and load data from SQL Server databases, excel and flat file sources into Data Warehouse.
  • Recreating existing application logic and functionality in the Azure Data Lake, Data Factory, SQL Database and SQL data warehouse environment.
  • Created Notebooks in Azure Data Bricks and integrated it with ADF to automate the same.
  • Used Azure Data Factory to orchestrate Databricks data preparation and load them into SQL Data warehouse.
  • Used various transformations like Filter, Aggregate, Conditional split, Derived column, Alter Row, Rank, Flatten, join etc. of Mapping data flows to transform and cleanse the data.
  • Responsible for ingesting data from Data Lake to Data Warehouse using Azure services such as Azure Data Factory, Azure Databricks.
  • Enabled CDC on the existing OLTP tables for continuous integration into data mart and created Power BI reports for stakeholders.
  • Involved in creation of database objects like tables, views, materialized views, procedures, and packages
  • Designed and developed Power BI graphical and visualization solutions with business requirement documents and plans for creating interactive dashboards.

Environment: SQL Server 2014, SSIS, SSRS, ETL, Agile, Business Intelligence Design Studio, MS Office 2016, Windows 10

SQL/SSIS Developer

Confidential - tampa fl

Responsibilities:

  • Design, develop, and maintain SQL standalone procedures, standalone functions, packages, triggers, standard views, materialized views, unique indexes, non-unique indexes, reference cursors, tables, and other SQL creations.
  • Involved in reviewing business requirements and analyzing data sources form Excel/Oracle SQL Server for design, development, testing, and production rollover of reporting and analysis projects within Tableau Desktop.
  • Created views in Tableau Desktop that were published to internal team for review and further data analysis and customization using filters and actions.
  • Created Heat Map showing current service subscribers by color that were broken into regions allowing business user to understand where we have most users vs. least users.
  • Design and develop Oracle SQL batch and real time processes for loading data from internal and external systems using SQL Loader.
  • Use Dynamic SQL and Execute Immediate functionality to perform DDL in SQL blocks and procedures.
  • Created dashboard using parameters, sets, groups and calculations. Created drill down and drill up in worksheets.
  • Involved in creating a dual-axis bar chart with multiple measures.
  • Created report schedules on Tableau server.
  • Blended data from multiple databases into one report by selecting primary keys from each database for data validation.
  • Combined views and reports into interactive dashboards in Tableau Desktop that were presented to Business Users
  • Reviewed basic SQL queries and edited inner, left, and right joins in Tableau Desktop by connecting live/dynamic and static datasets. Developed story telling dashboards in Tableau Desktop and published them on to Tableau Server, which allowed end users to understand the data on the fly with the usage of quick filters for on demand needed information.
  • Scheduled data refresh on Tableau Server for weekly and monthly increments based on business change to ensure that the views and dashboards were displaying the changed data accurately.
  • Participated in meetings, reviews, and user group discussions as well as communicating with stakeholders and business groups.

Environment: SQL Server 2014, SSIS, SSRS, ETL, Agile, Business Intelligence Design Studio, MS Office 2016, Windows 10

SQL Developer / Tableau/ Power BI

Confidential - New York, NY

Responsibilities:

  • Responsible for understanding and gathering business requirements and prepare interactive data visualizations in Tableau desktop and providing BI Solutions to the management for strategic decision making.
  • Responsible for extracting data in Tableau from Oracle database.
  • Experience working on Tableau Desktop - Connecting to multiple data sources, building quick views, advanced filtering, Sorting, Groups, Hierarchy, Create Sets, Actions, Table Calculations and using necessary Parameters, Forecast, Trend lines, Data Blending.
  • Analysis of sales for each group/department, over different dimensions like Region, squints, Organizations.
  • Creating dashboard that navigates to the Detail dashboard for specific drill down Analysis.
  • Plot Geographical maps like Filled Map and Symbol map.
  • Worked on detail level analysis of each product and transport to give top performing products and the correlation between the quantity and sales is designed using scatter plot.
  • Creating user filters at organization level.
  • Implemented action filters, parameters, calculated filed and set for preparing dashboards and worksheets in Tableau.
  • Created Request for Change (RFC) for moving database/ tech changes from Development to Testing and to Production during sprints.
  • Written different T-SQL scripts for automated execution of the tasks, Review T-SQL code to verify it meet standards for both performance and error handling
  • Created SQL server jobs on SQL Server Integration Services (SSIS) packages to automate the execution of the packages at a time depending on requirements.
  • Worked closely with both the offshore Application and Database teams to regularly monitor system for bottlenecks and implement appropriate solutions.
  • Worked on Logging, Error handling by using Event Handler, and Custom Logging for SSIS Packages.
  • Leveraging frameworks such as continuous integration and development to enable the rapid delivery of working code.
  • Created database maintenance planner for the performance of SQL Server, which covers Database integrity checks and re-indexing.
  • Proficient in creating solution driven views and dashboards by developing different charts types including

Environment: MS SQL Server 2008 R2, Reporting Services (SSRS), Integration Services (SSIS), Microsoft SharePoint Server, T-SQL, .NET

SQL Developer

Confidential

Responsibilities:

  • Create a Data Model while importing data, or from the Excel tables in the workbook.
  • Data tables in the Data Model can be viewed either in Data View or Diagram View.
  • Develop and implement data collection processes to improve and/or ensure data quality.
  • Filter and clean data collected from different sources.
  • Identify process improvement opportunities in the data analysis space.
  • Worked extensively with the users, different levels of management; conducted formal interviews and JAD sessions to identify requirements, functional specifications and database specifications.
  • Took participated in designing the dimensional model and 3NF model in Visio.
  • Designed and developed SSIS Packages to extract, cleanse, transform and load data from OLTP into data warehouse.
  • Used slowly changing dimension to monitor changes, add derived column to extract required information.
  • Developed, monitored, scheduled and executed SSIS package by using MS BIDS and SQL Server Agent including Scheduled Jobs, E-mail Notifications, Error Alerts and Error Logs as required; configured SQL mail agent for sending automatic emails with Success/Failure reports to assign recipients.

Environment: MS SQL Server 2012, Microsoft Visual Studio 2010 (SSIS, SSAS), Windows 7, MS Office 2010, MS Visio 2010, SharePoint 2010

SQL Developer

Confidential

Responsibilities:

  • Designing and creating SQL Server tables.
  • Assisting in SQL Server optimization including indexing and query optimization
  • Assisting in transferring databases, including Tempt to new servers using T-SQL scripts
  • Writing SQL Server stored procedures using TSQL
  • Writing, updating, and supporting reports using TSQL
  • Analyze end user database needs and provide efficient solutions
  • Editing XML
  • Collaborated with fellow team members through Team Foundation Server for effective project management which includes work issues and track on the update of versions.
  • Involved in creating data element maps between two distinct data models.

Environment: MS SQL Server 2008R2, OLTP, Business Intelligence Development Studio, Visual Studio 2010, SQL Server Profiler, SQL Server Management Studio, Excel, TFS, Windows XP.

We'd love your feedback!