Data Engineer Resume
Florham Park, NJ
SUMMARY:
- A Data Engineer/MSBI Developer with brand new skills, an insatiable intellectual curiosity, and the ability to mine hidden gems located within large sets of structured, semi - structured and unstructured data. Able to leverage a heavy dose of mathematics, programmatic and applied statistics with visualization and a healthy sense of exploration. Master’s in Computer Science and work domain experience include Health care, Financial, Retail sectors.
- Around 8 years of experience in Analysis, Architecture, Design, Development, Testing, Maintenance and User training of software application which includes support of various database technologies (SQL Server 2008/2012/2016 Azure Big Data, NoSQL, RDBMS and HDFS) environment.
- Excellent experience in Data warehousing projects with extensive usage of Azure Services, ETL Reporting & Analysis services like SQL Server, SSIS, SSRS, SSAS, Power BI Desktop and Power BI Cloud.
- Experience in Building cloud solutions with the Microsoft Azure HDInsight, Azure Data Lake Factory and Data Bricks in various projects.
- Worked with Azure Data Lake Store to capture data of any size, type, and ingestion speed in one single place for different operations.
- Having 4 years of experience in Microsoft Power BI Desktop, Power BI Administration and DAX. Generated various Power BI reports using Python Report lab and sent to Business users to improve their decision making.
- Extensive testing ETL experience using Informatica 9.1/8.6.1/8.58.1/7.1/6.2/5.1 (Power Center/ Power Mart) (Designer, Workflow Manager, Workflow Monitor and Server Manager) Teradata and Business Objects.
- Experience in using Automation Scheduling tools like Autosys and Control-M
- Experience in building Data Lakes. Build how the data will be received, validated, transformed and then published
- Created SSIS packages to Extract, Transform and load (ETL) data from Excel, Database, XML File and Flat file source by using different SSIS transformations such as Lookup, Derived Columns, Condition Split, Aggregate, Pivot Transformation and Slowly Changing Dimension, Merge Join and Union all
- Excellent hands-on experience with data modelling tool like Erwin and strong knowledge of Relational and Dimensional database modelling concepts with star schema and snowflake schema’s design and implementation
- Extensively worked SSAS Tabular Model Cubes and created dashboard reports using Power BI Desktop and worked on Static & Dynamic Row Level Security mechanisms in Power BI.
- Effectively Plan and Manage projects deliverable with on-site and offshore models and improve the client satisfaction. Experience in attending requirement review meetings and giving feedback to BA and Manager.
- Extensive hands-on experience in executing Automation Testing tasks - automation testing requirements, prepared automation test scripts in Load Runner, running tests, monitoring/ analyzing results, collecting test metrics and conducting test reporting.
- Strong analytical, interpersonal, communication, coordination, problem solving and decision-making skills.
PROFESSIONAL EXPERIENCE:
Confidential, Florham Park/NJ
Data Engineer
Responsibilities:
- Design and implement end-to-end data solutions (storage, integration, processing, visualization) in Azure and Databricks.
- Propose architectures considering cost/spend in Azure and develop recommendations to right-size data infrastructure
- Working closely with the Risk Decision Science team to operate data science, data analytics, data warehouse, and BI solutions and Interacts with Business Analysts, Users, and SMEs on requirements
- Recreating existing application logic and functionality in the Azure Data Lake, Data Factory, Databricks, SQL Database and SQL Data warehouse environment.
- Worked on Azure Data Lakes (ADLS) and Data Lake Analytics and an understanding of how to integrate with other Azure Services.
- Created User Defined Functions (UDF’s) to encapsulate frequently and commonly used business logic making the system more modular, secured, and extensible.
- Migrate data from traditional database systems to Azure bigdata platforms and optimizing Stored Procedures and long running queries using indexing strategies and query-optimization techniques.
- Migration of on premise data (Oracle/ NetSuite) to Azure Data Lake Store(ADLS) using Azure Data Factory(ADF V1/V2).
- Creating Databricks notebooks using SQL, Python, Scala and automated notebooks using jobs.
- Publishing the Power BI Desktop models to Power Bi Service to create highly informative dashboards, collaborate using workspaces, apps, and to get quick insights about datasets
- Work closely across teams (Support, Solution Architecture) and peers to establish and follow best practices while solving customer problems
Environment: Azure Data Factory, Databricks Azure Data Lake, Databricks, NetSuite, Power BI, SQL Server Data tools (SSDT), Python, power Shell, SQL profiler 2016.
Confidential
BI Engineer/Power BI Developer
Responsibilities:
- Successfully developed front end web base dashboard with trending data, animated pipeline charts, and demographic charts with drill downs functions that displays a breakdown of Regional/States sales, utilizing data aggregation from SQL statements.
- Developed Python based API (RESTful Web Service) to track sales and perform sales analysis using Flask, SQL Alchemy and PostgreSQL.
- Processed raw data at scale including writing scripts, web scraping, calling APIs, write SQL queries, etc
- Performed visualization using SQL integrated with Zeppelin on different input data and created rich dashboards
- Used Power BI Power Pivot to develop data analysis prototype and used Power View and Power Map to visualize reports and created workspace and content packs for business users to view the reports.
- Created reports from complex SQL queries and MDX queries. •
- Created Azure Blob Storage for Import/Export data to/from .CSV File.
- Used Power BI, Power Pivot to develop data analysis prototype, and used Power View and Power Map to visualize reports and Expertise in writing complex DAX functions in POWER BI and POWER PIVOT.
- Statistics, algorithms, data structures, relational databases, SQL programming (MySQL, Postgre SQL, Oracle.
- Used various sources to pull data into Power BI such as SQL Server, SAP BW, Oracle, SQL Azure etc.
- Scheduled Automatic refresh and scheduling refresh in power bi service.
- Wrote calculated columns, Measures query is in power bi desktop to show good data analysis techniques
- Implemented security measures at Power BI Service by implementing authentication and authorization methods
- Created Data warehouse Cubes in SQL Server Analysis Service (SSAS).
Environment: MS SQL Server 2017, Power BI, Azure Data lake, Data Bricks, Toad Data load, Teradata SQL Assistant, Erwin Data Modeler 7.2, SQL Server Data Tools (SSDT), Rapid SQL (2016), Azure SQL Database, Visual Studio 2018, Python script
Confidential, NJ
SQL MSBI/Power BI Developer
Responsibilities:
- Visualized data by creating Charts and Graphs (bar graphs, line charts, pie charts, Tree maps, Bubble Charts, Waterfall Charts, Bump Charts) based on client's need.
- Responsible in consolidating and enhancing the existing dashboards and pdf's using Power BI desktop and published reports to Power BI APP.
- Blended data from multiple databases into one report by selecting primary keys from each database for data validation.
- Used Power BI Power Pivot to develop data analysis prototype and used Power View and Power Map to visualize reports and created workspace and content packs for business users to view the reports Created reports from complex SQL queries and MDX queries. involved in Migration of SAP BO reports to Interactive Power BI Dashboards
- Develop ETL jobs with heavy transformations for data acquisition from SAP ECC and non-SAP systems.
- Collaborate with application architects and DevOps
- Worked on Advanced SQL to embed the Stored Procedures into ETL PySpark scripts.
- Implemented cell level security in cubes using MDX expressions to restrict users of one region seeing data of another region using SSAS
- Created calculated Measures for dashboard KPI's using DAX and Direct Query, build custom attributes for dashboard slicers and other filters using DAX.
- Created visualization dashboards for online reports helping clients identify opportunities
- Created calculated Measures for dashboard KPI's using DAX and Direct Query, build custom attributes for dashboard slicers and other filters using DAX.
- Deployed and scheduled the reports in the Power BI Service.
Environment: MS SQL Server 2016, Power BI, Toad Data load, Teradata SQL Assistant, SAP BO, T-SQL, SQL Profiler, Rapid SQL (2016), Azure, Visual Studio 2018, Team Foundation Server, SQL.
Confidential
SQL MSBI(SSIS/SSRS/SSAS) Developer
Responsibilities:
- Worked on Full life cycle development (SDLC) involving in all stages of development.
- Involved in system study, analyze the requirements by meeting the client and designing the complete system.
- Responsible for developing, support and maintenance for the ETL (Extract, Transform and Load) processes using Informatica Power Center 8.5.
- Implemented complex business rules in Informatica Power Center by creating re-usable transformations, and robust Mapplets.
- Created Database Objects - Schemas, Tables, Indexes, Views, User defined functions, Cursors, Triggers, Stored Procedure, Constraints and Roles.
- Used ETL to implement Slowly Changing Dimension to maintain historical data in Data Warehouse. Experience in Logical modelling using the Dimensional Modelling techniques such as Star Schema and Snowflake Schema and business on requirements gathering for Simple Finance S4 HANA project.
- Review existing code, lead efforts to tweak and tune the performance of existing Informatica processes
- Designed Dimensional Modelling using SSAS packages for End-User. Created Hierarchies in Dimensional Modelling.
- Involved in creating reports in Tableau and Maintaining server activities, user activity, and customized views on Server Analysis.
Environment: MS SQL Server 2016, T-SQL, SSIS, SSRS, SQL Profiler, Informatica power centre 8.5, Rapid SQL (2016), Visual Studio (2015), Tableau 8,10.3, Team Foundation Server (2015), C#, .Net.
Confidential
SQL BI/QlikView Developer
Responsibilities:
- Designed, developed, implemented, and supported QlikView dashboards. Integrated data sources and databases with QlikView and designed and developed data models and backend queries for presenting data.
- Designed transformation rules and processes to derive the correct data from the extracted data and transform into the required format and structure to support the business requirements.
- Loading the transformed data into the target database and the associated metadata into the enterprise metadata repository
- Designed various schemas which are used for Landing the data, staging for transforming the data in MS SQL SERVER 2012.
- Extensively worked with Team Foundation Server (TFS) 2012.
- Experience in creating complex SSIS packages using proper control and data flow elements.
Environment: MS SQL Server2012/2008R2/2008, SQL Server Management Studio (SSMS), (SSIS), (SSRS), (SSAS), T- SQL, SQL Profiler, MS Office, MS Excel, Team Foundation Server (2012)
Confidential
SQL developer
Responsibilities:
- Worked with various upstream and downstream customers in interfacing various systems and processes for Data extractions, ETL, Analytics and reporting needs.
- Created database triggers to implement business requirements, created complex Stored Procedures and Functions to support the front-end application
- Created and managed schema objects such as tables, views, indexes, procedures, triggers and maintaining Referential Integrity.
- Developed SQL scripts to Insert/Update and Delete data in MS SQL database tables.
- Developed and created data dictionary, advanced queries, views, indexes, and functions for databases.
Environment: Windows 2003 Server, MS SQL Server 2000, MS SQL Server 2005, SSRS, DB2, Oracle 9i, ASP, ODBC, VBScript, Windows 2000/XP, IIS 5.
TECHNICAL SKILLS:
Hadoop/Big Data Technologies: Azure Data Bricks, Azure Data lake and Azure Data Factory
Databases: NoSql, MS SQL Server 2016/ 2012, MS Access, Azure Data bricks, Oracle 8i/9i, Oracle 10g/9i DB2, Postgres
Programming Languages and Scripting: Java, Python, Scala, sql- postgre, sql server, oracle
Operating Systems: Windows, Linux, Mac OS X
Reporting/Analysis Tools: Power BI, SSRS (2016,2012), Qlikview, Tableau 10.3.
Web technologies: JSP, Servlets, JDBC, Java Script, CSSCI/CD Jenkins, Docker
Developing tools/BI/ETL: TOAD Visio, SSIS, DTS, Data factory, informatica power centre 8.6.1
Import Export Data, SQL Analyzer, Management studio, SQL Server 2012, Query Editor, SAP S4, HANA, Informatica 9.1.:
Data Modelling: Erwin, Visual Studio
Development Methodologies: Agile Methodology -SCRUM, Hybrid.