SUMMARY

• Over 9.5 years of experience in Data Engineering, Data warehouse developer and worked for 4 years and Data Engineering and worked for 5.4 years.
• Data Pipeline Design, Development, and Implementation as a Sr. Data Engineer/Data Developer and Data Modeler.
• Expertise working with Azure cloud services like Azure Data Lake Storage, Azure Data Factory, Azure Analytical Services, Azure Blob Storage, Azure Analysis Services, and Azure Synapse.
• Hands - On experience in Spark Core, Spark SQL, Spark Streaming, and creating the Data Frames handle in SPARK with Scala.
• Working experience in developing applications involving Big Data technologies like Map Reduce, HDFS, Hive, Sqoop, Oozie, HBase, Pig, Spark, Scala, Kafka, and ETL(DataStage).
• Experience in Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks.
• Designing and Developed Oracle PL/SQL and Shell Scripts, performed Data Import/Export, Data Conversions, and Data Cleansing.
• Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification, and Testing as per Cycle in both Waterfall and Agile methodologies.
• Ability to work effectively in cross-functional team environments, excellent communication, and interpersonal skills.

TECHNICAL SKILLS

Big Data Technologies: Hortonworks, Cloudera.

Databases: Snowflake, Cassandra, Oracle, MySQL, SQL Server, MongoDB.

Programming Languages: PySpark, Shell script, Perl script, Python

Tools: Hive, HBase, Apache Spark, PyCharm, Eclipse, Visual Studio, SQL*Plus, SQL Developer, SQL Navigator, SQL Server Management Studio, Eclipse, Postman, SQOOP, Flume, Kafka, Yarn.

Cloud Services: ADF, Azure Data Lake Storage(ADLS), Azure Synapse Analytics (SQL Data warehouse), Azure SQL Database, Azure Cosmos NoSQL DB, Azure Key vaults, Azure DevOps, Big Data Technologies like Hadoop, Apache Spark and Azure Data bricks.

CI/CD Tools: Terraform, Jenkins

Version Control: SVN, GIT, GitHub.

Operating Systems: Windows 10/7/XP/2000/NT/98/95, UNIX, LINUX, OS

Visualization/ Reporting: PowerBI, Tableau

PROFESSIONAL EXPERIENCE

Confidential, Seattle

Azure Data Engineer

Responsibilities:

• Worked on creating tabular models on Azureanalytic services for meeting business reporting requirements.
• Data Ingestion to one or more cloud Azure Services-(Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and cloud migration processing the data in Azure Databricks.
• Creating pipelines, dataflows and complex data transformations and manipulations using ADF and PySparkwith Databricks.
• Used Data Flow debug for effectively building ADF data flow pipelines. Improved performance by usingoptimization options by effectively using partitionsduring various transformations.
• Created complex ETL Azure Data Factory pipelinesusing mapping data flows with multiple Input/outputtransformations
• Worked onAzure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics (DW).
• Workedwith Azure SQL Database Import and Export Service.
• Used Azure Key vault as central repository for maintaining secrets and referenced the secrets in Azure DataFactory and also in Databricks notebooks.
• Create and maintain optimal data pipeline architecture in cloudMicrosoft Azure using Data Factory andAzure Databricks.
• Exposed transformed datain Azure Spark Databricks platformto parquet formats for efficient data storage.
• Analyzed data where it lives by Mounting Azure Data Lake and Blob to Databricks.
• Implemented workflows to process around 400 messages per second push the message to documentDB aswell as event hubused APP insights for log mechanism.
• Extensively worked on Azure Data Lake Analytics with the help of Azure Data bricks to implement SCD-1,SCD-2 approaches.
• Built a common SFTP download or upload framework using Azure Data Factory and Databricks

Confidential - Houston, TX

Data Warehouse Developer

Responsibilities:

• Experience in developing complex store procedures, efficient triggers, required functions, creating indexes and indexed views for performance.
• Excellent Experience in monitoring SQL Server Performance tuning in SQL Server
• Expert in designing ETL data flows using SSIS, creating mappings/workflows to extract data from SQL Server and Data Migration
• Transformation from Access/Excel Sheets using SQL Server SSIS.
• Efficient in Dimensional Data Modeling for Data Mart design, identifying Facts and Dimensions, and developing, fact tables, dimension tables, using Slowly Changing Dimensions (SCD).
• Experience in Error and Event Handling: Precedence Constraints, Break Points, Check Points, Logging.
• Experienced in Building Cubes and Dimensions with different Architectures and Data Sources for Business Intelligence and writing MDX Scripting.
• Developing of Features, Structure, Attributes, Hierarchies, Star and Snow Flake Schemas of Data Marts.
• Good working knowledge on Developing SSAS Cubes, Aggregation, KPIs, Measures, Partitioning Cube, Data Mining Models and Deploying and Processing SSAS objects.
• Experience in creating Ad hoc reports and reports with complex formulas and to query the database for Business Intelligence.
• Expertise in developing Parameterized, Chart, Graph, Linked, Dashboard, Scorecards, Report on SSAS Cube using Drill-down, Drill-through and Cascading reports using SSRS.

Environment: MS SQL Server 2016, Visual Studio 2017/2019, SSIS, MS Access, Team Foundation server, Git.