SUMMARY

• Overall 8+years of strong experience in Building end to end pipelines using Pyspark, Python, Azure services & in depth understanding of Distributed Systems Architecture and Parallel Processing Frameworks. experience in various Deep Learning frameworks and its implementation on various HW architectures and a good system level knowledge of RISC, VLIW, GPU & VPU hardware platforms and various operating systems such as Linux, Windows, QNX, RTEMS.
• Skilled with Object - Oriented design and programming with Python.
• Evaluated and implemented enterprise level metadata application with collibra in implementation for metadata application.
• Designed schemas with dimensional data and normalizing the dimensional tables and creating a Sub Dimension named Demographic as a subset to the Customer Dimension.
• Architecting the infrastructure on Google cloud platform using GCP services and automating GCP infrastructure using GCP Cloud Deployment manager.
• Engineered with various packages consists of NumPy, SQL Alchemy, matplotlib, beautiful soup, PySide, SciPy, PyTables.
• Experience in Data Governance practice and tools like Collibra, Rochade
• Worked with different development environments like IDE, text editor like sublime tools like IDLE, PyCharm, Eclipse, Net Beans, Py studio.
• Conceptualized EDM and DG principles strategizing data driven organization with an efficient way in contemplating data management environments.
• Passionate, with a proven track record in bringing clarity of Vision, Strategy and trusted Execution, consistently enabling organizations to harness and leverage Data Assets to drive operational efficiency and superior customer solutions.
• Data science evangelist with deep understanding of mathematical models and expertise in leveraging them for strategy, profitable growth, and personalization.
• Successful history of serving multiple business functions and driving measurable business results working at the intersection of Product, Marketing and Technology.
• Balanced skill sets across Technical Leadership ( UX, A/B testing, ML/Cloud Ops), Project/Program (Metrics, Schedule, Features) and Product Management (Agile, Scrum).
• Deeply committed to clear and impactful communications across noisy and vast organizational settings to get the story across
• Monitored projects in ordered to debug using JIRA (Agile).
• Optimized Model View Control (MVC) in implementing server-side architecture using Flask and Django for web applications.
• Developed Modern Data Warehouse solutions using Databricks and Azure Stack(ADF,ADLS)
• Optimizing the mappings by creating re-usable transformations and Mapplets. Debugging andperformance tuning of sources, targets, mappings, transformations, and sessions.
• Experienced in machine learning, decision science, statistics, testing methodologies, Software development.
• Established strong pipelines of end-to-end data based on Hadoop platform.
• Adapted proficient approach in data analysis and visualization by PyCharm using python libraries.
• Collaborate with DW/BI leads to understand new ETL pipeline development requirements.

TECHNICAL SKILLS

Programming Language: Python, Scala, R, Java, SAS

Technologies & Framework: Scikit-learn, Keras, TensorFlow, NumPy, Pandas, Matplotlib, Seaborn, ggplot, AWS, Hadoop, Apache Spark, Sage Maker, Docker

Machine Learning: Linear Regression, Logistic Regression, KNN, SVM, PCA, Decision Tree, Random- Forest, K-means clustering, Gradient Boosting, Ensemble Learning Algorithms

Database: SQL, My SQL, HDFS, HBase, PostgreSQL, MongoDB, Hive, Impala, Pig, Spark SQL, Cassandra, AWS Athena

Software and Tool: Tableau 10, Power BI, Jupyter Notebook, IntelliJ, Databricks, Jira, MS Office

PROFESSIONAL EXPERIENCE

Senior Data Engineer

Confidential, Malvern, PA

Responsibilities:

• Built complex Talend ETL work to migrate large data from flat files to database. Extracted files from mainframe into Talend execution 1 n server using multiple ftp components.
• Audited business problems into Big Data solutions in defining Big Data strategy and Roadmap. Installing, configuring, and maintaining Data Pipelines.
• Used Pyspark Spark-Streaming APIs to perform necessary transformations and actions on the fly for building the common learner data model which gets the data from Kafka in near real time and persists into Cassandra
• Working as an Azure Data Engineer where I create complex Data-Pipelines in Azure Data Factory, to process huge volumes of data for a regulatory reporting in this project.
• Skilled in creating end-to-end data engineering solutions on Microsoft Azure, which leverages some of the most important and popular Azure Data Services like Azure Synapse Analytics, Azure Data Lake Storage Gen2, Azure Databricks, etc.
• Created data pipelines to ingest, process and load data using ADF into Azure Synapse, which is Azure's enterprise data-warehousing service.
• Microsoft Security Assessment analysis for on-premises, Azure, and M365 (Exchange, SharePoint, OneDrive, Teams, Azure AD), via Microsoft and QS CSAT tools.
• Supported SharePoint, OneDrive, EOP, Security/Compliance, Azure AD, Azure AD Connect, OAuth, SAML, ADFS, ATP, AIP, Intune, Teams.
• Provide infrastructure and security consulting services to assist with the strategy, planning, design, and deployment of security configuration and solutions required to migrate data of on-premise SharePoint sites to SPO as well as enable OneDrive for Business for managed assets.
• Analyzed and Directed On-premises SharePoint and OneDrive migration to SharePoint online.
• Expert understanding of software engineering concepts and methodologies.
• Experience in designing, developing, and deploying enterprise directories for large organizations with multiple regions, locations, sites.
• Deep knowledge of core Active Directory environment and related components such as DNS, DHCP, PKI, Domain Controllers, AD Objects Administration, Group Policy, Sites/Subnet design and deployment, replication topology, disaster recovery.
• Deep knowledge enforcing security standards, compliance requirements, security procedures, and controls to ensure the integrity of AD and related systems.
• Working knowledge in PowerShell scripting.
• Experience with Active Directory Federation Services (ADFS), SAML, web Single Sign-on (SSO), OAuth, and related.
• Working knowledge of Azure AD Connect and Azure AD DS configurations.
• Experience leveraging or developing solutions using Access Security-related concepts: Identity, Governance and Administration, Authentication and Authorization, Identity Repositories such as Active Directory (AD) services.
• Extensive experience managing and delivering in the designing and implementation of enterprise-level Active Directory (AD) environments with a high number of domain controllers, and multiple forests and domains.
• Designing, implementing, and participating in the incident response processes specific to Azure AD and O365 deployments.
• Design Azure network infrastructure such as VNETs, subnets, Network Security Groups, route tables, ACLs) Azure AD, DNS, and domain management.
• Manage Active Directory (AD), User Permissions and Group Policies in accordance with security policies and procedures.
• Provide security architecture support for build, migration, and operation of Azure cloud infrastructure
• Used IDE tool to develop the application and JIRA for bug and issue tracking and Used JIRA to assign, track, report and audit the issues.

Environment: Python 3.4/2.7, Django 1.7, CherryPy, Golang, HTML5, CSS, Bootstrap, JSON, JavaScript, AJAX, RESTful web service, MongoDB, MySQL, jQuery SQLite, Elastic search.

Sr Big Data Engineer

Confidential, Chicago, IL

Responsibilities:

• Developed APIs for restful functionalities to enhance the application.
• Established data lake in Google Cloud using Google Cloud Storage, Big Query, and Big Table.
• Built shell scripts for raw data processing to cloud server like AWS S3 and redshift databases.
• Executed Python based GUI components for the front-end functionality such as selection criteria and developed a GUI for a Java script.
• Modeled UDFs in Hadoop Pyspark to perform transformations and loads.
• Deployed regression SQL to merge the validated Data into Prod environment.
• Source Analysis, tracing back the sources of the data and finding its roots though Teradata, DB2 etc
• Designed full CI/CD pipelines so that each commit will go through the standard process of the software lifecycle and gets tested well enough before it can make it to the production.
• Developed PowerShell scripts to configure the upgraded Exchange environment, migrate users, then monitorand maintain the new environment.
• Incorporated AGILE methodologies in managing life cycle developments of process.
• Worked on various data formats like AVRO, Sequence File, JSON, Map File, Parquet and ORC
• Developed HIVE environments for the refinement of ETL frameworks for load data from multiple sources.
• Automated critical process such as granting network permissions to GCP service accounts, granting Image user access to Compute Engine’s Service accounts.
• In addition, participating in requirement gathering, analyzing the requirements, creating the technical design specifications based on client functional specifications.
• Designing and developing complex mappings to migrate data from multiple sources into a common target area such as Data Marts and Data Warehouse using lookups, Source Qualifier, Router, Filter, Expression, Aggregator, Joiner, Normalizer, Sequence Generator, Update Strategy from varied transformation logic's in Informatica Power-center.
• Data Extraction from various source systems ranging from XML, Flat files to RDBMS like Oracle SQL Server, Teradata.
• Optimizing the mappings by creating re-usable transformations and Mapplets. Debugging and performance tuning of sources, targets, mappings, transformations, and sessions.
• Automating the Tasks using UNIX Scripts, Job scheduling and Mainframes CA7 Scheduler.
• Data flow creation using MS Visio and Dashboard creation using Excel.
• Involved in all phases of SDLC from requirement gathering, design, development, testing, user training, and support for the production environment.
• Tech Stack: Informatica Power Centre, Oracle, MS SQL Server
• Project Details: The project involves extracting data from various source systems like Flat file, XML, CSV file, and Oracle. Stored it in a staging layer and cleanse, compute the data as per requirement and generate the output in an XML file or JSON format. The XML, JSON is further transformed for UI reporting.
• Developed internal auxiliary web apps using Python Flask framework with Angular.js and Bootstrap CSS HTML framework.
• Developed LSTM Recurrent Neural Network for Time series prediction on top of Google Cloud Platform.
• Performed Principal Component Analysis to visualize firm based on their 10-k text content in 2-D space to evaluate the relation.
• Implemented MVC architecture in developing the web application with the help of Collibra framework.
• Responsible for Design, Deploy and Maintain Machine Learning models in production systems.

Environment: Hadoop Yarn, Spark-Core, Spark-Streaming, Spark-SQL, Scala, Python, Kafka, Hive, Sqoop, Amazon AWS, Elastic Search, Impala, Collibra, Tableau, Talend, Cloudera, MySQL, Linux, Shell scripting.

Big Data Engineer

Confidential, Greenwood Village, CO

Responsibilities:

• Built scalable distributed data solutions using Hadoop
• Optimized algorithms in Hadoop using PySpark Session, Spark-SQL, Data Frames and Pair RDDs.
• Implemented monitoring and established best practices around using Elastic search and used AWS Lambda to run code without managing servers.
• Developed dynamic web pages to view the reports generated utilizing Django Framework.
• Involved in using Terraform migrate legacy and monolithic systems to Amazon Web Services.
• Server management using by using Linux command in Troubleshoots.
• Worked on AWS services running in an environment for sensitive workloads and storing sensitive data.
• Used Terraform scripts to Automate Instances for Manual Instances that were launched before.
• Wrote and executed various MYSQL database queries from Python using Python-My SQL connector and My SQL db package.
• Designed the data models to be used in data intensive AWS Lambda applications which are aimed to do complex analysis creating analytical reports for end-to-end traceability, lineage, and definition of Key Business elements from Aurora.
• Developed PySpark programs and created the data frames and worked on transformations.
• Developed entire frontend and backend modules using Python on Django Web Framework and Utilize PyUnit, the Python unit test framework, for all Python applications. Designed the front end of the application using Python, HTML5, CSS3, AJAX, JSON and jQuery.
• Resolved ongoing problems and accurately documented progress of a project.
• Performed efficient code based on principles of Test-Driven Development (TDD) and continuous integration to keep in line with Agile Software Methodology principles.
• Improved MapReduce/Spark modules for Machine learning & predictive analytics using Hadoop on AWS.
• Implemented migration tasks from Oracle apps to PostgreSQL databases.
• Responsible for writing/modifying scripts using sh, ksh, and Bash for day-to-day administration.
• Developed entire frontend and backend modules using Python on Django including Tastypie Web Framework using Git.
• Responsible for gathering requirements, system analysis, design, development, testing and deployment.
• Participated in the complete SDLC process.
• Developed monitoring and notification tools using Python.
• Utilize PyUnit, the Python unit test framework, for all Python applications.
• Rewrite existing Java application in Python module to deliver certain format of data.
• Created Business Logic using Python to create Planning and Tracking functions.
• Generated Python Django Forms to record data of online users.
• Wrote Python routines to log into the websites and fetch data for selected options.
• Developed entire frontend and backend modules using Python on Django Web Framework.
• Developed Business Logic using Python on Django Web Framework.
• Developed views and templates with Python and Django's view controller and templating language to create a user- friendly website interface. Used PyQuery for selecting particular DOM elements when parsing HTML.
• Created Data tables utilizing PyQt to display User information and add, delete, update User records.
• Incorporated Jenkins for integration services.
• Used the Django Framework to develop the application.
• Build database mapping classes using Django models.
• Involved in AJAX driven application by invoking web services/API and parsing the JSON response.
• Developed internal auxiliary web apps using Python Flask framework with Angular.js and Bootstrap CSS / HTML framework.

Environment: Hadoop Yarn, Spark-Core, Spark-Streaming, Spark-SQL, Scala, Python, Kafka, Hive, Sqoop, Amazon AWS, Elastic Search, Impala, Cassandra, Tableau, Talend, Cloudera, MySQL, Linux, Shell scripting