SUMMARY

• 7+ years of experience as Data Engineer and Python Developer with expertise in Spark/Hadoop, Python/Scala and Cloud computing platforms AWS, GCP and Azure.
• Experience in Data Engineering, Data Pipeline Design, Development and Implementation as a Data Engineer and Data Modeler/Data Analyst.
• Experience in AWS, Azure DevOps, Continuous Integration, Continuous Deployment and Cloud Implementations.
• Extensive experience in Text Analytics, generating Data Visualization using Python creating dashboards using tools like Tableau
• Developed Consumer - based custom features and applications using Python, Django, HTML, CSS. Experienced with Software Development Life Cycle, Database designs, agile methodologies, coding, testing of enterprise applications and IDE's such as Jupiter Notebook, PyCharm, Emacs, Spyder and Visual Studio.
• Experience with Django and Flask high-level Python Web framework.
• Develop and maintain production level ETL pipelines using Python, Airflow and SQL.
• Experience building and maintain data analytics solutions over AWS, GCP and Azure
• Experience with Requests, NumPy, SciPy, Matplotlib, and Pandas python libraries during development lifecycle.
• Experience in maintaining data warehousing over BigQuery and Snowflake.
• Extensive Knowledge on developing Spark Streaming jobs by developing RDD’s (Resilient Distributed Datasets) using Scala, PySpark and Spark-Shell.
• Working with backend python automation, CI pipelines, Docker and cloud provisioning/automation.
• Proficient in managing entire data Engineering project life cycle and actively involved in all teh phases of project life cycle including data acquisition, data cleaning, Engineering, features scaling, features engineering, statisticalmodelling.
• Experience in writing Sub Queries, Stored Procedures, Triggers, Cursors, and Functions on SQL and PostgreSQL database.
• Experience in big data analysis and developing data models using Hive, PIG, and Map reduce, SQL with strong data architecting skills designing data-centric solutions.
• Experience working with data modeling tools like Erwin and ER/Studio.
• Experience in designing star schema, Snowflake schema for Data Warehouse.
• Experience in developing Map Reduce Programs using Apache Hadoop for analyzing teh big data as per teh requirement.
• Experience in designing star schema, Snowflake schema for Data Warehouse, ODS architecture.
• Hands on experience in cloud development architecture on Amazon AWS, EC2, Redshift and AMI and Basic on MS Azure.
• Experience in Data Analysis, Data Profiling, Data Integration, Migration, Data governance and Metadata Management, Master Data Management and Configuration Management.
• Experience with Jenkins and Enterprise GitHub. Worked on Jenkins continuous integration tool for deployment of project.
• Good Experience in Tableau Analytics. Build Dashboards for clients using Tableau.
• Experience in end to end design and deploy rich Graphic visualizations with Drill Down and Drop down menu option and Parameterized using Tableau.
• Worked on Loading and transforming large sets of structured, semi structured and unstructured data using Hadoop/Bigdata concepts.
• Proficient in SQL databases MSSQL Server, MySQL (RDBMS), Oracle DB, Postgres and MongoDB.

TECHNICAL SKILLS

Operating systems: UNIX, Linux, Windows

Databases: MySQL, PostgreSQL, Oracle, SQL Server, Cassandra, HBase

Mark-up Technologies: HTML5, CSS, NodeJS, Bootstrap, jQuery, SAX, Dom, XML

Cloud Platforms: AWS, Google Cloud Platform - Cloud Dataflow, Kubernetes,BigQuery and Microsoft Azure

Data Modeling Tools: Erwin Data Modeler, ER Studio

Big Data Ecosystems: Hadoop, Map Reduce, HDFS, Hive, sqoop, Spark, ZooKeeper, Flume, Yarn

Programming Languages: Python, PL/SQL, Java, JavaScript, HQL, Shell Scripting, Swift

SDLC: Waterfall, Agile, Scrum

Frameworks: Django, Flask, AngularJS, J2EE, Struts, MVC, Bootstrap

Development Tools: Team Foundation Server (TFS), Rational Application Developer (RAD)

Tools: Pytest, Selenium, PyCharm, Sublime, Eclipse, NetBeans, XCode, Jet Brains, IBM Infor sphere Data Stage

Reporting Tools: Tableau, PowerBI, SSRS

Python Libraries: OS, sys, threading, Pandas, NumPy, SciPy, Matplotlib

J2EE Technologies: JSP, Servlets, JDBC, Ajax, Web Services

Application Server: Tornado, WebLogic, Tomcat, SOAP

Web Servers: Web Logic, Web Sphere, Apache Tomcat, JBOSS.

Management Tech: SVN, Git, Jira, Maven.

PROFESSIONAL EXPERIENCE

Confidential, Alpharetta, GA

Data Engineer

Responsibilities:

• Responsible for gathering requirements, system analysis, design, development, testing and deployment.
• Designed Data Marts by following Star Schema and Snowflake Schema Methodology, using industry leading Data Modeling tools.
• Develop and maintain production level ETL pipelines using Python, Airflow and SQL.
• Worked on Snowflake database on queries and writing Stored Procedures for normalization.
• Worked with Snowflake’s stored procedures, used procedures with corresponding DDL statements, used JavaScript API to easily wrap and execute numerous SQL queries.
• Install and configure Apache Airflow for S3 bucket and Snowflake data warehouse and created dags to automate over Airflow/Composer.
• Designed and Developed ETL jobs to extract data from different sources and load it in data mart in Snowflake and managed Snowflake clusters such as launching teh cluster by specifying teh nodes and performing teh data analysis queries.
• Developed text mining toolkit for topic modelling and clustering on item reviews to identify and promote reliable products, using transformed phrases into a set of vectors in an n-dimensional space based on their TF-IDF scores.
• Developed classification models for market segmentation based on purchase history items and separation of potential customer based on defined characteristics for upselling opportunities on different products or campaigns.
• Worked on loading data from Linux file system to HDFS.
• Responsible to create organized responsive design and developed user interaction screens using React, Django, HTML5, CSS/CSS3, Bootstrap, JavaScript, jQuery.
• Developed Python and Django for teh backend development and front-end application using React, Webpack, Redux and MySQL for database.
• Responsible for setting up Python REST API framework using Django.
• Used React JS in components like JSX, creating React components, Virtual DOM, React Props, Lifecycle methods, working with React States and Events.
• Model View Component (MVC) architecture implemented for development of teh project and used Django Rest Framework to develop REST API.
• Used different python libraries like SciPy, NumPy, and Pandas to create models for predictive analysis.
• Experience with Unit testing/ Test driven Development (TDD), Load Testing.
• Experience in consuming web services (REST) with python programming language.
• Using REST Web services to integrate agile tool and XL release for SDLC in teh project.
• Worked on JSON and XML data of web application assets and its attributes.
• Participated in requirement gathering and worked closely with teh architect in designing and modelling.
• Rewrite existing Java application in Python module to deliver certain format of data.
• Used Ansible to configure and manage teh infrastructure. And worked on Jenkins CloudBees for CI/CD in production environment.
• Automation (Ansible) and deploying on AWS environment (EC2).
• Creating Amazon EC2 Cloud Instances using Amazon Web Services and configuration of launched instances with respect to specific applications.

Environment: Python, Django, React, HTML/CSS, Java, MySQL, REST Web Services, JavaScript, Jupiter, Anaconda, Docker, XL release, PyPI, NumPy, Pandas, JSON, PyCharm, Linux, Shell Scripting, TDD, Ansible, GitHub, Tableau, VersionOne (ALM), GIT, AWS,Snowflake, SQL Server 2000/2005, Reporting Services (SSRS)

Confidential, Mt. Laurel, NJ

Data Engineer

Responsibilities:

• Involved in analysis, specification, design, and implementation and testing phases of Software Development Life Cycle (SDLC) and used Agile methodology for developing application.
• Upgraded existing UI with HTML, CSS, jQuery and Bootstrap.
• Working as an application developer experienced with controllers, views and models in Django.
• Used Ansible to configure and manage teh infrastructure.
• Used AWS Cloud Watch to monitor and store logging information.
• Worked on Docker CE and curl.
• Implemented teh application using Python Spring IOC (Inversion of Control), Django Framework and handled teh security using Python Spring Security.
• Responsible for handling teh integration of database systems.
• Created Tables, Stored procedures and defined functions. Created SQL scripts for tuning and scheduling.
• Developed physical data models, data warehouse models and created DDL scripts to create database schema and database objects
• Used object/relational mapping (ORM) solution, technique of mapping data representation from MVC model to Oracle Relational data model with an SQL-based schema
• Implemented Performance tuning and improved teh Performance of Stored Procedures and Queries.
• Performed database operations and queries using MySQL.
• Worked on Jenkins for continuous integration to build code and deploy into remote servers.
• Extensively worked on Jenkins by configuring and maintaining for continuous integration and for end to end automation for all build and deployments.
• Deployed project into Amazon web services (AWS) using amazon elastic bean stalk.
• Created Realtime data pipelines and frameworks with kafka, spark streaming and loading data to Hbase.
• Worked on Apache spark for processing multiple datasets.
• Responsible for third party tool integration
• Wrote and executed automated test cases with selenium to automate manual testing solutions.
• Responsible for debugging and troubleshooting teh web application.
• Wrote unit testing codes using unit test, resolving bugs and other defects.
• Used JIRA to assign, track teh stories and tasks.
• Worked on GitHub enterprise.

Environment: Python, Django, Angular JS, HTML, CSS, JavaScript, jQuery, Sublime Text, Jira, GIT, PyBuilder, Kafka, Web Services, AWS, S3, Spark, Selenium, RedHat Linux,YARN.

Confidential, Chicago, IL

ETL Developer

Responsibilities:

• Experience in Importing and exporting data into HDFS and Hive using sqoop.
• Develop Data Stage jobs to extract, transfer and load teh data to populate different dimension tables, fact tables in teh daily and monthly marts.
• Build and maintain data warehousing solutions over GCP BigQuery.
• Implemented teh Secondary sorting to sort reducer output globally in map reduce.
• Designed and implemented an application for managing customers, purchases and invoices.
• Implemented data pipeline by chaining multiple mappers by using Chained Mapper.
• Developed Map-Reduce programs to clean and aggregate teh data
• Implement counters on Hbase data to count total records on different tables.
• Created Hive Dynamic partitions to load time series data.
• Handling continuous streaming data comes from different sources using flume and set destination as HDFS.
• Experience with CDH distribution and Cloudera Manager to manage and monitor Hadoop clusters.
• Write multiple SQL Queries for analytics over BigQuery GCP.
• Re-engineered various modules for implementing changes and created efficient system.
• Maintained program libraries, user's manuals, and technical documentation.
• Created Azure Run book to Scale up and Scale down Azure Analysis Services and Azure SQL Data warehouse.

Environment: Hadoop, HDFS, Map Reduce, Hive, Pig, Hbase, Sqoop, BigQuery, RDBMS/DB, Flat files, MysqlCSV, flume, GCP, Shell, JIRA, GIT,Azure.

Confidential

ETL Developer

Responsibilities:

• Involved in understanding teh scope of application, existing schema, data model and defining relationship within and between teh groups of data.
• Designed and developed ADHOC scripts for product pattern analysis: Performed data mining and chronological setup to correlate products to observe related trends and patterns and detect relations between categories and products using association mining.
• Designed Mappings between sources to operational staging targets, using Star Schema, implemented logic to Slowly Changing Dimensions.
• Analysed teh requirements and arranged teh business logic for ETL process.
• Interaction with teh business users to better understand teh requirements and document their expectations, handling teh current process, modifying and created teh jobs to teh updated requirements, handle teh load process to data mart and eventually data warehouse.
• Applied teh business logic to load teh data into Data warehouse.
• Used existing ETL standards to develop these mappings.
• Used report Manager for setting up roles and security on teh reports.
• Involved in Management and migration of SSIS packages in teh staging and pre-prod environments.

Environment: Sql server 2008R2, BIDS 2008 (SSIS,SSAS and SSRS),Agile, Jira.