SUMMARY

• Over 5 years of IT experience in software analysis, design, development and implementation of Big Data, Hadoop and Data Science Technologies
• Hands on experience with NoSQL Databases like HBase, Cassandra and relational databases like Oracle and MySQL.
• Experience working with MPP Databases like GreenPlum (Pivotal)
• Create reports for the BI team using Sqoop to export data into HDFS and Hive.
• Hands - on programming experience in various technologies like Python, R, SCALA
• Hands on experience in developing Spark applications using Spark tools like RDDtransformations, Spark core, Spark streaming and SparkSQL.
• Strong understanding and strongknowledge in NoSQL databases like HBase, MongoDB&Cassandra.
• Expert knowledge of data structures and algorithms.
• Extensive experience working on Hadoop stack including data ingestion tools like Kafka and Storm.
• Developed end to end Spark applications using Scala to perform various data cleansing, validation, transformation and summarization activities according to the requirements.
• Strong experience and knowledge of real time data analytics using Spark Streaming, Kafka and Flume.
• Working knowledge of Amazon's Elastic Cloud Compute (EC2) infrastructure for computational tasks and Simple Storage Service (S3) as Storage mechanism.
• Hands on experience in application development using PYTHON, RDBMS, and Linuxshell scripting.
• Experience with all stages of the SDLC and Agile Development model right from the requirement gathering to Deployment and production support.
• Also have experience in understanding of existing systems, maintenance and production support, on technologies such as Java, J2EE.
• Extensive experience in working with semi/unstructured data by implementing complex MapReduce programs using design patterns.
• Can understandJava development J2EE Frameworks like Struts, EJBs and Webservices.
• Working knowledge of Spark RDD, Data Frame API, Data set API, Data Source API, Spark SQL and Spark Streaming.
• Experience in writing MapReduce, YARN, PIG Scripts, Hive Queries, apache Kafka, Storm for analyzing Data
• Hands on experience on NoSQL databases such as HBase, Cassandra- bit knowledge on MongoDB
• Developing parser and loader map reduce application to retrieve data from HDFS and store to HBase and Hive

TECHNICAL SKILLS

Hadoop/Big Data Technologies: Hadoop 3.0, HDFS, MapReduce, HBase 1.4, Apache Pig 0.17, Hive 2.3, Sqoop 1.4, Apache Nifi Apache Impala 3.0, Oozie 4.3, Yarn, Apache Flume 1.8,Angular2.0, Kafka 1.1, Zookeeper 3.4

Hadoop Distributions: Cloudera, Hortonworks, MapR

Cloud: AWS, Azure, Azure SQL Database, Azure SQL Data Warehouse, Azure Analysis Services, HDInsight, Azure Data Lake and Data Factory.

Programming Language: Scala 2.12, Python 3.6, SQL, PL/SQL, Shell Scripting, Storm 1.0

Databases: Oracle 12c/11g, SQL

Database Tools: TOAD, SQL PLUS, SQL

Operating Systems: Linux, Unix, Windows 10/8/7

NoSQL Databases: HBase 1.4, Cassandra 3.11, MongoDB

Web/Application Server: Apache Tomcat 9.0.7, JBoss, Web Logic, Web Sphere

SDLC Methodologies: Agile, Waterfall

Version Control: GIT, SVN, CVS

PROFESSIONAL EXPERIENCE

Confidential, Clive, IA

Data Developer IV

Responsibilities:

• Python: Script reviewing for the data collection, reports development scripts, altercation. Made many different changes in the functionality. Created data frames for data type testing, did normalization analysis on data supporting data science team.
• Worked in Azure environment for development & deployment of Custom Hadoop Applications.
• Wrote pre-processing queries in python for internal spark jobs
• Worked on different kinds of manual rule writing which detected fraud through the internal web application using PQL.
• Loaded data from many different sources to the PostgreSQL database (Greenplum) which was parallel processed & integrated data science repository with the environment creating CI/CD pipelines.
• Wrote Table Schemas for Greenplum Database
• Supported architectural designing with the architects technically and by designing in Visio.
• Vendor/Offshore working: Have experience interacting and gathering requirements, supporting technically with vendors who worked externally to support the projects. FEEDZAI, PIVOTAL, FD, ERNST & YOUNG
• Performed real time integration and loading data from the Azure data box& mounting it onto fuse for bulk loads
• Wrote key spaces scripts in Cassandra and monitored cluster to create replication onto the database which follows gossip protocol.
• Created complex SSIS packages building pipelines and ETL for bulk & daily data injection.
• Worked on Hadoop HDInsight to inject Data Science Scoring models to Greenplum (Data Lake)
• Created multiple Sqoop jobs to inject TB’s of data from RDBMS to Hadoop

Environment: Hadoop, Cassandra, GreenPlum, SSIS, Sqoop, Zookeeper 3.4,MS Azure, Hadoop 3.0, Spark, Cloudera, Scala 2.12,DTCC, Python,Kafka, HDFS, Flume, UNIX, NoSQL, Kafka, Visio

Confidential, Washington, DC

Big Data/Hadoop Developer

Responsibilities:

• Worked as a Big Data/Hadoop Developer for providing solutions for big data problem.
• Worked in Azure environment for development and deployment of Custom Hadoop Applications.
• Developed Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data.
• Designed and implemented scalable Cloud Data and Analytical architecture solutions for various public and private cloud platforms using Azure.
• Involved in Various Stages of Software Development Life Cycle (SDLC) deliverables of the project using Agile Software development methodology.
• Experience in creating comprehensive web applications using MEAN stack implemented with MS SQL express.
• Involved in start to end process of Hadoop jobs that used various technologies such as Sqoop,PIG, Hive, MapReduce, Spark and Shells scripts.
• Implemented various Azure platforms such as Azure SQL Database, Azure SQL Data Warehouse, Azure Analysis Services, HDInsight, Azure Data Lake and Data Factory.
• Processed all DTCC confirm and affirm files based on established schedule.
• Created complex mappings in Talend 5.x
• Exported the analyzed data to the NoSQL Database using HBase for visualization and to generate reports.
• Imported and exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.

Environment: HBase, Sqoop 1.4, Zookeeper 3.4, Oozie 4.3, Hive 2.3, Pig 0.17, Agile, MS Azure, Hadoop 3.0, Spark, Cloudera, Scala 2.12,DTCC, Python,Kafka, HDFS, Flume, UNIX, NoSQL, Kafka, XML, JSON

Confidential, Chicago, IL

Big Data Developer

Responsibilities:

• As a Big DataDeveloper worked on Hadoop eco-systems including Hive, MongoDB, Zookeeper, Spark Streaming with MapR distribution.
• Involved in Agile methodologies, daily scrum meetings, sprint planning.
• Used Different Spark Modules like Spark core, Spark SQL, Spark Streaming, Spark Data sets and Data frames.
• Worked on MongoDB by using CRUD (Create, Read, Update and Delete), Indexing, Replication and Shading features.
• Extract Real time feed using Kafka and Spark Streaming and convert it to RDD and process data in the form of Data Frame and save the data as Parquet format in HDFS.
• Scripts were written for distribution of query for performance test jobs in Amazon Data Lake.
• Created Hive Tables, loaded transactional data from Teradata using Sqoop and worked with highly unstructured and semi structured data of 2 Petabytes in size.
• Implemented Apache Nifi flow topologies to perform cleansing operations before moving data into HDFS.
• Designed novel machine learning algorithms for predictive modeling and produce data insights (Data Science)
• Implemented multiple MapReduce Jobs in java for data cleansing and pre-processing.
• Build Hadoop solutions for bigdata problems using MR1 and MR2 in Yarn.
• Handled importing of data from various data sources, performed transformations using Hive, Pig, and loaded data into HDFS.
• Involved in identifying job dependencies to design workflow for Oozie&Yarn resource management.
• Configured Oozieworkflow to run multiple Hive and Pigjobs which run independently with time and data availability.
• Imported and exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BIteam.
• Support CloudStrategy team to integrate analytical capabilities into an overall cloud architecture and business case development.

Environment: Hadoop 3.0, Hive 2.3, Zookeeper 3.4, MapR, Agile, TeraData, MapReduce, Yarn, HDFS, Sqoop 1.4, Kafka 2.0, Apache Nifi, MongoDB,DTCC, Pig 0.17, HBase,Python, AWS, Scala, Spark, Oozie, Yarn, Cassandra

Confidential, Chicago, IL

Hadoop Developer

Responsibilities:

• Worked on analyzing Hadoopcluster and different Big Data analytic tools including Pig, Hive,HBase database and Sqoop.
• Involved in analyzing the specifications provided by the client and converting the specifications into the use cases.
• Involved in the design of the Referential Data Service module to interface with various databases using JDBC.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Responsible for loading the data from Oracle database, MySql into HDFS using Sqoop.
• Extensively used HTML and Ajax for client side development and validations.
• Exported data from HDFS environment into RDBMS using Sqoop for report generation and visualization purpose.
• Developed the business components using EJB Session Beans.
• Created Stateless Session EJB's for retrieving data and Entity Beans for maintaining User Profile.
• Used Azure Devops as an application server to deploy the code.

Environment: AJAX, Hadoop 2.3, Sqoop, Pig 0.15, Hive 1.9, HBase,MySql, HDFS 5, Struts, Kafka

Confidential

Data Science/Data Engineering Intern

Responsibilities:

• ML Implementation: Part of a team, responsible on developing full ML algorithm cycle, including, data extraction and preprocessing, design, implementation, visualization and results documentation.
• Data Analysis: Data Manipulation and Aggregation from a different source using Nexus, Toad, Business Objects, PowerBI and SmartView
• Implemented Agile Methodology for building an internal application.
• Focus on integration overlap and Informatica newer commitment to MDM with the acquisition of Identity Systems.
• Classification: Used multiple supervised learning algorithm, such as Logistic Regression, Decision trees, KNN, Naïve Bayes for classification problems.
• Fraud Detection: Applied various ML algorithm on data sets to predict credit risk and fraud detection.
• AWS Cloud Search: Used Python to develop data extraction processes from AWS.
• Big Data: Handled importing data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS.
• Data Preprocessing: Performed data cleaning and imputation of missing values using

Environment: Python, Anaconda, PYCHARM, Hadoop 2.3, Sqoop, Pig 0.15, Hive 1.9, HBase,MySql, HDFS, AWS