SUMMARY

• Over 6 years of extensive IT experience, with extensive working experience in Hadoop Eco System and Big Data Development with expertise in MapReduce, HDFS, YARN, HIVE, HIVE Query Language, Pig, Impala, Zookeeper, Oozie, Apache Spark and Zeppelin and broad scale experience in JAVA related technologies and J2EE framework integrated with various Database technologies.
• AWS Certified Developer - Associate.
• Experience in managing and analyzing massive dataset on multiple Hadoop frameworks like Cloudera and Hortonworks.
• Experience and good understanding of Big Data real time streaming technologies like Apache Spark and Apache Kafka.
• Expertise in data analytics and root cause analysis using HIVE, Pig, SparkSQL and DataFrames.
• Working experience and good understanding of teh various components of Apache Spark like RDD, DataFrames, Paired RDD and implementing various transformations and actions using Scala.
• Experience in Data Ingestion from external data source like MySQL and Oracle into HDFS using Flume and Sqoop in various file formats like text, xml and json.
• Worked on developing UDFs using JAVA for HIVE tables to provide customized functionalities dat is not provided by teh inbuilt Hadoop architecture.
• Extensive working noledge of Hadoop architecture and maintenance and monitoring of components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, YARN and Map Reduce programming paradigm.
• Good noledge and experience in working with various No-Sql databases like HBase, Cassandra and Amazon RedShift.
• Experience in working in Hadoop eco-system integrated to Cloud platform provided by AWS with several services like Amazon EC2 instances, S3 bucket and RedShift.
• Experience in implementing and development of SDLC phases using Object Oriented concepts in JAVA, J2EE using frameworks like Spring and Hibernate and implementing database connections using JDBC.
• Experience in working with web services like RESTful and SOAP in JAVA for data transfer.
• Good hands on experience in configuration, deployment and management of enterprise applications on Application Servers like Web Sphere, JBoss and Web Servers like Apache Tomcat.
• Expertise in working with various database like MySQL, Oracle and Microsoft SQLServer and data manipulation and analysis using SQL Queries and Stored Procedures.
• Strong understanding of Data warehouse concepts like ETL, Star Schema, Snowflake, data modeling experience using Normalization, Business Process Analysis, Dimensional Data modeling, physical & logical data modeling.
• Experience in developing applications using Waterfall and Agile methodologies and understand teh roles and responsibilities being a part of teh sprint.
• Experience in Code version control using Git and maintain repositories as a best practice.

TECHNICAL SKILLS

Programming Languages \Databases \: C, Core Java, J2EE, Scala.\Oracle, MySQL, SQL Server.\

Big Data Technologies\Operating Systems\: Hadoop 2.x, HDFS, Map Reduce v2, Hive \Windows, UNIX, Linux distributions \1.2.1, Pig 0.12/0.14, HBase 0.98.0, Impala, \(Centos, Ubuntu).\Hue, Sqoop, Kafka 0.8.2, Oozie, Flume \1.5.2, Zookeeper 3.4.6, Spark 1.6, Cloudera \CDH 4/CDH 5, and Hortonworks HDP \ 2.4/2.5/2.6. \

NoSQL Databases\Scripting and Query Languages\: HBase and Cassandra.\UNIX Shell scripting, SQL and PL/SQL.\

Hadoop Paradigms\Other Tools\: Map Reduce, YARN, In-memory computing, \Eclipse, Tableau 10.1, JUnit, SBT, Maven.\High Availability, Real-time Streaming.\

Amazon Web Services\: EC2 Instance, S3 Bucket, Amazon RedShift.\

PROFESSIONAL EXPERIENCE

Confidential, Plano, TX

Data Engineer

Responsibilities:

• Involvement in development and enhancement of code implemented for absolute data quality check.
• Performed JUnit testing of code blocks for checking code coverage, before using it in CICD implementation.
• Developed shell-scripts for automating various file and log processing within teh system.
• Involved in Data extraction from Teradata and loading it into Amazon S3 and perform various validations for teh same.
• Involved in implementing spark jobs and configuring teh parameters for teh same.
• Involved in resolving ad-hoc data issues related to day to day business related to loan services.
• Involved in synchronizing teh Financial Service Data Lake with teh Enterprise Data Lake.

Environment: Spark 2.2.1, AWS S3, Snowflake, Teradata, Scala 2.11.8, JAVA 1.8, JUnit-4, Shell Script

Confidential, Chicago, IL

Spark Developer

Responsibilities:

• Experience in handling more TEMPthan 2.5 TB of daily data from teh Point of Sales.
• Experience writing Scala code to process sales data on Spark framework 1.6 to get useful insights for business.
• Performed data ingestion from various sources using Kafka data pipeline.
• Experience in working in a data pipeline consisting of Kafka along with Apache Spark.
• Used various levels of transformations and actions to perform various operations like geographical drilling of data on teh fly to feed data into teh spark system.
• Used thevarious Sparkfunctionalities for improving teh performance and optimization of teh existing algorithms in Hadoop usingSparkContext,Spark-SQL, Data Frame, and Pair RDD's.
• CreatedSparkApplication to load data into Dynamic Partition Enabled Hive Table based on geographical locations dat helped in faster processing teh data.
• Used Databricks XML plug-in to parse teh incoming data in teh XML format, and as well generate teh required XML as output.
• Worked for filtering data according to tags in xml file dat refers to various events in a transaction at teh point of sales.
• Experience in writing Hive jobs to parse teh logs and structure them in tabular format to facilitate TEMPeffective querying on teh log data.
• Optimized teh data sets by creating dynamic partitioning in Hive.
• Developed business specific custom UDF's in Hive.
• Experience in performance tuning of Spark Applications for setting right Batch Interval time, correct level of Parallelism and memory tuning.
• Responsible for maintaining teh whole Big Data system on Amazon AWS RedShift.

Environment: Apache Hadoop 2.6.0, YARN, HDFS, Scala 2.10.5, Spark Core 1.6, Spark SQL, Java API, Hive 1.2.1, Kafka 0.8.2, Eclipse Neon, Hortonworks HDP 2.4.3, Zookeeper

Confidential, Omaha, NE

Hadoop Developer

Responsibilities:

• Involved in teh data acquisition, data pre-processing and data exploration of digital data of teh business.
• Worked with Data Scientists and QA team to understand, design and develop an end to end data flow requirements.
• Used Sqoop to acquire teh required data from teh source to teh HDFS using incremental approach.
• Used HIVEQL and Pig scripts for data analysis.
• Loaded teh data into HIVE and Impala tables for further analysis.
• Involved in creating HIVE tables and implement partitioning and bucketing for increasing teh speed of query processing.
• Performed various analysis and data trending to provide useful insights about teh customers to teh users.
• Designed HIVE UDFs using JAVA to implement customized functions.
• Used Sqoop for both import and export of teh required data in and out of teh Hadoop eco-system on demand.
• Involved in teh continuous enhancements and troubleshooting problems in teh production.
• Involved in designing Oozie workflow for scheduling and orchestrating teh whole data flow of teh system.
• Involved in close monitoring of Hadoop log files to troubleshoot any issues.
• Responsible for monitoring of Zookeeper for a smooth coordination among teh various components.
• Responsible for performing unit testing of deliverables for UAT.
• Used Tableau for Data visualization to identify teh role of various factors.
• Involved in identifying KPIs.

Environment: HiveQL, MySQL, HIVE 1.2.0, Impala 2.1.0, Eclipse (Neon), Hadoop 2.x, Cloudera CDH 4,Oracle 11g, PL/SQL, Toad 9.6, Flume, PIG, Sqoop 1.4.6, Unix, Tableau 9.3.1, Zookeeper 3.4.8

Confidential - Omaha, NE

ETL Offload - Data Engineer

Responsibilities:

• Experience in migrating data of size 15 - 20 TB from teh old Mainframe System to teh Hadoop Eco System.
• Worked closely with teh business analysts and System Engineers to convert teh Business Requirements into Technical Requirements.
• Used Sqoop 1.4.6 import techniques to fetch data from teh old system.
• Involved in designing Sqoop scripts and engineering teh map programs to ensure efficient parallelism of data.
• Involved in designing of HDFS storage to have efficient number of block replicas of data.
• Involved in structuring HIVE tables to store teh imported data extracted by Sqoop.
• Experience in writing ETL jobs using HIVE QL.
• Configured periodic incremental imports of data from Oracle into HDFS using Sqoop.
• Wrote Pig Scripts to generate MapReduce jobs and performed ETL procedures on teh data in HDFS.
• Used JAVA to design customized MapReduce programs on demand.
• Developed custom programs in JAVA from time to time make required changes in ETL.
• Experience setting up asynchronous inter cluster Oozie workflows.
• Followed simple Data Modeling techniques to enable dataflow easier and faster.
• Designed and developed Oozie workflows for sequence flow of job execution.
• Involved in Hadoop Cluster Administration, building cluster with different services, up gradation of HDP and Ambari Server.

Environment: JAVA 1. 7. 0, Oracle Server, HDP 2.4.2, MapReduce2, Hive 1.2.1, Pig 0.16.0, Sqoop 1.4.6, Oozie 4.2.0, Zookeeper 3.4.6, Apache Maven

Confidential

JAVA Developer

Responsibilities:

• Responsible for gathering requirements from business owners and checking teh feasibility of teh requirements.
• Worked with UI designers of teh application and developed objects and JAVA entities accordingly.
• Developing teh java logic to fetch data from teh front end and save data to database and vice-versa.
• Involved in writing efficient stored procedure in MySQL.
• Responsible for unit testing and preparing test case documents as per adherence to teh company.
• Involved in giving demos to business owner and take signoffs.
• Responsible for maintaining code versions in Git.
• Responsible for creating dashboards for senior management.
• Responsible for learning VB and including teh same in teh application.
• Involved in upgrading teh existing system dat was not dat interactive from teh patient point of view.
• Involved in developing event interfaces where doctors can put up their availability.
• Responsible for integrating attendance reports for doctors as well as patients required by senior management.

Environment: Java 6, JSF 2.1, Primefaces 3.3.1, HTML, CSS, JavaScript, RESTFUL webservices, MySQL

Confidential

JAVA Developer

Responsibilities:

• Responsible for gathering requirements from business owners and checking teh feasibility of teh requirements.
• Used Spring MVC and Hibernate Framework for data handling.
• Implemented Hibernate API for database connectivity.
• Developed Hibernate mapping configuration files to provide teh relation between java objects and database tables.
• Involved in UI design of teh application and developing teh screens as per teh approved designs.
• Developing teh Java logic to fetch data from teh front end and save data to database and vice-versa.
• WroteJUnittest cases to test teh server-side Modules.
• Responsible for writing efficient stored procedure in MySQL.
• Responsible for unit testing and preparing test case documents as per adherence to teh company
• Responsible for giving demos to business owner and take signoffs.
• Responsible for maintaining code versions in Git.

Environment: Java 6, Spring MVC, Hibernate, MySQL, Eclipse Helios, Git, Webservices like SOAP and RESTFUL, MySQL database