SUMMARY

• me has about 7+ years of professional IT experience which includes experience in Big data ecosystem experience in complete project life cycle (design, development, testing and implementation) of which over 3+ years of work experience in ingestion, storage, querying, processing and analysis of Big Data with hands on experience inHadoop Ecosystem (YARN, HDFS) and its components Hive, Pig, HBase, Sqoop, Hue, Kafka, Flume, Oozie, Zookeeper, Spark, SparkSQL andSparkStreaming.
• Worked Hands on in Hadoop clusters like Hortonworks, AWS Elastic Map Reduce and Cloudera.
• me has hands on experience in improving teh performance and optimization of teh existing algorithms in Hadoop usingSparkcontext,Spark - SQL, Data Frame, pair RDD's &SparkYARN.
• me has working experience on building spark applications using build tools like SBT, Maven and Gradle.
• me has good experience in dealing with different file formats like text, Sequence, RCFILE, ORC, Parquet, Avro and JSON and different compression formats like GZip, LZO, BZip2 and snappy.
• me has good noledge on relational databases like MySQL, Oracle and NoSQL databases like HBase, MongoDB. working noledge on UNIX /Linux systems including Experience on shell scripting working experience in handling semi/un-structured data from different data sources.
• Working experience in developing Map Reduce programs using Combiners, Map side join, Reducer side join, Distributed Cache, Compression techniques, Multiple Input & output.
• me has working experience in performing ad-hoc analysis on structured data using HiveQL, joins and Hive UDF's good exposure to Counters, Shuffle & Sort parameters, Dynamic Partitions, Bucketing for performance improvement.
• me has worked in using IDE like Eclipse and Intellij IDEA
• me has working noledge in Java and SQL in application development and deployment.

TECHNICAL SKILLS

Big Data Associated: HDFS, MapReduce, Pig, Hive, Sqoop, Flume, HBase, Oozie, Apache Spark, Spark SQL, Spark Streaming.

Process/Data Modeling: MS Visio, UML Diagrams and ER Studio

Cluster Manager Tools: HDP Ambari, Cloudera Manager, Hue

ETL/ELT/Databases: HBase, MongoDB, Spark SQL, MS Access, Oracle, DB-II, My SQL, SQL Developer, SQL Server 2000/2005/2008 and Toad

Languages: C, C++, Java, PL/SQL, Python, Scala

Web-Technologies: HTML, DHTML, XML, CSS

Microsoft Technologies: ASP.NET, C#.Net, VB.Net, ADO.NET, SharePoint, Word, Excel and PowerPoint.

Operating Systems: Linux, Ubuntu, RHEL, Windows 2000/2003/2008/ XP/7/8/10.

IDE: Eclipse and Intellij IDEA

PROFESSIONAL EXPERIENCE

Confidential, Chicago, IL

Hadoop/Spark Developer

Responsibilities:

• Worked with lambda architecture in handling and processing batch and real-time data.
• Using Sqoop, ingested teh Data from data warehouse to HDFS.
• Using Kafka, collected real-time streaming and log data from web applications and click stream data, analyzing a part of data using spark streaming and rest stored into HDFS for future use.
• Worked in writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HiveQL) and Worked with Hive Tables, Hive queries, Partitioning, Bucketing.
• PerformedDataProfiling, identifydataquality and validating rules regarding dataintegrity anddataquality as it relates to teh impact on business requirements.
• Build spark applications using SBT builds.
• Used Spark SQL to process teh huge amount of structured data.
• Connected Tableau server to publish dashboard to a central location for portal integration.
• Creation of metrics, attributes, filters, reports, and dashboards created advanced chart types, visualizations and complex calculations to manipulate teh data.

Environment: Cloudera Manager, Sqoop, Java (jdk1.8 Version), Hive, Spark, Spark-SQL, Scala, Tableau.

Confidential, NYC, NY

Hadoop/Spark Developer

Responsibilities:

• Worked in Ingesting flat files from local Unix file systems to HDFS and using Sqoop ingested structured data from legacy RDBMS systems to
• Developed teh code for Importing and exporting data into HDFS and Hive using Sqoop
• Exploring with teh Spark for improving teh performance and optimization of teh existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.
• Used Data Frame API in Scala for converting teh distributed collection of data organized into named columns, developing predictive analytic using Apache Spark Scala APIs.
• Worked using Apache Hadoop ecosystem components like HDFS, Hive, Sqoop and Worked with Spark and Scala.
• Writing Hive join query to fetch info from multiple tables, writing multiple Map Reduce jobs to collect output from Hive Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting on teh dashboard.
• Utilized Oozie workflow to run Hive Jobs Extracted files through Sqoop and placed in HDFS and processed.

Environment: Hadoop, Spark, HDFS, Scala, Hive, Java, Spring, Map Reduce, Sqoop, Spring MVC, Big Data, Spark SQL, JDBC, Oozie, Pig, Flume

Confidential, Seattle, WA

Hadoop Developer

Responsibilities:

• Worked on analyzing data using different big data analytic tools including Pig, Hive and MapReduce.
• Created Pig Latin scripts to sort, group, join and filter teh enterprise wise data.
• Implemented Partitioning, Dynamic Partitions, and Buckets in Hive on Avro files to meet teh business requirements.
• Implemented Data Integrity and Data Quality checks using Linux scripts.
• Used flume to tail teh application log files into HDFS.
• Involved in scheduling of Hive and pig jobs using Oozie workflow.
• Involved in performance tuning and memory optimization of map-reduce and Hive applications.
• Worked on end to end automation of application.
• Responsible for continuous Build/Integration with Jenkins and deployment using XL Deploy.
• Actively involved in code review and bug fixes and enhancements.

Environment: Hadoop, HDFS, MySQL, Apache Hive, Pig, MapReduce, MySQL, Core Java, Shell Scripting, Eclipse, Git, Jenkins.

Confidential

SQL/PL-SQL Developer

Responsibilities:

• Created custom PL/SQL procedures to read data from flat files to dump to Oracle database using SQL * Loader.
• Developed PL/SQL Procedures and database triggers for teh validation of input data and to implement business rules.
• Created records, tables, collections for improving performance by reducing context switching.
• Created database objects like packages, procedures, and functions according to teh client requirement.
• Used SSIS to create ETL packages to validate, extract, transform and load data to data warehouse databases, data mart databases to store data to OLAP databases.
• Created teh PL/SQL packages, procedures, functions applying teh business logic to load teh data to relevant tables database and Converted different source system data into oracle format T-SQL.
• Created and manipulated stored procedures, functions, packages and triggers using TOAD.
• Responsible to tune ETL mappings to optimize load and query Performance
• Developed Oracle Forms for form end user using oracle form builder 10g.
• Extensively used teh advanced features of PL/SQL like Records, Tables, Object types and Dynamic SQL.
• Tune ETL procedures and STAR schemas to optimize load and query Performance.

Environment: Oracle 10g, T-SQL, SQL*Plus, SQL*Loader, PL/SQL Developer, Web Services, SSIS, SSRS, TOAD.

Confidential

Java/Web Developer

Responsibilities:

• Involved in various phases of Software Development Life Cycle (SDLC) as design development and unit testing.
• Developed and deployed UI layer logics of sites using JSP, XML, JavaScript, HTML/DHTML and Ajax.
• CSS and JavaScript were used to build rich internet pages.
• Agile Scrum Methodology been followed for teh development process.
• Designed different design specifications for application development that includes front-end, back-end using design patterns.
• Developed proto-type test screens in HTML and JavaScript.
• Involved in developing JSP for client data presentation and, data validation on teh client side with in teh forms.
• Developed teh application by using teh Spring MVC framework.
• Collection framework used to transfer objects between teh different layers of teh application.
• Developed data mapping to create a communication bridge between various application interfaces using XML, and XSL.
• Spring IOC being used to inject teh parameter values for teh Dynamic parameters.
• Developed JUnit testing framework for Unit level testing.
• Actively involved in code review and bug fixing for improving teh performance.
• Documented application for its functionality and its enhanced features.
• Created connection through JDBC and used JDBC statements to call stored procedures.

Environment: Spring MVC, J2EE, Java, JDBC, Servlets, JSP, XML, Design Patterns, CSS, HTML, JavaScript, Junit, Apache Tomcat, My SQL Server 2008.