SUMMARY:

• Overall 7 years of experience as an IT professional in software development which includes 4 years of experience in Big - data related technologies on various domains like Insurance and Banking.
• Working experience in Apache Hadoop ecosystem components like HDFS, Map Reduce, Pig, Hive, Impala, HBase, SQOOP, Flume, Oozie, Spark, Scala and Kafka.
• Experience in working with major Hadoop distributions like Cloudera 5.x.
• Worked on Cloudera Impala and Apache Spark for real-time analytical processing
• Experience in optimizing Map Reduce Programs using combiners, partitioners and custom counters for delivering the best results
• Experience in writing Pig and Hive scripts and extending the core functionality by writing custom UDF’s.
• Good knowledge on File formats like sequence File, RC, ORC, Parquet and compression techniques like, gzip, snappy and LZO.
• Extensively worked on Hive and Impala
• INTEGRATION with various Hadoop Eco-System Tools:
• Integrated Hive and HBase for better performance.
• Integrated Impala and HBase for real-time analytics
• Integrated Hive and Spark SQL for high performance
• Did spark and HBase Integration for OLTP
• Worked on Apache Flume for collecting and aggregating huge amount of log data and stored it on HDFS for doing further analysis
• Experience in Importing Traditional RDBMS data to HDFS Using Sqoop and Exporting data from HDFS to RDBMS to generate reports
• Knowledge on Amazon’s S3 AWS bucket.
• Experience in writing both time and data driven workflows using Oozie.
• Solid understanding of algorithms, data structures and object-oriented programming
• Knowledge on NoSQL columnar databases like HBase
• Experience in managing and troubleshooting Hadoop related issues
• Good knowledge and understanding of Java and Scala programming languages
• Knowledge on Linux and shell scripting
• Diverse experience in utilizing Java tools in business, Web, client-server platforms using core java, JSP, Servlets, Spring, Structs, Hibernate, Java Database Connectivity (JDBC) and application servers like Apache Tomcat
• Improved the performance and optimization of the existing algorithms in Hadoop using SparkContext, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.
• Hands on experience in working on Spark SQL queries, Data frames, import data from Data sources, perform transformations, perform read/write operations, save the results to output directory into HDFS.
• Implemented POC’s using Kafka, spark Streaming and Spark SQL.
• Knowledge in using SQL Queries for backend database analysis

TECHNICAL SKILLS:

Big data/Hadoop: HDFS, Map Reduce, YARN, HIVE, Impala, PIG, Sqoop, Oozie, Flume, HBASE, Kafka

Apache Spark: Spark Core, Spark SQL, Spark Streaming, Scala

Hadoop Distribution: Cloudera

Java/J2EE Technologies: Java, J2EE, Servlets, JDBC, XML, AJAX, REST

Programming Languages: Java, Scala, Linux shell scripts

NoSQL DB Technologies: HBase

Database: Reporting Tools

Oracle, MySQL, DB2: Tableau

Web Servers: Tomcat

Web Technologies: HTML5, CSS, XML, JavaScript

Operating Systems: Ubuntu (Linux), Win 95/98/2000/XP, cent os,OS-390

PROFESSIONAL EXPERIENCE:

Confidential, Chicago, IL

Hadoop Developer

Responsibilities:

• Played a Hadoop developer role and involved in all the phases of the project, starting from POC’s till implementation
• Performed advanced procedures like text analytics and processing, using the in-memory computing capabilities of Spark using Scala.
• Used Spark API over Hadoop YARN to perform analytics on data in Hive.
• Developed Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data.
• Involved in data migration activity using Sqoop JDBC drivers for oracle and IBM db2 connectors.
• Worked on full and incremental imports and created Sqoop jobs
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Validated data using pig scripts to eliminate the bad records.
• Loaded data from LINUX file system to HDFS and created separate directory for every four hour window.
• Extensively used Pig for data cleansing and other validations.
• Created data model for structuring and storing the data efficiently. Implemented partitioning and of tables in HBase.
• Involved in creating Hive tables, loading the data and writing Hive queries which will run internally in map reduce way.
• Worked with various Hadoop file formats, including ORC and parquet.
• Modeled Impala partitions extensively for data separation to perform faster processing of data, and followed best practices for tuning.
• Involved in integration of Hive and HBase.
• Implemented bucketing, partitioning and other query performance tuning techniques.
• Worked on Oozie workflow engine for job scheduling.
• Involved in building high performance batch and interactive data processing applications, on Hive jobs.
• Hands on experience in exporting the results into relational databases using Sqoop for visualization and to generate reports for the BI team using Tableau.
• To read files we used xml parsing technique in spark by writing code in Scala (POC).
• Developed Impala queries to process the data and generate the data cubes for visualizing and reports.

Environment:: Hadoop(CDH5.x), Linux, HDFS, Map Reduce, Sqoop, Impala, Pig, Oozie, HBase, Tableau, Spark, Scala, Oracle, IBM DB2, Eclipse,WinScp

Confidential, Madison, WI

Hadoop Developer

Responsibilities:

• Involved in copying data generated by various services to HDFS for further processing using Flume.
• Used Oozie and Zookeeper operational services for coordinating cluster and scheduling workflows.
• Wrote Java API for HBase transactions on HBase tables and involved in building Oozie work flows.
• Processing the data from S3 AWS bucket(source data to shared storage area) and loading into HDFS.
• Wrote complex Impala queries using aggregate and windowing functions.
• Loading the data from the different Data sources like (DB2) into HDFS using Sqoop and load into Impala tables, which are partitioned.
• Involved in integration of Impala and HBase
• Stored the customer’s data onto HBase for further transactions and historical data onto Impala.
• Hands on experience in reviewing and managing Hadoop log files.
• Wrote Java API (REST) web services.
• Loaded data from LINUX file system to HDFS using shell script.
• Extensively used Pig for validations.
• Hands on writing Map Reduce code to make unstructured data as structured data and for inserting data into HBase from HDFS.
• Performed optimization on existing MapReduce programs by using customized partitioner, combiner
• Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms.
• Wrote CLI commands using HDFS.
• Performed data analysis on large datasets of policy, period, customer and Claims data.
• Added Log4j to log the errors.
• Used Eclipse for writing code.
• Involved in creating Impala tables, loading with data and writing Impala queries for real-time analytical processing
• Monitored the health of Map Reduce Programs which are running on the cluster.

Environment:: Cloud era, Hadoop, Linux, MapReduce, HDFS, Impala, Pig, Shell Scripting, Java, NoSQL, Eclipse, Oracle, Filezilla, Spark, Scala

Confidential, Pheoniz, AZ

Java Developer

Responsibilities:

• Involved in various phases of Software Development such as modeling, system analysis and design, code generation and testing using AGILE Methodology.
• Participated in daily stand up meetings.
• Designed and Developed web interface in J2EE framework using Struts framework (MVC Controller) HTML as per Use Case specification.
• Involved in developing JavaScript for client data presentation and, data validation on the client side with in the forms.
• Created connection through JDBC and used JDBC statements to call stored procedures.
• Produced visual models of the system by generating UML use-case diagrams from the requirements.
• Designed, developed and deployed application using Eclipse and Tomcat application Server.
• Classes are designed by using Object oriented Design (OOD) concepts like encapsulation, inheritance etc.
• Created Custom Tags to rescue the common functionality.
• Participated and review of the module using the user requirement documents.
• Involved in testing the module as per user requirements.

Environment:: Java, Eclipse, Struts, JDBC, JSP, Servlets, HTML, JavaScript, hibernate.

Confidential

Application Developer

Responsibilities:

• Responsible for Different kind of projects (B2B & Migration) for Confidential Loyalty management systems.
• Performed the Code changes and testing for break-fix requests.
• Involved in all phases from analyze to Test phase sign - off.
• Daily status reports preparation and attending daily status calls with all stakeholders of the project.
• Daily test execution tracking, defect tracking and coordination with Dev & Business team’s onsite.
• Test Environment Set up of all the jobs of an application is a major activity.
• Executed the batch jobs for generating the files with application layout format (up-stream files and in-stream files) of all the assigned projects.
• Implemented web page layout using struts tiles libraries, and performed validations using Struts validation framework.
• Developed front-end using JSP, HTML, JavaScript and CSS.
• Daily test execution tracking, defect tracking and coordination with Development teams.
• Completing the SIT phase and provided Technical support during UAT phase.
• Executed UNIX scripts that creates & sends upstream and downstream data to Market DB environment.
• Preparation of test conditions, test scripts and test data based on the requirement for all assigned projects, executed test results and updating the same with the screenshots in Quality center tool.

Environment:: z/OS 390,Cobol, DB2, JCL,IMS DB, Vsam, JAVA, JSP, Structs, File-aid, Changeman, Unix, Oracle, SQL, QC tool, Control-M.