PROFESSIONAL SUMMARY:

• Having 7+ years of total IT Experience, including 3+ years hands - on experience in Big-data/Hadoop Technologies.
• 4+ years of extensive experience in JAVA/J2EE Technologies, Database development, ETL Tools, Data Analytics.
• Very good experience in the Application Development and Maintenance of SDLC projects using various technologies such as Java/J2EE, JavaScript, Data Structures and UNIX shell scripting.
• Expertise in all components of Hadoop Ecosystem- Hive, Hue, Pig, Sqoop, HBase, Impala, Flume, Zookeeper, Oozie, and Apache Spark.
• Having sound experience in Big Data HadoopEcosystems experience in Ingestion, storage, querying, processing and analysis of big data.
• Excellent understanding/knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, YARN and Mapreduce programming paradigm.
• Experience in importing and exporting data from different RDBMS like MySql, Oracle and SQL Server into HDFS and Hive using Sqoop.
• Knowledge on Decriptivestatistical methods such as Central Tendency, Distribution of the Data and Inferential Statistical methods.
• Experience in developing custom Mapreduce programs using Apache Hadoop to perform Data Transformation and analysis as per requirement.
• Experience in writing PIG scripts and Hive Queries for processing and analyzing large volumes of data.
• Experience in creating PIG and HIVE UDFs using java in order to analyze data sets.
• Experience in collecting the log data from different sources like (web servers and social media) using Flume and storing on HDFS to perform MapReduce jobs.
• Experience in Spark Streamingin order to ingest data from multiple data sources into HDFS.
• Good Hands on Experience in Stream Processing including Storm and Spark Streaming.
• Experience in Design & Development, tuning and maintenance of NoSQL databases such as MongoDB, HBase, Cassandra and its Integration with Hive.
• Used Hbase in accordance with PIG/Hive as and when required for real time low latency queries.
• Developed multiple POCs using PySpark and deployed on the Yarn cluster, compared the performance of Spark, with Hive and SQL.
• Hands on experience in advanced Big-Data technologies like Spark Ecosystem (Spark SQL, MLlib, SparkR and Spark Streaming), Kafka and Predictive analytics (MLlib, R ML packages including Oxdata's ML library H2O)
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Spark SQL and Scala.
• Worked on reading multiple data formats on HDFS using PySpark.
• Involved in database design, creating Tables, Views, Stored Procedures, Functions, Triggers and Indexes.
• Strong experience in Data Warehousing and ETL using Datastage.
• Experience in developing the complex SQL queries, unions and multiple table joins and experience with views.
• Experience with various Business Intelligence tools and SQL databases.
• Extensively worked on data extraction, Transformation and loading (ETL) data from various sources like Oracle, SQL Server and flat files.
• Experience in preparing and executing unit test plan and unit test cases after software development.
• Proficient in Retail, Telecom and Banking Domains.
• Ability to work effectively with associates at all levels within the organization.

TECHNICAL SKILLS:

Programming Languages: SQL, C, C++, Java, Core Java, Java 8, J2EE, Python, Scala, Pig Latin, HiveQL and Unix shell scripting

Big Data Technologies: Hadoop, HDFS, MapReduce, Elasticsearch, Hive, Pig, HBase, Impala, Hue, Sqoop, Storm, Talend, Kafka, Oozie, SparkSql, MlLib, SparkR, Flume, Zookeeper, Cassandra, Spark 1.4, IBM BigInsights, Cloudera CDH4, DataMeer and Hortonworks.

Databases: Oracle, MySQL, SQL Server, DB2, Familiar with NoSQL (HBase, Cassandra, MongoDB)

Scripting & Query Languages: UNIX Shell scripting, SQL and PL/SQL.

Web Technologies: JSP, Servlets, JavaBeans, JDBC, AWT, Swing, JSF, XML, CSS, HTML, XHTML, JavaScript, AJAX.

Hadoop Paradigms: MapReduce, YARN, In: memory computing, High Availability, Real-time Streaming.

Operating Systems: Windows, UNIX, Linux distributions (Centos, Ubuntu), Mac OS

Other Tools: Eclipse, Tableau 9.0, JUnit, QTP, JIRA, QC (Quality Center)

PROFESSIONAL EXPERIENCE:

Confidential

Java/Hadoop/Spark Developer

Responsibilities:

• Installed and configured Apache Hadoopclusters using YARN for application development and Apache toolkits like Apache Hive, Apache Pig, HBase, Apache Spark, Zookeeper, Flume, Kafka and Sqoop.
• Worked extensively on Spark, Hive, Sqoop, Shell, Pig, Scala and Python.
• Enhanced performance for PIG and Hive using TEZ.
• Used Sqoop to efficiently transfer data between databases and HDFS and used Flume to stream the log data from servers.
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, and Scala.
• Developed Sparkcode using Scala and Spark-SQL/Streamingfor faster testing and processing of data.
• Developed multiple POCs using Scala and deployed on the Yarn cluster, compared the performance of Spark, withCassandraand SQL.
• Used IMPALA to write and query the Hadoopdata in HDFS orNoSql HBaseand pull the data from Hive tables.
• Descriptive StatisticalAnalysis using dashboards, centering, dispersion and visualization
• Involved in loading data from UNIX file system to HDFS.
• Developed Sqoop scripts for data ingestioninto Hadoopenvironment.
• Used SparkAPI over Hadoop YARN to perform analytics on data in Hive.
• Created tasks for incremental load into staging tables, and schedule them to run.
• Used Apache Kafka as messaging system to load log data, data from UI applications into HDFS system.
• Used Pig to perform data transformations, event joins, filter and some pre-aggregations before storing the data onto HDFS.
• Created Hive External tables with partitioning to store the processed data from MapReduce.
• Implemented Different analytical algorithms using MapReduce programs to apply on top of HDFS data.
• Implemented Hive optimized joins to gather data from different sources and run ad-hoc queries on top of them.
• Wrote Hive Generic UDFs to perform business logic operations at record level.
• Implemented Data classification algorithms using MapReduce design patterns.
• Extensively worked on creating combiners, Partitioning, distributed cache to improve the performance of MapReduce jobs.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig and used Zookeeper to coordinate the clusters.
• Implemented test scripts to support test driven development and continuous integration.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Troubleshooting the cluster by reviewingHadoopLOGfiles.
• Worked on reading multiple data formats on HDFS using PySpark
• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python and Scala.
• Developed multiple POCs using PySpark and deployed on the Yarn cluster, compared the performance of Spark, with Hive and SQL/Teradata.
• Used Zookeeper for various types of centralized configurations.

Environment: Hadoop 2.3.0, Spark core, Scala, Python, SparkSql, PySpark, Spark Streaming, Hive, Impala, Pig, Sqoop, Kafka, Zookeeper, Control-M, Java 8, and UNIX Shell Scripting.

Confidential, Charlotte, NC

Hadoop/ ETL Developer

Responsibilities:

• Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users
• Responsible for building scalable distributed data solutions using Hadoop.
• Used Datastage Tool for extraction, transformation and load (ETL) of data in the data warehouse.
• Involved in writing MapReduce programs using combiners and partitioner.
• Developed shell scripts to periodically perform incremental import of data from third party API to Amazon AWS.
• Created and optimized Hive queries to analyze the customer frustration level which helped reduce the call traffic and ensured better customer satisfaction
• Involved in creating Hive tables, loading structured data and writing hive queries which will run internally in MapReduce way.
• Developed Validation scripts for Data Ingestion.
• Created PIG scripts to load, transform and store the data from various sources into HIVE metastore.
• Monitored OOZIE workflows and managing the coordinator schedules and rerunning the failed jobs to ensure the data is processed accurately and in a timely manner.
• Monitoring and managing daily jobs, Processing around 200k files per day and monitoring those through RabbitMQ and Apache Dashboard application.
• Gained good experience in Amazon Web Services (EC2, S3,EMR) and Imported data (csv, plain text) from Amazon S3 using HDFS commands
• Developed and maintained ETL (Extract, Transformation and Loading) mappings to extract the data from multiple source systems like Oracle, SQL server and Flat files and loaded into Teradata.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Experienced in managing and reviewing Hadoop log files.
• Involved in loading data from UNIX file system to HDFS.
• Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
• Load and transform large sets of structured, semi structured and unstructured data.
• Responsible to manage data coming from different sources.
• Supported MapReduce Programs those are running on the cluster.
• Involved in creating new table structures and modifying existing tables and fit into the existing Data Model.
• Created Teradata SQL Queries, creating Tables, and Views by following Teradata best practices.
• Tested the different sources such as Flat files, Mainframe Legacy Flat Files, SQL server 2008 to load into the Teradata data Warehouse.

Environment: Apache Hadoop, Amazon AWS, Hive, Pig, Oozie, Sqoop, RabbitMQ, Data stage, Teradata, Oracle 11g, Hbase, Toad, MS-SQL Server, Unix Shell Scripting.

Confidential, Tampa, FL

Java developer

Responsibilities:

• Involved in Various Stages of Software Development Life Cycle (SDLC) deliverables of the project using the AGILE Software development methodology.
• Attended Daily scum meeting, sprint grooming/review and demo with management and other teams.
• Implemented the application using Spring MVC Framework and handled the security using spring security.
• Developed Controller Classes using spring MVC, spring AOP frame work.
• Designed and Developed End to End customer self service module using annotation based Spring MVC, Hibernate, Java Beans and JQuery.
• Developed the User Interface using JSP, JQuery, HTML5, CSS3, Node JS, Bootstrap JS and Angular JS.
• Implement functionality such as searching, filtering, sorting, categories, validating using Angular framework.
• Used Bootstrap library to quickly build project UI's and used AngularJS framework to associate HTML elements to models.
• Used Angular directives, working on attribute level, element level and class level directives.
• Implemented Bean classes and configured in spring configuration file for dependency injection.
• Implemented data access layer i.e. DAO Classes using Hibernate as an ORM tool and configured xml files according to hibernate framework.
• Implemented persistence layer using Hibernate that use the POJOs to represent the persistence database.
• Created mappings among the relations and written named HQL queries using Hibernate.
• Design common framework for REST API consumption using Spring Rest Templates.
• Used Design Patterns like Facade, Data Transfer Object (DTO), MVC, Singleton and Data Access Object (DAO).
• Written SQL and PL/SQL queries like stored procedures, triggers, indexes, views.
• Used Log4j, Junit for logging and Testing.
• Documented all the SQL queries for future testing purpose.
• Prepared test case scenarios and internal documentation for validation and reporting.
• Responsible for maintaining and expanding AWS (Cloud Services) infrastructure using AWS (SNS, SQS).
• Coordinating with the QA team, and resolving the QA defects.
• Wrote services to store and retrieve user data from the Mongo DB.
• Worked with WebSphere application server that handles various requests from Client.
• Deploying fixes and updates using IBM WebSphere application server.
• Experience in developing automated unit testing using Junit, Mockito frameworks.
• Used Git controls to track and maintain the different version of the project.

Environment: Java, Spring, Hibernate, Web Services, Html 5, CSS3, Java Script, Angular JS, Bootstrap, MAVEN, WebSphere, Eclipse, IBM ODM, JUnit, Mockito, JQuery, log4j, Mongo DB, Windows, SOAP UI, REST Client, AWS, Git.

Confidential, Boston, MA

Java Developer

Responsibilities:

• Gathered requirements, designed and implemented the application using Java/J2EE technologies.
• As technical team lead for a 4-member team ensuring that Software Development Life Cycle is in place with project deadlines and delivers high quality code.
• Implemented Hibernate and spring to replace EJBs and Struts respectively in the earlier architecture.
• Involved in designing ER diagrams using Erwin Data Model Tool.
• Involved in Container/System maintenance which involves build, deployment, configuration of servers, troubleshooting on the server issues, analyzing the application server performance tool WILY logs, identifying the bottlenecks in the system and fine tune them with appropriate directions and performance improvements.
• Involved in preparing use-case diagrams, sequence diagrams and class diagrams using Rational Rose, UML.
• Involved in writing complex SQL queries, Stored Procedures in PL/SQL to access the data from Oracle database.
• Played a pivotal role in identifying the bottlenecks and possibilities of running into issues with the change requirements and proposed implementation of the project modules. Take appropriate action to save cost for the clients.
• Involved on 24X7 production support.
• To provide suggestions for various development and enhancement projects in FDR.

Environment: Core Java, JSP, Servlet, EJB, JSF, Spring Framework, Struts, Web Services, XML, ANT, Hibernate, Oracle 10g, HTML, JavaScript, Shell Script, Sun Solaris10, Rational Rose, Rational Clear case, WebSphere V6.1 and Sun One web server.

Confidential

Software Developer

Responsibilities:

• Gathering requirements for data mapping rules from existing system to new system.
• Interacted with designers for preparing the detailed design and UML diagrams, delegated and communicated effectively with the team members and the QA team
• Extracted data from various distributed databases and loaded into Oracle Database on a monthly basis using SQL*LOADER and external table.
• Created PL/SQL stored procedures and functions in order to validate various new orders, it included deploying the entire packages using exceptions, constraints, auditing tools.
• Created and scheduled numerous shell scripts useful for loading files into Oracle and also involved in automating the file transfer process, report generation process
• Involved in the data extraction and data conversion of fixed format and CSV files using external tables
• Developed processing logic for data validation procedures and functions for verifying and uploading into respective schemas
• Involved with DBAs in creation of tablespaces, users, privileges, indexes, constraints, triggers, synonyms, roles, partitioning etc.
• Wrote anonymous PL/SQL blocks to BULK COLLECT large data and assist in data migration between multiple databases.
• Wrote complex SQL queries using joins, subqueries and correlated subqueries
• Documented test plans, test cases, test scripts, and validations based on design specifications for unit testing, system testing, functional testing, regression testing, prepared test data for testing, error handling and analysis.
• Performance tuning of Data Stage jobs and PL/SQL scripts to handle millions of data.
• Engaged in Unit testing and Peer reviews of codes.

Environment: ORACLE 10g/9i, MySQL 5.0, Solaris 9, Toad, Oracle, UNIX.