PROFESSIONAL SUMMARY

• 9+ years of IT experience in analysis, design and development using Hadoop, Java and J2EE.
• 3 plus years of experience with Hadoop, HDFS, MapReduce and Hadoop Ecosystem including Pig & Hive.
• Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm.
• Knowledge of Data Analytics and Business Analytics processes.
• Real time streaming the data usingSparkwith Kafka.
• Hands on experience withSparkstreaming to receive real time data using Kafka.
• Experience in ingesting streaming data intohadoopusingSpark, Storm Framework and Scala.
• Hands on experience in installing, configuring and deployment of Hadoop ecosystem components like Hadoop ap Reduce, YARN, HDFS, HBase, Oozie, Hive, Pig, Impala and Spark, Storm, Kafka, Tableau, Sqoop, Pig, HCatalog, Zoo Keeper, Amazon Web Services and Flume.
• CreatingSparkSQL queries for faster requests.
• Experience in analyzing data using HiveQL, Pig Latin, HBase and custom Map Reduce programs in Java
• Continuous delivery/Continuous Integration (CD/CI) using Jenkins/Cloud Beens and hostage of QC.
• Experience in building, maintaining multiple Hadoop clusters of different sizes and configuration and setting up the rack topology for large clusters also in Hadoop Administration/Architecture/Developer with multiple distributions like Horton Works & Cloudera.
• Experienced with test frameworks for Hadoop using MRUnit.
• Performed data analytics using PIG, Hive, Language R for Data Scientists within the team.
• Worked extensively on Data Visualization tool Tableau, Graph DataBase like Neo4J.
• Worked on 32+ node Apache/Cloudera 4.3.2 Hadoop cluster for PROD Environment and used tools like sqoop, Flume for data ingestion from different sources to Hadoop system and Hive/Sparksql to generate reports for analysis.
• Configured Splunk to perform the web analytics.
• Good technical Skills in Oracle 11i, SQL Server, ETL Development using Informatica Qlikview, Cognos, SAS.
• Experience in managing and reviewing Hadoop log files.
• Experience in analyzing data using HiveQL, Pig Latin, HBase and custom Map Reduce programs in Java.
• Responsible for smooth error - free configuration of DWH-ETL solution and Integration with Hadoop.
• Extending Hive and Pig core functionality by writing custom UDFs.
• Worked on multiple stages of Software Development Life Cycle including Development, Component Integration, Performance Testing, Deployment and Support Maintenance.
• Worked extensively with Data migration, Data cleansing, Data profiling, and ETL Processes features for data warehouses.
• Extensive experience in middle-tier development using J2EE technologies like JDBC, JNDI, JSP, Servlets, JSP, JSF, Struts, Spring, Hibernate, and Web Services.

TECHNICAL SKILLS

Hadoop/Big Data: HDFS, Mapreduce, HBase, Pig, Hive, Sqoop, Flume, Cassandra, Impala, Oozie, Zookeeper, MapR, Amazon Web Serivces, EMR, MRUnit, Spark, Storm, Greenplum, Datameer, Language R, Ignite.

Java & J2EE Technologies: Core Java, JDBC, Servlets, JSP, JNDI, Struts, Spring, Hibernate and Web Services (SOAP and Restful)

IDE s: Eclipse, Net beans, MyEclipse, IntelliJ

Frameworks: MVC, Struts, Hibernate, Spring

Programming languages: C,C++, Java, Python, Ant scripts, Linux shell scripts, R

Databases: Oracle 11g/10g/9i, MySQL, DB2, MS - SQL Server, MongoDB, Couch DB. Graph DB

Web Servers: Web Logic, Web Sphere, Apache Tomcat

Web Technologies: HTML, XML, JavaScript, AJAX, SOAP, WSDL, Restful WS

Network Protocols: TCP/IP, UDP, HTTP, DNS, DHCP

ETL Tools: Informatica, IBM Infosphere, Qlikview and Cognos

PROFESSIONAL EXPERIENCE

Confidential, Dallas, TX

Sr. Hadoop Developer

Responsibilities:

• Worked on evaluation and analysis of Hadoop cluster and different big data analytic tools including Pig, Hbase database and Sqoop.
• Continuous delivery/Continuous Integration (CD/CI) using Jenkins/CloudBeesand hostage of QC App.
• Hands on experience withSparkstreaming to receive real time data using Kafka.
• Ingesting streaming data intohadoopusingSpark, Storm Framework and Scala.
• CreatingSparkSQL queries for faster requests
• Hands on experience withSparkstreaming to receive real time data using Kafka
• Conducted information sharing and teaching sessions to facilitate increased awareness of industry trends and upcoming initiatives by ensuring compliance between business strategies and goals and solution architecture designs
• Implemented Cloudera Manager on existing cluster
• Configured TLS security for Cloudera Manager and configured Hadoop security for CDH 5 using Kerberos through Cloudera Manager
• Performance tuned the application at various layers - MR, HIVE, CDH, Oracle
• Used Spark streaming for the real time processing of the data from HGFS.
• Used Qlikview to create visual interface of the real time data processing.
• Implemented partitioning, dynamic partitioning and bucketing in hive
• Imported, exported data from various databases NETEZZA, ORACLE, and MYSQL into HDFS.
• Implemented Pub/Sub model using Apache Kafka for real-time transactions to load in HDFS.
• Automated the process from pulling the data from data sources to Hadoop and exporting the data in the form of Jason files in to specified location.
• Migrated the Hive queries to Impala
• Worked on various file formats Avro, SerDe, Parquet,JSON and Text by using snappy compression.
• Created analysis batch job prototypes using Hadoop, Pig, Oozie, Hue and Hive
• Designed, documented operational problems by following standards and procedures using a software reporting tool JIRA.

Environment: Hadoop, HDFS, Map Reduce, Spark, SOLR, Hive, Impala, Pig, Sqoop, Java, UNIX shell scripting, Oracle, Netezza, MySql, Qlikview

Confidential, Pittsburg, PA

Hadoop Developer

Responsibilities:

• Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database and Sqoop.
• Responsible for building scalable distributed data solutions using Hadoop.
• Implemented nine nodes CDH3 Hadoop cluster on Red hat LINUX.
• Involved in loading data from LINUX file system to HDFS.
• Worked on installing cluster, commissioning & decommissioning of data node, name node recovery, capacity planning, and slots configuration.
• Developed performance utilization charts, optimized and tuned SQL and designed physical databases.
• Assisted developers with Teradata load utilities and SQL.
• Researched Sources and identified necessary Business Components for Analysis.
• Gathered the required information from the users.
• Interacted with different system groups for analysis of systems.
• Created tables, views in Teradata, according to the requirements.
• Created HBase tables to store variable data formats of PII data coming from different portfolios.
• Implemented a script to transmit sysprin information from Oracle to Hbase using Sqoop.
• Implemented best income logic using Pig scripts and UDFs.
• Implemented test scripts to support test driven development and continuous integration.
• Worked on tuning the performance Pig queries.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Responsible to manage data coming from different sources.
• Involved in loading data from UNIX file system to HDFS.
• Load and transform large sets of structured, semi structured and unstructured data
• Cluster coordination services through Zookeeper.
• Experience in managing and reviewing Hadoop log files.
• Job management using Fair scheduler.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and
• Troubleshooting, manage and review data backups, manage and review Hadoop log files.
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Experience with Re-engineered customer account software systems used by brokerage teams. Web developer for user interfaces to trading inquiries, support parallel systems.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.

Environment: MapReduce, Java (jdk1.6), Flat files, Oracle 11g/10g, Netezza, UNIX, Sqoop, Hive, Oozie.

Confidential, Oaks, PA

Hadoop/Consultant

Responsibilities:

• Worked extensively on importing data using scoop and flume.
• Responsible for creating complex tables using hive.
• Created partitioned tables in Hive for best performance and faster querying.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS.
• Experience with professional software engineering practices and best practices for the full software development life cycle including coding standards, code reviews, source control management and build processes.
• Worked collaboratively with all levels of business stakeholders to architect, implement and test Big Data based analytical solution from disparate sources
• Involved in source system analysis, data analysis, data modeling to ETL (Extract, Transform and Load)
• Written multiple MapReduce procedures to power data for extraction, transformation and aggregation from multiple file formats including XML, JSON, CSV & other compressed file formats.
• Handling structured and unstructured data and applying ETL processes.
• Develop Hive queries for the analysts
• Prepare Developer (Unit) Test cases and execute Developer Testing.
• Create/Modify shell scripts for scheduling various data cleansing scripts and ETL loading process.
• Supports and assist QA Engineers in understanding, testing and troubleshooting.
• Written build scripts using ant and participated in the deployment of one or more production systems
• Production Rollout Support that includes monitoring the solution post go-live and resolving any issues that are discovered by the client and client services teams.
• Designed, documented operational problems by following standards and procedures using a software-reporting tool JIRA.

Environment: MapReduce, Java (jdk1.6), Flat files, Oracle 11g/10g, Netezza, UNIX, Sqoop, Hive, Oozie.

Confidential, Newton Upper Falls, MA

Hadoop Developer

Responsibilities:

• Involved in collecting business requirements from the Business partners and subject Matter Experts.
• Responsible to manage data coming from different data sources.
• Developed Map Reduce programs to perform data filtering for unstructured data.
• Designed and developed UDF’S to extend the functionality in both PIG and HIVE.
• Wrote Hive queries for data analysis to meet the business requirements.
• Created Partitioned Hive tables and worked on them using Hive.
• Import and Export of data using Sqoop between MySQL to HDFS on regular basis.
• Used Flume to channel data from different sources to HDFS.
• Created HBase tables to store data depending on column families.
• Worked with administrator to set up and monitor the Hadoop cluster
• Supported Map Reduce Programs which are running on the cluster.
• Involved in HDFS maintenance and loading of structured and unstructured data.
• Developed scripts and batch jobs to schedule various Hadoop programs.
• Designed and Maintained Oozie workflows to manage the flow of jobs in the cluster.
• Actively updated the upper management with daily updates on the progress of project that include the classification levels that were achieved on the data.
• Environment: Java, Hadoop, Map Reduce, HDFS, Pig, Hive, HBase Linux, MySQL, Ubuntu.

Confidential

Sr. Java Developer

Responsibilities:

• Designed the application by implementing Struts Framework based on MVC Architecture.
• Designed and developed the front end using JSP, HTML and JavaScript and JQuery.
• Developed framework for data processing using Design patterns, Java, XML.
• Developed custom Tags in Struts.
• Used the light weight container of the Spring Framework to provide architectural flexibility for Inversion of Controller (IOC).
• Used Spring IOC for dependency injection to Hibernate and Spring Frameworks.
• Designed and developed Session beans to implement the Business logic.
• Developed EJB components that are deployed on Web logic Application Server.
• Written unit tests using Junit Framework and Logging is done using Log4J Framework.
• Designed and developed various configuration files for Hibernate mappings.
• Designed and Developed SQL queries and Stored Procedures.
• Developed Web Services for sending and getting data from different applications using SOAP messages.
• Actively involved in code reviews and bug fixing.
• Applied CSS (Cascading style Sheets) for entire site for standardization of the site.
• Offshore co-ordination and User acceptance testing support.
• Environment: Java 5.0, Struts, Spring 2.0, Hibernate 3.2, Web Logic 7.0, Eclipse 3.3, Oracle 10g, Junit 4.2,Maven, Windows XP, HTML, CSS, JavaScript, and XML.

Confidential

Java Developer

Responsibilities:

• Involved in the analysis & design of the application using Rational Rose.
• Object Oriented Analysis and Design using UML include development of class diagrams, Sequence diagrams and State diagrams and implemented these diagrams in Microsoft Visio.
• Developed the various action classes to handle the requests and responses.
• Designed and created Java Objects, JSP pages, JSF, JavaBeans and Servlets to achieve various business functionalities. Created validation methods using JavaScript and Backing Beans.
• Involved in writing client side validations using JavaScript, CSS.
• Involved in the design of the Referential Data Service module to interface with various databases using JDBC.
• Used Hibernate framework to persist the employee work hours to the database.
• Spring framework features were extensively used.
• Developed and configured using BEA Web Logic Application Server.
• Developed the build scripts using Ant.
• Involved in designing test plans, test cases and overall Unit testing of the system.
• Developed controllers and actions encapsulating the business logic.
• Developed classes and interface with underlying web services layer.
• Designed web services for the above modules.
• Prepared documentation and participated in preparing user's manual for the application.

Environment: Java, Rational Rose, Spring 2.0, Hibernate 3.2, Web Logic, Eclipse, SQL Server 2008, Junit 4.2,Ant, Windows XP, HTML, CSS, JavaScript, and XML.

Confidential

Jr. Software engineer

Responsibilities:

• Offshore co-ordination and User acceptance testing support.
• Involved in gathering user requirements and followed by analysis and design.
• Involved in Developing JSP's, action classes, form beans, response beans, EJB's.
• Developed Struts configuration files and action classes the Transactional Model to handle many requests.
• Developed Struts configuration files and action classes the Transactional Model to handle many requests. Offshore co-ordination and User acceptance testing support.
• Involved in gathering user requirements and followed by analysis and design.
• Involved in Developing JSP's, action classes, form beans, response beans, EJB's.
• Developed Struts configuration files and action classes the Transactional Model to handle many requests.
• Participated in developing helper classes which interact with middleware stubs.
• Extensively used XML to code configuration files.
• Developed PL/SQL stored procedures, triggers

Environment: Java, JSP, Struts, SQL, PL/DSQL, Eclipse, Oracle, Windows XP, HTML, CSS, JavaScript, and XML.