SUMMARY

• Hadoop Developer with over all 7 years of IT experience in the field of Big Data with strong JAVA background.
• Widely worked on Hadoop Distributed File System, Parallel processing systems which includes Map Reduce, Hive, pig, Scoop, Oozie and flume.
• Experience working on Cloudera, MapR and Amazon Web Services(AWS).
• Implemented various use cases through Map Reduce jobs.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems/ Non - Relational Database Systems.
• Experience in designing job/workflow, scheduling jobs through Oozie and Zookeeper.
• Experience in Managing scalable Hadoop clusters including Cluster designing, provisioning, custom configurations, monitoring and maintaining using different Hadoop distributions: Cloudera CDH, Apache Hadoop.
• Experience in building the CI system with Databricks, GitHub, Jenkins, and AWS.
• Excellent understanding of Apache Crunch to develop data pipelines that ingests data from multiple data sources and process them.
• Worked on Pig for ETL Transformations and optimized Hive Queries.
• Experience in installation and setup of various Kafka producers and consumers along with the Kafka brokers and topics.
• Expertise in Implementing SOLR index cron jobs.
• Good at using version control tools like GITHUB and SVN.
• Experience with NoSQL databases like HBase and Cassandra.
• Knowledge on reading and writing data to Cassandra.
• Collaborating with business users/product owners/developers to contribute to the analysis of functional requirements.
• Working Knowledge in architecting Hadoop solutions including hardware recommendations, network topology design, storage configurations, benchmarking, performance tuning, administration and support.
• Expertise in designing and creating various analytical reports and Automated Dashboards to help users to identify critical KPIs and facilitate strategic planning in the organization.
• Strong understanding of Data warehouse concepts, ETL, data modeling experience using Normalization, Business Process Analysis, Reengineering, Dimensional Data modeling, physical & logical data modeling.
• Experience in working with different relational databases like MySQL, MS SQL and Oracle.
• Experience in Database design, Entity relationships, Database analysis, Programming SQL, Stored procedure’s PL/ SQL, Packages and Triggers in Oracle and SQL Server on Windows and LINUX.
• Detailed knowledge and experience of Design, Development and Testing Software solutions using Java and J2EE technologies.
• Expertise in various faces of Software Development including analysis, design, development and deployment of applications using Servlets, JSP, Java Beans, EJB, JSTL, JMS, Struts, Spring Framework, JSF, JDBC, JNDI and Hibernate.
• Expertise in developing and maintaining the Web Applications using the Web Server Tomcat.
• Experience with front end technologies like HTML, CSS and Javascript.
• Strong analytical skills with ability to quickly understand client business needs.
• Involved in meetings to gather information and requirements from the clients. Leading the Team and involved in Onsite, Offshore co-ordination.
• Very active participant in Hadoop user groups and Strata Big data conferences.
• Good knowledge in evaluating big data analytics libraries (MLlib) and use of Spark-SQL for data exploratory.
• Responsible for developing data pipeline using HDInsight.
• Knowledge on handling Hive queries using Spark SQL that integrate with Spark environment.
• Experience in Developing Shell scripts to automatically compile, package, deploy and test J2EE applications
• Good experience in Spark and its related technologies like SparkSQL, Spark Streaming.
• Experienced in working with spark eco system using Sparksql and Scala queries on different formats like Text file, CSV file.
• Written shell scripts for automation of job.

TECHNICAL SKILLS

Big Data Technologies: Hadoop, HDFS, Map Reduce, Hive, Pig, Impala, Sqoop, Kafka, Flume, Zookeeper, Cloudera, Spark, Oozie.

NOSQL Databases: Hbase, Cassandra

BI Tools: Tableau, Base SAS, SAS Enterprise Guide, SAS Enterprise Miner, IBM Cognos

Programming Languages: Java, C, C++, Python, Shell scripting

Web Technologies: HTML, J2EE, CSS, JavaScript, AJAX, Servlets, JSP, DOM, XML, XSLT, XPATH

Java Framework: Struts, Spring, Hibernate

Databases: MySQL, SQL, Oracle, SQL Server, Microsoft Excel

Software Engineering: UML, Object Oriented Methodologies, Scrum and Agile methodologies

Operating System: Linux, Windows 7, Windows 8, XP, windows vista

Work Environments: Eclipse, VisualStudio .NET, JUnit, Log4j, Putty

PROFESSIONAL EXPERIENCE

Confidential, Richardson, TX

Sr. Hadoop Developer

Responsibilities:

• Installed, configured and maintained Apache Hadoop cluster for application development and Hadoop echo system like Hive, Pig, HBase, Zookeeper and Sqoop.
• Developed multiple MapReduce jobs in java for data cleaning.
• Worked on installing cluster, commissioning & decommissioning of DataNodes, NameNode recovery, capacity planning, and slots configuration.
• Implemented NameNode backup using NFS for High availability.
• Used Pig as ETL tool to do transformations, event joins and some pre-aggregations before storing the data onto HDFS.
• Responsible for developing data pipeline using flume, Sqoop and pig to extract the data from weblogs and store in HDFS.
• Used GIT as data repository.
• Installed Oozie workflow engine to run multiple Hive and Pig Jobs.
• Created Hive tables and involved in data loading and writing Hive UDFs.
• Experience in optimization of Map reduce algorithm using combiners and partitions to deliver the best results and worked on Application performance optimization for a HDFS/Cassandra cluster.
• Responsible for coding using Cassandra driver.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports.
• Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
• Worked on HBase, Cassandra.
• Automated workflows using shell scripts to pull data from various databases into Hadoop.
• Responsible for creating a Solr schema from the Indexer settings
• Written SOLR queries for various search documents.
• Involved in loading data from Linux file systems, servers, java web services using Kafka producers and consumers.
• Deployed Hadoop Cluster in Fully Distributed and Pseudo-distributed modes.
• Shell scripting for analyzing Sqoop/hive/spark logs and auditing the results and load data into hive Table.
• Expert in implementing advanced procedures like text analytics and processing using the in-memory computing capabilities like Spark written in Scala.
• Developed shell scripts for automating monitoring services for hadoop cluster.
• Involved in converting Hive/Sql queries into Spark transformations using Spark RDD’s.

Environment: Hadoop, MapReduce, Hive, HDFS, PIG, Sqoop, Oozie, Solr, Apache, Kafka, Spark, Shell scripting, Scala, HBase, Zookeeper, Cassandra, Git and Unix/Linux.

Confidential, Kansas City, MO

Hadoop Developer

Responsibilities:

• Worked on Big Data Hadoop cluster implementation and data integration in developing large-scale system software.
• Installed and configured MapReduce, HIVE and the HDFS; implemented CDH3 Hadoop cluster on Centos. Assisted with performance tuning and monitoring.
• Involved in the Mapr5.1 upgrade installation and configuration of a Hadoop cluster.
• Assessed existing and EDW technologies and methods to ensure our EDW/BI architecture meet the needs of the business and enterprise and allows for business growth.
• Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the EDW.
• Capturing data from existing databases that provide SQL interfaces using Sqoop.
• Worked extensively with Sqoop for importing and exporting the data from HDFS to Relational Database systems/mainframe. Loading data into HDFS.
• Develop and maintains complex outbound notification applications that run on custom architectures, using diverse technologies including Core Java, J2EE, SOAP, XML, JMS, JBoss and Web Services.
• Created Hive queries that helped market analysts spot emerging trends by comparing fresh data with EDW reference tables and historical metrics.
• Experience in writing storm topology to accept events from Kafka producer and emit to Cassandra.
• Enabled speedy reviews and first mover advantages by using Oozie to automate data loading into the Hadoop Distributed File System and PIG to pre-process the data.
• Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems.
• Managed and reviewed Hadoop log files.
• Involved in developing code to write canonical model JSON records from numerous input sources to Kafka Queues.
• Tested raw data and executed performance scripts.
• Shared responsibility for administration of Hadoop, Hive and Pig.
• Developed Hive queries for the analysts.
• Helped business processes by developing, installing and configuring Hadoop ecosystem components that moved data from individual servers to HDFS.
• Created Cassandra tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
• Supported code/design analysis, strategy development and project planning.
• Developed multiple MapReduce jobs in Java for data cleaning and preprocessing.
• Assisted with data capacity planning and node forecasting.
• Collaborated with the infrastructure, network, database, application and BI teams to ensure data quality and availability.
• Involved in writing queries in Sparksql using Scala.
• Administrator for Pig, Hive and Cassandra installing updates, patches and upgrades.
• Handling structured and unstructured data and applying ETL processes.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.
• Coding complex Oracle stored procedures, functions, packages, and cursors for the client specific applications.
• Production Rollout Support and resolving any issues that are discovered by the client and client services teams.
• Expertized in implementing Spark using Scala and Sparksql for faster testing and processing of data responsible to manage data from different sources.

Environment: Hadoop, MapReduce, HDFS, Hive, Cassandra, Java (jdk1.6), MapR, Kafka, IBM DataStage 8.1(Designer, Director, Administrator), PL/SQL, Pig, Scala, Oozie, Spark, UNIX Shell Scripting.

Confidential, Minnetonka, MN

Senior J2EE & Hadoop Developer

Responsibilities:

• Involved in designing and developingHadoop MapReduce jobs Using JAVA for the batchprocessingto search and match the scores.
• Used Rational Rose for developing Use case diagrams, Activity flow diagrams, Class diagrams and Object diagrams in the design phase.
• Used Struts with Tiles in the MVC framework for the application.
• Extensively worked on Servlets, JSPs, Struts 1.3 and Tiles, JavaScript, Expression language, JSTL, JSP custom tags.
• Used Sqoop to import and export data from HDFS to RDBMS.
• Involved in using Solr Cloud implementation to provide real time search capabilities on the repository with terabytes of data.
• Involved in developing Hadoop MapReduce jobs for merging and appending therepository data.
• Hands on experience insetting up Hbase Column based storage repositoryforarchivingand retro data.
• Developed XML schema and DOM parser for all the XML document used for Data Transfers and developed XSLT code for them.
• Configured Struts-config.xml, ejb-jar.xml and web.xml on the application.
• Used JBoss Application Server 4.0 as application server and Harvest for build and deployment.
• Used ANT as build-tool for building J2EE applications.
• Used Hibernate 3.0 in data access layer to access and update information in the database.
• Used Java Message Service (JMS 1.1) for reliable and asynchronous exchange of important
• Involved in agile SCRUM methodology implementation.
• Involved in various performance projects to increase the response time of the application.
• Involved in development of batch processing application using Multi-threaded executorpools for faster processing.
• Responsible for writing Pig UDFs and Hive UDFs.
• Handled importing of data from various data sources, performance transformation using Hive.
• Created various calculated fields and created various visualizations and dashboards using tableau desktop.
• Published the dashboards created on Tableau desktop onto Tableau server.
• Used Log4j for logging and debugging and used JUnit extensively for testing.
• Handling the scalability tool for the framework.
• Code refactoring to optimize the calls to various system components.
• Experience working with off-shore teams and communicating daily status on issues, road-blocks.

Environment: Java, J2EE, Tableau Desktop, Tableau Server, Hadoop, Hbase, Solr cloud, Pig Latin, JavaBeans, Agile SCRUM, Struts1.3, Hibernate3.0, Jboss Application Server, Eclipse, Rational Clear case, CXF 2.2.4, Java Script, JUnit, Jboss, XML Web services, ANT, UML, Unix, Windows NT/2000.

Confidential

JAVA Developer

Responsibilities:

• Involved in the analysis, design, and development and testing phases of Software Development Life Cycle (SDLC).
• Analysis, design and development of Application based onJ2EEusingStruts and Tiles,Spring 2.0 andHibernate 3.0.
• Involved in interacting with the Business Analyst and Architect during the Sprint Planning Sessions.
• Used XML Web Services for transferring data between different applications.
• Used Apache CXF web service stack for developing web services and SOAP UI and XML-SPY for testing web services.
• Used JaxB for binding XML to Java. Used SAX and DOM parsers to parse xml data. Used Xpath to parse XML documents.
• Hibernate was used forObject Relational mappingwith Oracle database.
• Worked with Spring IOCfor injecting the beans and reduced the coupling between the classes.
• Involved in developing the user interface usingStruts.
• Implemented Spring IOC (Inversion of Control)/DI (Dependency Injection)for wiring the object dependencies across the application.
• Integratedspring and Hibernate ORMframework for persistence and usedHibernateDaoSupport with Hibernate Template to access the data.
• Implemented spring transactionmanagement for implementing transactions for the application.
• Implemented design patterns for Service Locator.
• Performed unit testing usingJunit 3, EasyMock Testing Framework for performing Unit testing.
• Worked onPL/SQLstored procedures usingPL/SQL Developer.
• Involved in Fixing theproductionDefects for the application.
• Used ANT as build-tool for building J2EE applications.

Environment: Java 1.6, Struts, PL/SQL, Spring IOC, Spring Transaction Management, Hibernate 3.0, Springs2.0, JSP 2.0, Oracle 11g, Eclipse, JUnit 3, PL/SQL Developer, Application Server, JDBC, ANT, Maven, UML Struts 1.2.3, XML Web Services.

Confidential

Software Engineer

Responsibilities:

• Involved in the Design, Coding, Testing and Implementation of the web application.
• Developed JSP Java Server Pages starting from HTMLs and detailed technical design specification documents. Pages included HTML, CSS, JavaScript, Hibernate and JSTL.
• Developed SOAP based requests for communicating with Web Services.
• Used agile systems and strategies to provide quick and feasible solutions to the organization.
• Implemented HTTP Modules for different applications in Struts Framework that uses Servlets, JSP, ActionForm, ActionClass and ActionMapping.
• Developing web applications using MVC Framework, Spring, Struts, Hibernate.
• Involved in the creation of custom interceptors for Validation purposes.
• Analyzed and fixed defects in the Login application.
• Involved in configuration and deployment of application on the JBoss Application.
• Involved in dynamic creation of error elements on demand when there is an error.
• Involved in Ajax - based Rich Browser User Interfaces.
• Ensured design consistency with client’s development standards and guidelines.
• Improved user experience by designing and creating new web components and features.

Environment: Java, J2EE, Struts, SOAP web services, SOA, Spring, Hibernate, JavaScript, jQuery, JBoss Application Server, Oracle, AJAX, JSP, Servlets, Eclipse, CVS Source control, Linux.

Confidential

Jr. Developer

Responsibilities:

• Involved in understanding the functional specifications of the project.
• Assisted the development team in designing the complete application architecture
• Involved in developing JSP pages for the web tier and validating the client data using JavaScript.
• Developed connection components using JDBC.
• Designed Screens using HTML and images.
• Cascading Style Sheet (CSS) was used to maintain uniform look across different pages.
• Involved in creating Unit Test plans and executing the same.
• Deployed web modules in Tomcat web server.

Environment: Java, JSP, J2EE, Servlets, Java Beans, HTML, JavaScript, JDeveloper, Tomcat WebServer, Oracle, JDBC, XML.