SUMMARY

• Hadoop/Java Developer with 7+ years of overall IT experience in a variety of industries, which includes hands on experience in Big Data technologies
• 5+ years of comprehensive experience in Big Data processing using Apache Hadoop, Adminand its ecosystem (MapReduce, Pig, Hive, Sqoop, Flume and HBase), Spark and Zipkins.
• Provided key oversight throughout teh development lifecycle, with involvement in requirements gathering and analysis, application review.
• Experience in installing, configuring and maintaining teh HadoopCluster
• Installed and configured Hive, Pig, Sqoop, Flume and Oozie on theHadoopcluster.
• Knowledge of administrative tasks such as installing Hadoop (on Ubuntu) and its ecosystem components such as Hive, Pig, sqoop.
• Involved in HDFS maintenance and administering it throughHadoop - Java API.
• Worked with application team via scrum to provide operational support, installHadoopupdates, patches and version upgrades as required.
• Hands on experience withSpark-Scala programming.
• Expertise in writing Hadoop Jobs for analyzing data using Hive QL (Queries), Pig Latin (Data flow language), and custom MapReduce programs in Java.
• Wrote Hive queries for data analysis to meet teh requirements
• Created Hive tables to store data into HDFS and processed data using Hive QL
• Developed Pig Latin scripts using operators such as LOAD, STORE, DUMP, FILTER, DISTINCT, FOREACH, GENERATE, GROUP, COGROUP, ORDER, LIMIT, UNION, SPLIT to extract data from data files to load into HDFS.
• Extending Hive and Pig core functionality by writing custom UDFs
• Experience in working with MapReduce programs using Apache Hadoop for working with Big Data
• Responsible for continuous monitoring and managingElasticMapReduce cluster through AWS console.
• Experience with ETL - Extract Transform and Load - Talend Open Studio, Informatica.
• Good knowledge in Linux shell scripting or shell commands, Python and WLST.
• Expertise in designing python scripts to interact with middleware/back end services.
• Experience in using IDEs like Eclipse, IntelliJ.
• Working knowledge on Scala.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa
• Hands on experience in configuring and working with Flume to load teh data from multiple sources directly into HDFS
• Hands on experience in provisioning and managingHadoopclusters on publiccloudenvironment open stackcloudplatform.
• Extensive experience with SQL, PL/SQL and database concepts
• Knowledge of NoSQL databases such as Hbase, Cassandra.
• Knowledge of job workflow scheduling and monitoring tools like Oozie (hive, pig) and Zookeeper (Hbase).
• Have very good experience in exporting, extracting of analyzed data and generating various visualizations using Business Intelligence toolTableaufor better analysis of data.
• Good understanding of XML methodologies (XML, XSL, XSD) including Web Services like REST, SOAP.
• UsedApachecamelto integrate different applications. DevelopedCamelOrchestration layers to integrate different components of teh application.
• Strong experience as a senior Java Developerin Web/intranet, Client/Server technologies using Java, J2EE, Servlets, JSP, EJB, JDBC.
• Good Inter personnel skills and ability to work as part of a team. Exceptional ability to learn and master new technologies and to deliver outputs in short deadlines

TECHNICAL SKILLS

Hadoop/Big Data: HDFS, Mapreduce, HBase, Pig, Hive, Sqoop, Flume, MongoDB, Cassandra, Oozie, Zookeeper, Kafka, Spark

Java & J2EE Technologies: Core Java, Servlets, JSP, JDBC, Java Beans, Apache Camel

IDE’s: Eclipse, IntelliJ, JBoss Developer Studio

Frameworks: MVC, Struts, Hibernate, Spring

Programming languages: C, C++, Java, Python, Linux shell scripts, Scala 2.0

Databases: Oracle 11g/10g/9i, MySQL, DB2, MS-SQL Server

Web Servers: Web Logic, Web Sphere, Apache Tomcat

Web Technologies: HTML, XML, JavaScript, AJAX, REST, SOAP, WSDL

ETL Tools: Informatica, Talend

PROFESSIONAL EXPERIENCE

Confidential, Kansas city, MO

Hadoop Developer

Responsibilities:

• Worked with Sqoop for importing and exporting teh data from HDFS to Relational Database systems and vice-versa.
• Created Hive tables (external, internal) with static and dynamic partitions and performed bucketing on teh tables to provide efficiency.
• Used Hive QL to analyze teh partitioned and bucketed data and compute various metrics for reporting.
• Used Impala and Written Queries for fetching Data from Hive tables.
• Used Spark-SQL to Load JSON data and create SchemaRDD and loaded it into Hive Tables and handled Structured data using Spark SQL.
• Involved in creating external tables in Hive and also loading them using Sqoop.
• Loading teh data into hive tables for blocking and matching.
• Involved in creating dynamic Oozie workflow using Spark.
• Utilized SparkSQL to extract and process data by parsing using Datasets or RDDs in HiveContext, with transformations and actions (map, flatMap, filter, reduce, reduceByKey).
• Worked with HQL and Criteria API from retrieving teh data elements from database.
• Involved in code walk through, reviewing, testing and bug fixing.
• Involved in UNIT testing.
• Involved in Requirement Gathering to connect with BA.
• Working Closely with BA & vendor for creating technical Documents like High level Design and low Level Design specifications.
• Worked in Agile methodology and used Scrum for Development and tracking teh project.

Environment: Hadoop, HDFS, Spark, Hive, Sqoop, Imapala Cloudera CDH5, Oozie, IntelliJ, Jira, Git.

Confidential, Basking Ridge, NJ

Java/Hadoop Developer

Responsibilities:

• Designing and developing Web applications for Claims Processing using Apache Camel, Spring and Restful web services.
• Developed and maintained applications (web and desktop), coordinating requirements gathering and analysis, application review, data reporting, business logic, and technical specifications.
• Helped to mentor a total of 8 Java/Hadoop Developers, 3 Junior Developers and 5 Senior Developers
• Completed Code Reviews, Reviewed demos for End to end Development solutions and provided expertise helped to implement a new technology stack.
• Worked on deployment, installation, configuration and issues with Application servers like Apache JBoss, and Apache Tomcat 7.
• Worked in Agile Development (Rally) and maintained good level of communication in team and good understanding of Service Oriented Architecture (SOA).
• Developed projects and products using SDLC (Software development life cycle), from initiation, planning, designing, execution and implementation, development.
• Built teh responsive UI screens using HTML, CSS, JavaScript, JSP and Bootstrap framework.
• Experience in developing Applications with Web Services, WSDL, SOAP.
• IntegratedSpring& Hibernate frameworks to develop end to end application.
• Implemented transaction management, high level autantication and authorization, logging and Exception Handling usingSpringSecurity.
• Worked on building web services using Spring and CXF, offering both REST and SOAP interfaces.
• Implemented Apache Camel routing to integrate with different systems and provide end-to-end communications between teh web services and other enterprise services.
• Configured theLDAP, and SiteMinder Technology for Autantication process.
• Experience in performing Unit test by using JUnit.
• Importing and exporting data into HDFS using Sqoop.
• Created PIG Scrips to perform Extract Transform and Load (ETL) operations.
• Involved in creating Hive Tables, loading with data and writing Hive queries.
• Loaded data in elastic search from Datalake and Optimized teh full search function using Elastic Search.
• Implemented distributed tracing by integrating zipkins.

Environment: JAVA/J2EE, JDK 1.7, HTML, CSS, Servlets, JSP, XML, XSLT, WSDL, SOAP, CXF, REST, JUNIT, SOAPUI, SpringBoot Framework, Apache CXF Web services, JBOSS, Apache Camel, Zipkins, Hive, Sqoop, Pig, Git.

Confidential, Chicago, IL

Hadoop developer

Responsibilities:

• UsedSqooptool to extract data from a relational database intoHadoop.
• Worked on HUE interface Environment for querying teh data.
• Analyzed teh web log data using teh HiveQL.
• Installed and configured Hive and also written Hive UDFs.
• Used HiveQL for writing Efficient JOINS on HIVE Tables without degrading Performance.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way
• Worked on JAVA program for connectivity to HBase.
• Used Oozie workflow to automate all jobs for pulling teh data from HDFS and loading data into Hive tables.
• Involved in enhancing teh speed performance using ApacheSpark.
• DevelopedSparkcode using scala andSpark-SQL for batch processing of data.
• Involved in parsing JSON data into structured format and loading into HDFS/Hive usingspark streaming.
• Involved in running Hadoop streaming jobs to generate terabytes of xml format data.
• Used Zookeeper to co-ordinate cluster services.
• Involved in build applications using Maven and integrated with CI servers likeJenkinsto build jobs.
• Used Git to check-in and checkout code changes.
• Maintained, audited and built new clusters for testing purposes using teh cloudera manager.

Environment: Hadoop, HDFS, Spark, Hive, Hbase, Sqoop, Cloudera CDH5, Oozie, IntelliJ, Zookeeper, Jenkins, Chef, Git, Talend.

Confidential, Framingham, MA

Hadoop developer

Responsibilities:

• Installing and configuring fully distributed Hadoop Cluster.
• Installing Hadoop Eco-system Components (Pig, Hive and Hbase).
• Creating and configuring Hadoop cluster in Cloudera.
• InstalledHadoop, Map Reduce, HDFS, AWS and developed multiple MapReduce jobs in PIG and Hive for data cleaning and pre-processing.
• Consulting onHadoopecosystem:Hadoop, Admin, MapReduce, Hbase, Sqoop, amazonElastic Map Reduce (EMR)
• Coordinating and managing relations with vendors, IT developers and end users.
• Managing teh work streams, process and coordinate teh team members and their activities to ensure dat teh technology solutions are in line with teh overall vision and goals.
• Analyzed teh web log data using teh HiveQL.
• Developed custom aggregate functions usingSparkSQL and performed interactive querying.
• Worked on analyzingHadoopcluster and different big data analytic tools including Pig, Hbase NoSQL database and Sqoop.
• Integrated Cassandra Querying Language called CQL for Apache Cassandra.
• Installed and configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in java for data cleaning and preprocessing.
• Developed workflows using custom MapReduce, Pig, Hive, Sqoop.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Written teh Apache PIG scripts to process teh HDFS data and send teh data to Hbase.
• Used Kafka to load data in to HDFS and move data into NoSQL databases.
• Configuration of various database connectivity (Oracle11g, SQL Server 2005)
• Involved in runningHadoopjobs for processing millions of records of text data. Troubleshoot teh build issue during teh Jenkins build process. ImplementDockerto create containers for Tomcat Servers, Jenkins.
• Experience in supporting data analysis projects by using Elastic MapReduce on teh Amazon Web Services (AWS)cloud. Performed Export and import of data into S3.
• Created MapReduce jobs using Hive/Pig Queries.
• Generated various marketing reports usingTableauwithHadoopas a source for data.
• Involving in troubleshooting, performance tuning of reports and resolving issues withinTableau Server and Reports.
• Cluster coordination services through Zoo Keeper.
• Installed and configured Hive and also written Hive UDFs.

Environment: Cassandra, MapReduce, HDFS, Hive, Flume, Cloudera Manager, Sqoop MySQL, UNIX Shell Scripting, Zookeeper, Tableau, Git, Spark, Kafka, Elastic Search, Docker, Ruby on Rails.

Confidential, Chicago, IL

HadoopDeveloper

Responsibilities:

• Responsible for coding MapReduceprogram,Hivequeries, testing and debugging teh MapReduce programs.
• DevelopedPiglatin scripts in teh areas where extensive coding needs to be reduced to analyze large data sets.
• Installed and configured Hive and also implemented various business requirements by writing HIVE UDFs.
• UsedSqooptool to extract data from a relational database intoHadoop.
• Worked closely with data warehouse architect and business intelligence analyst to develop solutions.
• Responsible for performing peer code reviews, troubleshooting issues and maintaining status report.
• Involved in creating Hive Tables, loading with data and writing Hive queries, which will invoke and run MapReduce jobs in teh backend.
• Installed and configured Hadoop cluster in DEV, QA and Production environments.
• Strongly recommended to bring in Elastic Search and was responsible for installing, configuring and administration.
• Performed upgrade to teh existing Hadoop clusters.
• Enabled Kerberos for Hadoop cluster Autantication and integrate with Active Directory for managing users and application groups.
• Implemented Commissioning and Decommissioning of new nodes to existing cluster
• Worked with systems engineering team for planning new Hadoop environment deployments, expansion of existing Hadoop clusters.
• Responsible for data ingestions using Talend.
• Worked on Integration of Big data andcloudplatforms Using Talend.
• Experienced in migrating Hive QL into Impala to minimize query response time.
• Monitoring workload, job performance and capacity planning using Cloudera Manager.
• Worked with application teams to install OS level updates, patches and version upgrades required for Hadoop cluster environments.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig, Hive and Sqoop.

Environment: Hadoop, HDFS, Map Reduce, Hive, Flume, Sqoop, Cloudera CDH4, HBase, Oozie, Pig, Eclipse, Talend.

Confidential

Java/HadoopDeveloper

Responsibilities:

• Worked on analyzing, writing Hadoop MapReduce jobs using Java API, Pig and Hive.
• Responsible for building scalable distributed data solutions using Hadoop.
• Installed Oozie workflow engine to run multiple Hive and Pig jobs.
• Involved in loading data from edge node to HDFS using shell scripting.
• Worked on installing cluster, commissioning & decommissioning of datanode, namenode high availability, capacity planning, and slots configuration.
• Created HBase tables to store variable data formats of PII data coming from different portfolios.
• Worked with NoSQL databases like Hbase in creating Hbase tables to load large sets of semi structured data coming from various sources.
• Implemented test scripts to support test driven development and continuous integration.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Load and transform large sets of structured, semi structured and unstructured data
• Formulated ETL processes (Using Talend) to Extract Data.
• Develop ETL mappings and workflows using Talend for source system’s data extraction and data transformations.
• Experience in managing and reviewing Hadoop log files.
• Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Worked on python scripts to analyze teh data.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.

Environment: Hadoop, HDFS, Pig, Sqoop, HBase, Shell Scripting, Python, Ubuntu, Linux Red Hat, Talend.

Confidential

JAVA Developer

Responsibilities:

• Involved in Analysis, design and coding on J2EE Environment.
• Implemented MVC architecture using Struts, JSP, and EJB's.
• Worked on Hibernate object/relational mapping according to database schema.
• Presentation layer design and programming on HTML, XML, XSL, JSP, JSTL and Ajax.
• Designed, developed and implemented teh business logic required for Security presentation controller.
• Used JSP, Servlet coding under J2EE Environment.
• Designed XML files to implement most of teh wiring need for Hibernate annotations and Struts configurations.
• Responsible for developing teh forms, which contains teh details of teh employees, and generating teh reports and bills.
• Involved in designing of class and dataflow diagrams using UML Rational Rose.
• Used CVS for maintaining teh Source Code Designed, developed and deployed on Apache Tomcat Server.
• Created and modified Stored Procedures, Functions, Triggers and Complex SQL Commands using PL/SQL.
• Involved in teh Design of ERD (Entity Relationship Diagrams) for Relational database.
• Developed Shell scripts in UNIX and procedures using SQL and PL/SQL to process teh data from teh input file and load into teh database.
• Used Core Javaconcepts in application such as multithreaded programming, synchronization of threads used thread wait, notify, join methods etc.
• Creating cross-browser compatible and standards-compliant CSS-based page layouts.
• Involved in maintaining teh records of teh patients visited along with teh prescriptions they were issued in teh Database.
• Performed Unit Testing on teh applications dat are developed.

Environment: Unix (Shell Scripts), Eclipse, Java(jdk1.6), J2EE, JSP1.0, Servlets, Hibernate, JavaScript, JDBC, Oracle 10g, UML, Rational Rose 2000, WebLogic Server, Apache Ivy, JUnit, SQL, PL/SQL, CSS, HTML, XML