SUMMARY

• Certified Cloudera Developer with over 8 years of experience in Designing, Developing, and integrating applications using Hadoop, Hive, PIG, Java, and J2EE technologies.
• Over 3 years experience writing Map - reduce jobs in Java
• Over 3 years experience writing Pig and Hive scripts
• Experience in writing Map Reduce programs using Apache Hadoop for analyzing Big Data.
• Hands on experience in writing Ad-hoc Queries for moving data from HDFS to HIVE and analyzing the data using HIVE QL.
• In depth knowledge of Hadoop Architecture and Hadoop daemons such as Name Node, Secondary Name Node, Data Node, Job Tracker and Task Tracker.
• In depth knowledge of analyzing data using Hive and Pig.
• Expertise in developing UDFs to extend the Hive and PIG functionality using Java.
• Experience in importing and exporting data using Sqoop from Relational Database Systems to HDFS.
• Experience in extracting data from both Relational systems and Flat Files.
• Experience in designing and implementing of secureHadoopcluster using Kerberos.
• Experience in performing data validation using HIVE dynamic partitioning and bucketing.
• Extensive experience with SQL, PL/SQL and database concepts.
• Loaded streaming log data from various web servers into HDFS using Flume.
• Good Knowledge in Amazon AWS concepts like EMR and EC2 web services which provides fast and efficient processing of Big Data.
• Hands on experience in installing, configuring and using Apache Hadoop ecosystems such as Map Reduce, HIVE, PIG, SQOOP, FLUME, STORM, KAFKA and OOZIE.
• Experience loading data to Hive partitions and creating buckets in Hive.
• Logical Implementation and interaction with HBase.
• Worked in Multiple Environment in installation and configuration.
• Working knowledge in KAFKA messaging Framework.
• Working knowledge in Python.
• Strong understanding of NoSQL databases like HBase, MongoDB & Cassandra.
• Experience with handling complex data processing workflows using Oozie.
• Expertise in scheduling all Hadoop/hive/Sqoop/Map Reduce jobs using Oozie.
• Extensive Knowledge in Java, J2ee, Servlets, JSP, JDBC, Struts and Spring Framework.
• Background with traditional databases such as Oracle, SQL Server, Teradata and ETL tools / processes.
• Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
• Fluent with the core Java concepts like I/O, Multi-threading, Exceptions, Regular Expressions, Collections, Data-structures and Serialization.
• Expert in setting up SSH, SCP, SFTP connectivity between UNIX hosts.
• Experience in using version control management tools like CVS, SVN and Rational Clear Case.
• Good understanding of Scrum methodologies, Test Driven Development and continuous integration.
• Strong technical and analytical skills, problem solving, communication and documentation skills.

TECHNICAL SKILLS

Hadoop/Big Data: HDFS, MapReduce, Hive, Pig, Sqoop, Flume, Oozie, and ZooKeeper.

No SQL Databases: HBase,Cassandra, MongoDB

Languages: Java, PL/SQL, Pig Latin, HiveQL, UNIX shell scripts

Java/J2EE Technologies: JDBC, JNDI, JSON, JSTL, JMS, Java Script, JSP, Servlets, EJB, JSF, JQuery

Frameworks: MVC, Struts, Spring, Hibernate

Operating Systems: HP-UNIX, RedHat Linux, Ubuntu Linux

Web Technologies: HTML, DHTML, XML, AJAX, WSDL, SOAP

Web/Application servers: Apache Tomcat, WebLogic, JBoss

Databases: Oracle 9i/10g/11g, DB2, SQL Server, MySQL, Teradata

Tools: and IDE Eclipse, NetBeans, Toad, Maven, ANT, JDeveloper, DB Visualizer

Version Controls: CVS, Tortoise SVN, Rational ClearCase

PROFESSIONAL EXPERIENCE

Confidential, CA

Hadoop Java Developer

Responsibilities:

• Responsible for requirements gathering and analyzing the data sources.
• Responsible for building scalable distributed data solutions using Hadoop.
• Worked very closely with the Hadoop Administrator to set up the Hadoop cluster.
• Worked on analyzingHadoopcluster and different Big Data analytic tools including Pig, Hive HBase database and SQOOP.
• Importing and exporting data into HDFS and Hive using SQOOP.
• InstalledHadoop, Map Reduce, HDFS, and Developed multiple map reduce jobs in PIG and Hive for data cleaning and pre-processing.
• Involved in setting up a nine node Hadoop cluster on Red hat LINUX.
• Generate OBIEE reports to verify the Hive tables data.
• Used Pig as ETL tool to do Transformations, even joins and some pre-aggregations before storing the data onto HDFS.
• Experience working in spark eco system using sparksql and scala queries on different formats like Text file, CSV file.
• Develop, design and support complex Tableau reports that measure key business metrics
• Proficient in Mongodb, java script and MySql.
• Involved in loading data from LINUX file system to HDFS.
• Load and Transform large sets of structured and semi structured data.
• Exported data from HDFS environment into RDBMS using Sqoop for report generation and visualization purpose.
• Created Hive tables to store variable data formats of PII data coming from different portfolios.
• Implemented a script to transmit sysprin information from Oracle to Hive using Sqoop.
• Designed and developed MapReduce jobs to analyze the data.
• Implemented best income logic using Pig scripts and UDFs.
• Implemented Test scripts to support Test Driven development and continuous integration.
• Worked on tuning the performance Pig queries.
• Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
• Responsible to manage data coming from different sources.
• Involved in loading and transforming large sets of structured, semi structured and unstructured data.
• Cluster coordination services through Zookeeper.
• Developed Spark scripts by using Scala Shell commands as per the requirement.
• Experience in managing and reviewing Hadoop log files.
• Used Kafka and Storm for processing application log data in real time.
• Job management using Oozie scheduler.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
• Responsible for cluster maintenance, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
• Experience using Amazon Web Services
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.
• Involved in email campaign launch and served as point of contact for anything related to data, pulling customers list, matching and finished training in Strong View email platform software.

Environment: Mapr, Hadoop, MapReduce, HDFS, Spark, Hive, Pig, SQL, Oracle, Netezza, Sqoop, Flume, Oozie, Java 8, Eclipse, HBase, Shell Scripting, Storm, Scala, python, Linux Red Hat, Python, Tableau, Kafka.

Confidential, Malvern, PA

Hadoop Developer

Responsibilities:

• Worked on Distributed/Cloud Computing (Map Reduce/Hadoop, Hive, Pig, HBase, SQOOP, Flume, Spark, AVRO, Zookeeper etc.), Cloudera distributedHadoop(CDH5).
• Handling raw data from various subsystems and Load the data from different subsystems into to HDFS for further processing.
• Developed Apache Pig scripts and UDF’s extensively for data transformations and calculating Statement date formats and aggregates the monitory transactions.
• Integrated Apache Kafka for data ingestion.
• Created Hive tables to store the processed results in a tabular format.
• Developed the SQOOP scripts in order to make the interaction between Pig and MySQL Database.
• Involved in gathering the requirements, designing, development and testing.
• Writing the script files for processing data and loading to HDFS.
• Data ingestion to HBase and Hive using Storm bolts
• Working knowledge experience in Python.
• Used SVC tool extensively for building jar files of MapReduce programs and deployed to Cluster.
• Built reusable Hive UDF libraries for business requirements which enabled users to use these UDF's in Hive querying.
• Implemented performance-tuning techniques along various stages of the Migration process.
• Developed Hive Scripts for implementing dynamic Partitions.
• Experienced in handling different optimization join operations like Map join, Sorted Bucketed Map join etc.
• Implemented Partitioning, Dynamic Partitions and Buckets in Hive for analytical processing by business users.
• Expertise in performance tuning on Hive Queries, joins and different configuration parameters to improve query response time.
• Completely involved in the requirement analysis phase.
• Implemented POC writing programs in Scala and data processed using SparkQL.
• Conducted POC forHadoopand Spark as part of NextGen platform implementation. Implemented recommendation engine using Scala.
• Setting up cron job to deletehadooplogs/local old job files/cluster temp files.
• Monitor System health and logs and respond accordingly to any warning or failure conditions.
• Tested raw data and executed performance scripts.
• Setup Hive with MySQL as a Remote Metastore .
• Exported analyzed data to relational databases using Sqoop for visualization to generate reports for the BI team.
• Moved all log/text files generated by various products into HDFS location.
• Implemented POC for using APACHE IMPALA for data processing on top of HIVE.
• StreamlinedHadoopjobs and workflow operations using Oozie workflow.
• Experienced in writing unit test cases, implement unit test cases using JUnit.
• Worked with testing teams and resolved defects.

Environment: Hadoop(CDH5), UNIX, Scala, Python, Storm, Spark, SparkQL, Map Reduce, Apache Pig, Hive, Impala, SQOOP, Java, Eclipse, Kafka, MySQL and oozie.

Confidential, San Jose, CA

Java/J2EE Developer

Responsibilities:

• Created design documents and reviewed with team in addition to assisting the business analyst / project manager in explanations to line of business.
• Developed Hibernate JDBC code for establishing communication with database.
• Responsible for understanding the scope of the project and requirement gathering.
• Involved in analysis, design, development and testing of the application
• Developed the web tier using JSP, Struts MVC to show account details and summary.
• Developed the application using Struts Framework that uses Model View Controller (MVC) architecture with JSP as the view.
• Used Struts Tiles Framework in the presentation tier.
• Developed presentation layer using HTML5, and CSS3, Ajax.
• Used AJAX for asynchronous communication with server
• Used Multi-Threading to handle more users.
• Used RESTful Web services for transferring data between applications.
• Wrote test cases which adhere to a Test Driven Development (TDD) pattern.
• Used Java Message Service (JMS) for reliable and asynchronous exchange of important information such as Credit card transactions report.
• Utilized Hibernate for Object/Relational Mapping purposes for transparent persistence onto the Oracle database.
• Used Spring framework for dependency injection/Inversion of control (IOC), and integrated Struts and Hibernate frameworks.
• Involved in writing Spring Configuration XML files that contains declarations and other dependent objects declaration.
• Used Tomcat web server for development purpose.
• Involved in creating and running of Test Cases for JUnit Testing.
• Used Oracle as Database and used Toad for queries execution and also involved in writing SQL scripts, PL/SQL code for procedures and functions.
• Used CVS for version controlling.
• Developed application using Eclipse and used build and deploy tool as Maven.
• Used Log4J to print the logging, debugging, warning, info on the server console.

Environment: Java 8, Spring framework, Spring Model View Controller (MVC), Struts 2.0, XML, Hibernate 3.0, UML, Java Server Pages (JSP) 2.0, Servlets 3.0, JDBC4.0, JUnit, Log4j, MAVEN, Win 7, HTML, RESTClient, Eclipse, Agile Methodology, Design Patterns, WebSphere 6.1.

Confidential

Java/J2EE Developer

Responsibilities:

• Created design documents and reviewed with team in addition to assisting the business analyst / project manager in explanations to line of business.
• Responsible for understanding the scope of the project and requirement gathering.
• Involved in analysis, design, development and testing of the application
• Developed the web tier using JSP, Struts MVC to show account details and summary.
• Used Struts Tiles Framework in the presentation tier.
• Designed and developed the UI using Struts view component, JSP, HTML, CSS and JavaScript.
• Used AJAX for asynchronous communication with server
• Utilized Hibernate for Object/Relational Mapping purposes for transparent persistence onto the Oracle database.
• Used Spring framework for dependency injection/Inversion of control (IOC), and integrated Struts and Hibernate frameworks.
• Involved in writing Spring Configuration XML files that contains declarations and other dependent objects declaration.
• Developed Web pages using JSPs with Tag libraries, HTML, and JavaScript.
• Writing J2EE code using Spring, Hibernate to upload input CSV files for credit risk data.
• Implemented Dependency Injection (IOC) feature of spring framework to inject dependency into objects and AOP is used for Logging.
• Designed and developed persistence layer build on ORM framework and developed it using Hibernate.
• Implemented various Design patterns like Business Delegate, Data Transfer Objects DTO, Service locator, Session Facade and Data Access Objects DAO patterns.
• Involved in writing SQL, Stored procedure and PL/SQL for back end. Used Views and Functions at the Oracle Database end.
• Developed various documents within the application using XML by using Eclipse as IDE tool.
• Developed SOAP requests to interact with billing schedule system.
• Used Tomcat web server for development purpose.
• Used Web Services (SOAP & WSDL) to exchange data between Server part and Comerica Bank.
• Involved in creating and running of Test Cases for JUnit Testing.
• Used Oracle as Database and used Toad for queries execution and also involved in writing SQL scripts, PL/SQL code for procedures and functions.
• Used CVS for version controlling.
• Developed application using Eclipse and used build and deploy tool as Maven.
• Used Log4J to print the logging, debugging, warning, info on the server console.

Environment: Java, J2EE, JUnit, AJAX, XML, JavaScript, Spring, Struts, Hibernate, Log4j, CVS, Maven, Eclipse, Apache Tomcat, and Oracle.