SUMMARY

• 7+years of professional experience in developing, implementing, configuring, testing Hadoop ecosystem components and maintenance of various web based applications using Java, J2EE.
• More than 3 years of experience as Hadoop Developer with sound knowledge in Hadoop ecosystem technologies.
• Good knowledge of hadoop daemons like Name Node, Secondary Name Node, Data Node, Job Tracker and Task Tracker.
• Excellent understanding of distributed storage systems like HDFS and batch processing systems like Map reduce and Yarn.
• Experience in developing efficient MapReduce programs.
• Experience in using MR Unit to analyze the data and testing the MapReduce jobs.
• Experience in using Hive QL for analyzing, querying and summarizing data.
• Good understanding of No SQL databases like Cassandra, Hbase and MongoDB.
• Experience in fetching data into HDFS from various databases like MySQL, Oracle,DB2using Sqoop.
• Experience in collecting, aggregating and loading log files into HDFS using Apache Flume.
• Experience in working with file formats like Parquet, Avro, RC File etc.
• Experience in using Pig Latin for manipulating data.
• Used Pig as ETL tool to do transformations, event joins, filter and some pre - aggregation.
• Experience in writing Pig scripts, Pig Latin UDFs and Hive UDFs to load data files.
• Experience in scheduling and monitoring job work flows using Oozie and Zookeeper.
• Good understanding in processing of real-time data using Spark and Scala.
• Explored Cloudera Impalaand Apache Spark for real-time analytical processing.
• Experience in using Apache Solr for indexing and load balanced querying.
• Queried both Managed and External tables created by Hive using Impala.
• Knowledge in Cassandra read and write paths and internal architecture.
• Experienced with SparkContext, Spark-SQL, Data Frame, Pair RDD's, Spark YARN.
• Experience in working with different Hadoop distributions like CDH and Hortonworks.
• Hands-on experience with message brokers such as Apache Kafka and RabbitMQ.
• Worked on Configuring Zookeeper, Kafkacluster.
• Hands on experience in working with real time streaming applications using Storm.
• Good knowledge of integrating Spark Streaming with Kafka for real time processing of streaming data.
• Experience in using build management tools like Maven and Ant.
• Good Knowledge in Report Designing using PentahoBA suite.
• Developed ETL job in Talend to load data from ASCII, Flat files.
• Hands on Evaluation of ETL (Talend) and OLAP tools and recommend the most suitable solutions based on business requirements.
• Experience in generating reports using Tableau by connecting to Hive.
• Hands on experience on UNIX scripting.
• Knowledge on AWS (Amazon EC2) Hadoop distribution.
• Experience in using Kerberose for authenticating the end users using hadoop in a secure mode.
• Expertise in design and development of Web Applications involving J2EE technologies with Java, Spring, EJB, Servlets, JSP, Struts, Web Services, XML, JMS, JDBC and etc.
• Experience in using Eclipse and Net Beans IDE’s.
• Extensive experience in using Relational databases like Oracle, SQL Server, DB2 and MySQL.
• Expertise in using Tomcat server and also application servers like JBoss and Web Logic.
• Experience in all phases of software development life cycle.
• Support development, testing, and operations teams during new system deployments.

TECHNICAL SKILLS

Programming Languages: C, C++, Java, Python, Scala.

Hadoop Eco System: HDFS, Map Reduce, Hive, Pig, Sqoop, Flume, Zookeeper, Oozie, Kafka, Storm, Spark, Impala.

Database: MySQL, Oracle, SQL Server,DB2.

No SQL Databases: HBase, Cassandra, MongoDB.

Scripting Languages: HTML, CSS, JavaScript, XML.

Tools:, Technologies and Utilities: Servlets, Struts, JDBC, JSP, Web Services.

Business Intelligence Tools: Pentaho, Tableau.

Operating Systems: Windows XP/7/8, 10, Linux RedHat/Ubuntu/Cent OS.

Cluster Monitoring Tools: Horton Works, Cloudera Manager.

Web/ App Servers: Apache Tomcat Server, JBOSS, Web Logic.

IDE's: Eclipse, Microsoft Visual Studio, Net Beans.

PROFESSIONAL EXPERIENCE

Hadoop Developer

Confidential - Eden Prairie, MN

Responsibilities:

• Imported data from RDBMS to HDFS using Sqoop.
• Managed data coming from different sources and involved in HDFS maintenance and loading of structured and unstructured data.
• Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in HDFS.
• Performed storing of Solr indexes in HDFS.
• Performed several aggregation logics using Spark and Scala in the backend.
• Helped market analysts by creating Hive queries to spot the emerging trends by comparing fresh data with HDFS reference tables and historical metrics.
• Defined the job flows in Oozie to automate the process of data loading into the HDFS and Pig.
• Designed HBase schema to avoid Hot spotting and exposed the data from HBase tables to RESTAPI on UI.
• Transformed raw data from several data sources into baseline data by developing Pig scripts and loaded the data into HBase tables.
• Worked on Scalajobs using Spark for data processing using RDD's and Dataframe API.
• Worked on Creating Kafkatopics, partitions, writing custom partitioner classes.
• Used Kafka to store events from various system and processed using Spark Streaming to perform near real time analytics.
• Involved in creating POCs to ingest and process streaming data using Spark streaming and Kafka.
• Automated the batch processing by developing Shell Scripts and processed the daily jobs through Maestro scheduler.
• Co-ordinate with the cross-functional team to ensure that applications are properly tested, configured, and deployed.

Environment: HDFS, YARN, Pig, Hive, HBase, Spark, Scala, Solr, Sqoop, Oozie, Linux, Maven, GIT, Jenkins.

Hadoop Developer

Confidential - Lyndhurst, NJ

Responsibilities:

• Pre-processed large sets of structured and semi-structured data, with different formatslike Text Files, Avro, Sequence Files, and JSON Record.
• Involved in Preparing the Design Documentslike Sequence Diagrams, External Design Report & Internal Design Report.
• Worked on POCs and involved in fixing performance & storage issues using different Compression Techniques, Custom Combiners and Custom Partitioners.
• Involved in Developing Hive scripts to parse the raw data, populate staging tables and store the refined data in partitioned tables in the Hive.
• Developed utility classes in Hive using UDFs.
• Developed scripts to automatically transfer several hundreds of tables from Hive to MySQL using Sqoop..
• Worked with HBase, a No SQL database for POC in storing URL's and images - semi structured and unstructured data coming from UNIX and a variety of portfolios.
• Did an assessment on the Cassandra clusters for performance improvements and to knock out the existing time outs.
• Stored the analyzed results back into the Cassandracluster.
• Used MR Unit for testing raw data and executed performance scripts.
• Assisted with data capacity planning and node forecasting.
• Analyzed Migration plans for various versions of Cloudera Distribution of ApacheHadoopto draw the impact analysis, and proposed the mitigation plan for the same.
• Experience in analyzing the latest versions, solutions, and software in theHadoopEco System and proposing them to the stakeholders with the proven solutions for the better architecture of the project

Environment: Hadoop, Hive, Hbase, SQOOP, Oozie, Cassandra, Java, Unix, MySQL, Cloudera, Maven, GIT.

Hadoop Developer

Confidential - Naples, FL

Responsibilities:

• Performed importing of data from different data sources, did transformations using Mapreduce.
• Loaded data into HDFS from RDBMS using Sqoop.
• Responsible for Managing and scheduling jobs on a Hadoopcluster.
• Performed data cleaning and preprocessing by using MapReduce.
• Wrote Pig scripts for analyzing the data to know the customer behavior.
• Implemented Partitioning and bucketing in Hive.
• Used Impala to pull Hive table data for faster query processing.
• Generated final reporting data using Tableau for testing by connecting to the corresponding Hive tables using Hive ODBC connector.
• Performed data cleansingusing Pig.
• Used Sqoop to export the analyzed data to the relational databases for visualization and also to generate reports for the BI team.
• Pre-processed the data using Pig UDFs for analysis.
• Performed querying of both Managed and External tables created by Hive using Impala.
• Automated the process of loading the data into HDFS by developing Oozie workflows.
• Monitored System health logs and responded accordingly to any warning or failure conditions.
• Managed and reviewed Hadooplog files.

Environment: Cloudera, Hadoop, MapReduce, HDFS, Sqoop, Hive, Pig, Oozie, Impala, Linux, XML, Tableau, Java

Java Developer

Confidential

Responsibilities:

• Involved in Requirement analysis and design, development of the application using Java Technologies.
• Developed the login screen so that the application can be accessed only by authorized and authenticated administrators.
• Used HTML, CSS, JSP's to design and develop front end and used Java Script to perform user validation.
• Performed Designing, developing, and configuring server side J2EE components like EJB, Java Beans, and Servlets.
• Involved in Creating tables, functions, triggers, sequences and stored procedures in PL/SQL.
• Implemented business logic by developing Session Beans.
• Used JDBC-API to communicate with the Database.
• Used tomcat server to deploy the application.
• Developed application using Waterfall model software methodology.
• Involved In technical documentation of project.

Environment: Java, HTML, CSS, JSP, Servlets, EJB, JDBC, PL/SQL, Tomcat, Oracle8i.

Associate Java Developer

Confidential

Responsibilities:

• Used struts MVC Architecture to develop the modules.
• Developed The UI using JavaScript, JSP, HTML, and CSS for interactive cross browser functionality and complex user interface.
• Used Servlets and Session beans for creating Business Logic and deployed them on Web logic server.
• Application design using MVC struts framework.
• Developed complex SQL Queries and PL/SQL Stored procedures.
• Prepared the Functional, Design and Test case specifications.
• Performed some database side validations by writing Stored Procedures in Oracle.
• Used JUNIT for unit testing of the application.
• Provided Technical support for production environments resolving the issues, analyzing the defects, providing and implementing the solution defects. Resolved more priority defects as per the schedule.

Environment: Java, HTML, Java Script, CSS, Oracle, JDBC, Swing and Eclipse.