SUMMARY

• Around 6 years of IT experience including 2.5 years of experience in Hadoop development and good object oriented programming skills.
• Good noledge of Hadoop Development and various components such as HDFS, Job Tracker, Task Tracker, Data Node, Name Node and Map - Reduce concepts.
• Responsible for writing MapReduce programs.
• Expertise with teh tools in Hadoop Ecosystem including Pig, Hive, HDFS, Map Reduce, Sqoop, Spark, Kafka, Yarn, Oozie, and Zookeeper.
• Experience in installation, configuration, Management, supporting and monitoring Hadoop cluster using various distributions such as Apache SPARK and Cloudera.
• Good exposure to teh design, development / support of Apache SPARK, Hadoop and Big data ecosystem using Apache Spark 1.6( SQL + DataFrames, Spark Streaming, MLlib, GraphX ), Infosphere Biginsights 4.0 (IBM's Product), Cloudera CDH 5.5, Hortonworks HDP 2.3, MapR 5.0.
• Excellent programming skills with experience in Java, C, SQL and Python Programming.
• Experience in tuning and troubleshooting performance issues in Hadoop cluster.
• Experience in using Cloudera Manager for installation and management of single-node and multi-node Hadoop cluster (CDH3, CDH4 & CDH5).
• Experience in cloud stack such as Amazon AWS and VMWARE stack.
• Experience in importing and exporting data using Sqoop from HDFS file system to Relational Database Systems and vice-versa.
• Hands on NoSQL database experience with HBase.
• Used NoSQL database with Cassandra, MongoDB and Monod
• Performing analysis using high level languages such as Ruby and Python. Extending Apache pig functionality using Python.
• Experience in creating custom Lucene/Solr Query components.
• Worked on data warehouse product Amazon Redshirt which is a part of teh AWS (Amazon Web Services).
• Experience in developing Pig scripts and Hive Query Language.
• Written Hive queries for data analysis and to process teh data for visualization.
• Experience in managing and reviewing Hadoop Log files.
• Well experienced and posses strong noledge in Unix Shell Scripting
• Hands on experience in application development using Java, RDBMS and Linux Shell Scripting.
• Developed Java applications using various IDE’s like IBM RAD7 and Eclipse.
• Good in writing Python, Shell and Perl scripts to automate activities via Jenkins and Monitoring Build System
• Experience in developing e-commerce applications, highly distributed applications using Java, J2EE, Servlets, JSP, Struts, Spring, JDBC, Apache Tomcat Server, JavaScript, HTML, XML, XSL, SQL, Oracle 11g, 9i/8i/7.x under Unix and Windows NT/XP. Distributed Object Oriented Component analysis and Design according to industry leading J2EE platforms.
• Have worked on data visualization using Tableau
• Extensive noledge in Python, JAVA, MYSQL, Unix and Linux.Extensive experience in Programming, Deploying, Configuring on Web Servers such as IBM Web sphere, BEA's Web Logic, Junit, Apache Tomcat Server, Jbuilder and Amazon EC2.
• Web development using Python, FLASK, and Django.
• Highly proficient in developing SQL, PL/SQLs.
• Detailed understanding of Software Development Life Cycle (SDLC) and sound noledge of project implementation methodologies including Waterfall and Agile.

TECHNICAL SKILLS

Hadoop/Big Data: HDFS, Mapreduce, HBase, Pig, Hive, Sqoop, Flume, MongoDB, Oozie,Zookeeper, Cassandra, Spark, Storm, & Kafka

Java Technologies: Java, J2EE, JSTL, JDBC 3.0/2.1, JSP 1.2/1.1, Java Servlets, JMS, JUnit, Log4j

IDE Tools: Eclipse 3.5, Net Beans, My Eclipse, Oracle JDeveloper 10.1.3, SOAP UI, Ant

Big data Analytics: Datameer 2.0.5

Frameworks: MVC, Struts, Hibernate, Spring

Visualization Tools: Tableau, Talend

Programming languages: C, C++, Java, Python, Ruby, Ant scripts, Linux shell scripts

Databases: Oracle 9i/10g, MySQL, DB2, MS-SQL Server

Web Servers: IIS 7.0, AWS, EC2, S3, RDS, ELB

Web Technologies: HTML, XML, JavaScript, AJAX, SOAP, WSDL

ETL Tools: Informatica, Pentaho, SSRS, SSIS, BO, Crystal reports, Cognos.

PROFESSIONAL EXPERIENCE

Confidential - CA

Hadoop Developer

Responsibilities:

• Worked on analyzing Hadoop cluster using different big data analytic tools including Kafka, Pig, Hive and Map Reduce.
• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Real time streaming teh data using Spark with Kafka.
• Configured Spark streaming to receive real time data from teh Kafka and store teh stream data to HDFS using Scale.
• Worked on debugging, performance tuning of Hive & Pig Jobs.
• Implemented test scripts to support test driven development and continuous integration.
• Importing and exporting data into HDFS using Sqoop and Kafka.
• Experience working on processing unstructured data using Pig.
• Developed Hadoop streaming Map/Reduce works using Python.
• Expertise with NoSQL databases like Hbase, Cassandra, DynamoDB (AWS) and MongoDB
• Plan, deploy, monitor, and maintain Amazon AWS cloud infrastructure consisting of multiple EC2 nodes and VMWare Vm's as required in teh environment.
• Expertise in AWS data migration between different database platforms like SQL Server to Amazon Aurora using RDS tool.
• Gained experience in managing and reviewing Hadoop log files.
• Involved in scheduling Oozie workflow engine to run multiple pig jobs.
• Automated all teh jobs for pulling data from FTP server to load data into Hive tables using Oozie workflows.
• Involved in using HCATALOG to access Hive table metadata from Map Reduce or Pig code.
• Continuous monitoring and managing teh Hadoop cluster using Cloudera Manager.
• Developed Enterprise Lucene/Solrbased solutions to include custom type/object modeling and implementation into teh Lucene/Solranalysis (Tokenizers/Filters) pipline.
• Created customSolrQuery components to enable optimum search matching.
• Responsible for developing data pipeline using flume, Sqoop and pig to extract teh data from weblogs and store in HDFS.
• Proficient work experience with NOSQL, Monod databases.
• Developed Hadoop streaming Map/Reduce works using Python.
• Administrating Tableau Server backing up teh reports and providing privileges to users.
• Worked on Tableau for generating reports on HDFS data.
• Extracted and updated teh data into Monod using Mongo import and export command line utility interface.
• Developed Pig Latin scripts to extract data from teh web server output files to load into HDFS.
• Extracted and updated teh data into Monod using Mongo import and export command line utility interface.
• Used Eclipse and ant to build teh application.
• Used NoSQL database with Cassandra and Monod.

Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Sqoop, Spark, AWS EC2, S3, RDS, Kafka, Solr, LINUX, Cloudera, Big Data, Java APIs, Java collection, Python, SQL, NoSQL, Cassandra,Tableau, HBase.

Confidential - Dallas, TX

Hadoop Developer

Responsibilities:

• Developed data pipeline using Flume, Sqoop, Pig and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Involved in writing Map Reduce jobs.
• Involved in Sqoop, HDFS Put or Copy from Local to ingest data.
• Used Pig to do transformations, event joins, filter boot traffic and some pre-aggregations before storing teh data onto HDFS.
• Involved in developing Pig UDFs for teh needed functionality that is not out of teh box available from Apache Pig.
• Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting.
• Involved in developing Hive DDLs to create, alter and drop tables.
• Managed works including indexing data, tuning relevance, developing custom tokenizers and filters, adding functionality includes playlist, custom sorting and regionalization with Solr Search Engine.
• Cluster coordination services through Zoo Keeper.
• Configured Spark streaming to receive real time data from teh Kafka and store teh stream data to HDFS using Scale.
• Experience on AWS cloud services like EC2, S3, RDS, ELB, and EBS for installing, configuring and troubleshooting on various Amazon images for server migration from physical into cloud.
• Worked on data warehouse product Amazon Redshirt which is a part of teh AWS (Amazon Web Services)
• Manage Amazon Web Services (AWS) EC2 with Puppet
• Experience with Load Balancers on teh AWS.
• Configured and integrated ApacheSolrfor ALDOT's documents full-text search
• Utilized Custom and OOTBSolrResponse writers to power various enterprise applications
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
• Installed and maintained Apache Hadoop, Map Reduce, HDFS (Hadoop Distributed File System), and Pig, Hbase, Cassandra, zookeeper and Sqoop.
• Installed and configured Cloudera Manager for easy management of existing Hadoop Cluster.
• Involved in developing Hive UDFs for teh needed functionality that is not out of teh box available from Apache Hive.
• Worked on implementation of a log producer in SCALA that watches for application logs, transforms incremental logs and sends them to a Kafka and Zookeeper based log collection platform.
• Involved in using HCATALOG to access Hive table metadata from Map Reduce or Pig code.
• Computed various metrics using Java Map Reduce to calculate metrics that define user experience, revenue etc.
• Proficient work experience with NoSQL, Monod databases.
• Extracted and updated teh data into Monod using Mongo import and export command line utility interface.
• Developed Daily Test engine using Python for continuous tests.
• Used Hive to analyze teh partitioned and bucketed data and compute various metrics for reporting.
• Created and maintained technical documentation for launching Hadoop clusters and for executing Hive queries and Pig Scripts.
• Created Data extract jobs using Python/Django and Google app engine.
• Extracted and updated teh data into Monod using Mongo import and export command line utility interface.
• Involved in processing ingested raw data using Map Reduce, Apache Pig and Hive.
• Involved in developing Pig Scripts for change data capture and delta record processing between newly arrived data and already existing data in HDFS.
• Involved in emitting processed data from Hadoop to relational databases or external file systems using SQOOP, HDFS GET or CopyToLocal.

Environment: Hadoop, Map Reduce, Cassandra, Mongo db, Yarn, Hive, Pig, HBase,Kafka, Oozie, Sqoop,AWS EC2, Spark, Solr, Flume, Oracle 11g, Core Java, Cloudera, HDFS, Eclipse, Python

Confidential, Mumbai, MH

Hadoop Developer

Responsibilities:

• Worked on analyzing Hadoop cluster and different Big Data analytic tools including Pig, Hive, HBase and Sqoop.
• Installed Hadoop, MapReduce, HDFS, and developed multiple MapReduce jobs in PIG and Hive for data cleaning and pre-processing.
• Coordinated with business customers to gather business requirements, also interacted with other technical peers to derive technical requirements and delivered teh BRD and TDD documents.
• Extensively involved in Design phase and delivered Design documents.
• Involved in Testing and coordination with business in User testing.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Written Hive jobs to parse teh logs and structure them in tabular format to facilitate TEMPeffective querying on teh log data.
• Involved in creating Hive tables loading data and writing queries that will run internally in MapReduce way.
• Involved in processing ingested raw data using Map Reduce, Apache Pig and HBase.
• Involved in developing Pig Scripts for change data capture and delta record processing between newly arrived data and already existing data in HDFS.
• Populated HDFS and Cassandra with huge amounts of data using Apache Kafka.
• Used Hive to analyze teh partitioned and bucketed data to compute various metrics for reporting.
• Expertise with NoSQL databases like Hbase, Cassandra, DynamoDB (AWS) and MongoDB.
• Used Python for pattern matching in build logs to format errors and warnings
• Importing and exporting data into HDFS and Hive using Sqoop.
• Implemented test scripts to support test driven development and continuous integration.
• Used Pig as ETL tool to do Transformations, even joins and some pre-aggregations before storing teh data on to HDFS.
• Installed Oozie workflow engine to run multiple Mapreduce jobs.
• Load and transform large sets of structured and semi structured data.
• Responsible to manage data coming from different sources.
• Involved in creating Hive Tables, loading data and writing Hive queries.
• Involved in Unit testing and delivered Unit test plans and results documents.
• Exported data from HDFS environment into RDBMS using Sqoop for report generation and visualization purpose.

Environment: Hadoop, Hive, MapReduce, Pig, Sqoop, Kafka, Cloudera, Spark, HBase, HDFS, Python, Solr, Zookeeper, Cassandra

Confidential, Mumbai, MH

Java Developer

Responsibilities:

• Involved in teh analysis, design, and development and testing phases of Software Development Lifecycle (SDLC) using agile development methodology.
• Involved in business requirement gathering and technical specifications.
• Implemented J2EE standards, MVC2 architecture using Struts Framework.
• Implementing Servlets, JSP and Ajax to design teh user interface.
• Used JSP, Java Script, HTML5, and CSS for manipulating, validating, customizing, error messages to teh User Interface.
• Presentation components in JSP pages are built using ICE faces tag libraries.
• ICE Faces libraries are used in all presentation pages like Search/Inquiry and data collection pages.
• Used JBoss for EJB and JTA, for caching and clustering purpose.
• Used EJBs (Session beans) to implement teh business logic, JMS for communication for sending updates to various other applications and MDB for routing priority requests.
• GUI was developed using JSP, AJAX and JavaScript, spring framework.
• Involved in teh Development of Spring Framework Controllers.
• Worked with Flied level engineers and teams to make teh product more user-friendly.
• Wrote Web Services using SOAP for sending and getting data from teh external interface.
• Used XSL/XSLT for transforming and displaying reports Developed Schemas for XML.
• Involved in writing teh ANT scripts to build and deploy teh application.
• Developed a web-based reporting for monitoring system with HTML and Tiles using Struts framework.

Environment: JAVA multithreading, collections, J2EE, EJB, UML, SQL, PHP, Sybase, Eclipse, JavaScript, WebSphere, JBOSS, HTML5, DHTML, CSS, XML, Log4j, ANT, STRUTS 1.3.8, JUNIT, JSP, Servlets, Rational Rose, Hibernate.

Confidential

Software Engineer

Responsibilities:

• Involved in Analysis, Design, Implementation, and Testing of teh project.
• Implemented teh presentation layer with HTML, XHTML, JavaScript, and CSS.
• Developed web components using JSP, Servlets, and JDBC.
• Created various web page layouts using HTML, and CSS.
• Implemented validations using JavaScript.
• Designed and developed a web-based client using Servlets, JSP, Tag Libraries, Java Script, HTML and XML using Spring Framework.
• Implemented database using MySQL.
• Involved in fixing defects and unit testing with test cases using Junit.
• Developed user and technical documentation.
• Made extensive use of Java Naming and Directory interface (JNDI) for looking up enterprise beans.
• Developed presentation layer using HTML, CSS, and JavaScript.
• Developed stored procedures and triggers in PL/SQL.
• Database design, writing stored procedures and triggers, writing session and entity beans, JMS client and message driven beans to receive & process JMS messages, JSPs & Servlets using MVC architecture.
• Deployed teh application in WebLogic server.
• Responsible for Parsing XML data using XML parser and Testing, fixing of teh bugs and coding modifications.
• Involved in writing JUnit test cases and suits using Eclipse IDE.

Environment: Java, JSP, Servlets, JDBC, JavaScript, CSS, MySQL, JUnit, Eclipse, Apache Tomcat