SUMMARY:

• 7+ years of experience in full software development life cycle from concept through the delivery of applications and customizable solutions with emphasis on Object Oriented Programming, Java/J2EE, SQL and Hadoop / Big Data technologies .
• Hands on experience on Hadoop Ecosystem components . (HDFS, MapReduce, PIG, HIVE, HBASE, SQOOP, Flume, Oozie, ZooKeeper, Kafka, Spark(Scala), Storm, MongoDB, Cassandra)
• Involved in Installation and configuration of Hadoop Ecosystem components along with Hadoop Admin .
• Hands on experience on Teradata Migration to Hadoop Platform .
• Extensive knowledge on Hive, Spark - Sql Development .
• Exposure to Teradata development.
• Working on implementing Spark and Strom frame work.
• Automated Spark-SQL scripts using Unix shell script and Involved in Optimization of Hive Queries .Also used Optimization techniques for better performance.
• Good understanding of Hadoop architecture and components of Hadoop Clusters (Job Tracker, Task Tracker, Name Node and Data Node).
• Performed Data Ingestion to Hadoop file system from different data sources.
• Analyzed different file formats and large data sets by running Hive queries and Pig scripts .
• Experienced in working with senior level managers, business people and developers across multiple disciplines.
• Using Apache Flume, collected and stored streaming data(log data) in HDFS.
• Extensively used Apache Sqoop for efficiently transferring bulk data between Apache Hadoop and relational databases (Teradata).
• Automated sqoop, hive and pig jobs using Oozie scheduling.
• Extensive knowledge in NoSQL databases like Hbase, MangoDB, Cassandra .
• Have good knowledge on writing and using the user defined functions in HIVE, PIG and MapReduce .
• Developed tests cases and POC's to benchmark and verify data flow through the Kafka clusters .
• Extensive exposure to all aspects of Software Development Life Cycle (SDLC) i.e. Requirements Definition for customization, Prototyping, Coding and Testing .
• Experience in resolving on-going production and maintenance issues and bug fixes.
• Proficiency in developing SQL queries.
• Exposure to Java development projects.
• Excellent communication skills.

TECHNICAL SKILLS:

Big Data Technologies: Pig, Hive, Sqoop, Flume, HBase, Kafka-Storm, Spark with Scala, Oozie, ZooKeeper, Hadoop Distributions(Cloudera, Hortonworks)

Hadoop Paradigms: MapReduce, Yarn

Java Technologies: Java/J2EE - JSP, Servlets, JDBC, JunitWeb Technologies: Ajax, JavaScript, JQuery, HTML, CSS, XML

Programing Languages: Java, Scala,C/ C++, Python

Databases: MySQL, MS-SQL Server, SQL, Oracle 11g, NoSQL

Web Services: REST, AWS, SOAP, WSDL, UD

Tools: Ant, Maven, JUnit

Servers: Apache Tomcat, WebSphere, JBoss

IDE s: MyEclipse, Eclipse, IntelliJ IDEA, NetBeans, WSAD

Web/UI: HTML, Java Script, XML, SOAP, WSDL

ETL/BI Tools: Talend, Tableau

PROFESSIONAL EXPERIENCE:

Confidential, Fremont, CA

Hadoop/Spark Developer

Responsibilities:

• Worked on analyzing Hadoop cluster using different big data analytic tools including Kafka, Pig, Hive and Map Reduce .
• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Real time streaming the data using Spark with Kafka .
• Configured Spark streaming to receive real time data from the Kafka and store the stream data to HDFS using Scale.
• Worked on debugging, performance tuning of Hive & Pig Jobs .
• Implemented test scripts to support test driven development and continuous integration.
• Importing and exporting data into HDFS using Sqoop and Kafka .
• Experience working on processing unstructured data using Pig.
• Developed Hadoop streaming Map/Reduce works using Python.
• Expertise with NoSQL databases like Hbase, Cassandra, DynamoDB (AWS) and MongoDB.
• Plan, deploy, monitor, and maintain Amazon AWS cloud infrastructure consisting of multiple EC2 nodes and VMWare Vm's as required in the environment.
• Expertise in AWS data migration between different database platforms like SQL Server to Amazon Aurora using RDS tool.
• Gained experience in managing and reviewing Hadoop log files.
• Involved in scheduling Oozie workflow engine to run multiple pig jobs .
• Automated all the jobs for pulling data from FTP server to load data into Hive tables using Oozie workflows.
• Involved in using HCATALOG to access Hive table metadata from Map Reduce or Pig code.
• Continuous monitoring and managing the Hadoop cluster using Cloudera Manager .
• Developed Enterprise Lucene/Solr based solutions to include custom type/object modeling and implementation into the Lucene/Solr analysis (Tokenizers/Filters) pipline.
• Created custom Solr Query components to enable optimum search matching.
• Responsible for developing data pipeline using flume, Sqoop and pig to extract the data from weblogs and store in HDFS.
• Proficient work experience with NOSQL, Monod databases.
• Developed Hadoop streaming Map/Reduce works using Python .
• Administrating Tableau Server backing up the reports and providing privileges to users.
• Worked on Tableau for generating reports on HDFS data.
• Extracted and updated the data into Monod using Mongo import and export command line utility interface.
• Developed Pig Latin scripts to extract data from the web server output files to load into HDFS .
• Extracted and updated the data into Monod using Mongo import and export command line utility interface.
• Used Eclipse and ant to build the application.
• Used NoSQL database with Cassandra and Monod .

Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Sqoop, Spark, AWS EC2, S3, RDS, Kafka, Solr, LINUX, Cloudera, Big Data, Java APIs, Java collection, Python, SQL, NoSQL, Cassandra, Tableau, HBase.

Confidential, Concord, CA

Hadoop Developer

Responsibilities:

• Installed and configured multi-nodes fully distributed Hadoop cluster .
• Involved in installing Hadoop Ecosystem components.
• Responsible to manage data coming from different sources.
• Involved in Hadoop Cluster environment administration that includes adding and removing cluster nodes.
• Supported Map Reduce Programs those are running on the cluster .
• Involved in HDFS maintenance and administering it through Hadoop-Java API .
• Configured Fair Scheduler to provide service-level agreements for multiple users of a cluster.
• Maintaining and monitoring clusters. Loaded data into the cluster from dynamically generated files using Flume and from relational database management systems using Sqoop.Managing nodes on Hadoop cluster connectivity and security.
• Resolved configuration issues with Apache add-on tools.
• Involved in enhancing the speed performance using Apache Spark .
• Developed Spark code using scala and Spark-SQL for batch processing of data.
• Involved in parsing JSON data into structured format and loading into HDFS/Hive using spark streaming .
• Used Pig as ETL tool to do transformations, event joins, filter both traffic and some pre-aggregations before storing the data onto HDFS.
• Involved in writing Flume and Hive scripts to extract, transform and load the data into Database.
• Cluster capacity planning, performance tuning, cluster Monitoring, Troubleshooting .

Environment: Cloudera Hadoop, Linux, HDFS, Hive, Sqoop, Flume, Pig, Spark, Zookeeper, HBase.

Confidential, Kenilworth, NJ

Hadoop Developer

Responsibilities:

• Application installation of Hadoop, Hive, Map Reduce & Sqoop .
• Analyzing the log files using Hive queries and writing UDF's f or Hive queries.
• Creating Hive tabular partitions and buckets.
• Moving all log files generated from various sources to HDFS for further processing through Flume.
• Used R programming language to graphically analyse the data. Used Tableau to represent data.
• HDFS support and maintenance and Adding/Removing a Node, Data Rebalancing .
• Developed MapReduce application using Hadoop, MapReduce programming and HBase.
• Developed the Pig scripts and Hive Reports .
• Writing Mapreduce programs and tested using MRUnit .
• Analyzed the advantage of introduction of Cassandra NoSQL database into the present system.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users.

Environment: Hadoop, Java-MapReduce, Hive, Pig, Cassandra, HBase, Ubuntu, Eclipse, Sqoop.

Confidential, LA

Java Developer

Responsibilities:

• Responsible for gathering and analyzing requirements and converting them into technical specifications.
• Involved in all the phases of (SDLC) including Requirements Collection, Design & Analysis of the Customer Specifications from Business Analyst.
• Implemented various J2EE Design patterns like Business Delegate, Data Access Object, and Model View Controller.
• Designed JSP Tag libraries for reusable web interface components (Tabbed UI, dynamic menus, and editable combo boxes).
• Developed the front end of the application using Struts framework.
• Validations were performed using Struts validator.
• Developed Spring Controllers, Domain utility classes to implement business logic and transaction management.
• Used Spring AOP for creating various advices for transaction and security features.
• Used Hibernate as the ORM tool for data persistence. Involved in developing the mapping files (hbm.xml) for the persistence classes.
• Involved in creating and manipulating the Database Objects such as Tables and Views using DDL and DML. Also involved in optimization of queries.
• Developed HQL queries to retrieve data using Hibernate.
• Extensively used Unit testing using JUnit.
• Unix Shell Scripting was used for creating the deployment environments for Deployment into test servers and production servers.
• Used Log4J to perform logging in the applications. Used Ant as a build tool.

Environment: J2EE, MDB, Hibernate, XML, CVS, Oracle Web logic Application Server, Oracle 9i, Log4J, JUnit, Unix, Ant.