SUMMARY

• Cloudera Certified Apache Hadoop Administrator (CCAH)
• Having 8 Years of professional experience in Software Development with core java and Hadoop/Big Data technologies.
• About 3 years of experience with Hadoop Ecosystem including HDFS, MapReduce, Hive, Pig, Flume, Sqoop, impala, ZooKeeper, Hue, Oozie and HBase.
• Experience implementing big data projects using Cloudera
• Installed, Configured and Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Pig, HBase, Zookeeper and Sqoop.
• Experience in installing, configuring and optimizing ClouderaHadoopversion CDH3, CDH 4.X and CDH 5.X in a Multi Clustered environment.
• Experience in Implementing High Availability of Name Node and Hadoop Cluster capacity planning to add and remove the nodes.
• Hands - on experience on major components in Hadoop Ecosystem including HDFS, Yarn, Hive, Flume, Zookeeper, Oozie and other ecosystem Products.
• Commissioning and Decommissioning Hadoop Cluster nodes Including Balancing HDFS block data.
• Balanced cluster after adding/removing nodes or major data cleanup
• Configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml andhadoop-env.xml based upon the job requirement.
• Added security to the cluster using Kerberos.
• Monitoring and support through Ganglia and Nagios
• Experience in Rebalance on HDFS cluster
• Automating system tasks using Puppet.
• Cluster Management using Cloudera Manager
• Experience in resolving a NameNode Checkpoint failure.
• Configuring Sqoop and Exporting/Importing data into HDFS
• Experience in Setting up Data Ingestion tools like Flume and Sqoop
• Collected the logs data from web servers and integrated into HDFS using Flume.
• Wrote theshellscriptsto monitor the health check ofHadoopdaemon services and respond accordingly to any warning or failure conditions.
• Configured NameNode high availability and NameNode federation
• Expertise in Importing and exporting data using Sqoop from HDFS to RDBMS and Troubleshooting issue related to Sqoop jobs.
• Wrote Pig scripts to load and aggregate the data.
• Monitored workload, job performance and capacity planning using Cloudera Manager
• Having a Knowledge on Hbase
• Installed and configured Hive
• Used Hive and Pig to analyze data from HDFS.
• Goal oriented, reliable, highly motivated and disciplined
• Highly motivated team player with zeal to learn new technologies.
• Effective interpersonal skills with abilities to meet deadlines & work under pressure.

TECHNICAL SKILLS

Big Data Ecosystems: Hadoop, YARN, MapReduce, HDFS, Hive, Pig, Sqoop, Flume, Zookeeper, Oozie, Hbase, kafka, Spark, Impala.

Scripting Languages: JavaScript, Python, Shell

Programming Languages: Core Java, C/C++

Web Technologies: HTML, XML, JavaScript

Version Control: CVS, Rational Clear Case

Hadoop Distributions: Cloudera Distribution

Databases: Oracle 9i/10g, SQL

Operating Systems: Windows variants, Linux Ubuntu, Centos

Other Tools: MSOffice

PROFESSIONAL EXPERIENCE

Confidential, Hoffman Estates, IL

Hadoop Admin

Responsibilities:

• Installed, Configured and Maintained Apache Hadoop clusters for application development and Hadoop tools like Hive, Hbase, Zookeeper and Sqoop.
• Extensively involved in Installation and configuration of Cloudera distribution Hadoop NameNode, Secondary NameNode, Resource Manager, Node Manager and DataNodes.
• Collected the logs data from web servers and integrated into HDFS using Flume.
• Worked on installing cluster, commissioning & decommissioning of DataNodes, NameNode recovery, capacity planning, and slots configuration.
• Installed Oozie workflow engine to run multiple Hive Jobs
• Worked with Kafka for the proof of concept for carrying out log processing on distributed system.
• Developed data pipeline using Flume, Sqoop and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
• Configured various property files like core-site.xml, hdfs-site.xml, yarn-site.xml, mapred-site.xml andhadoop-env.xml based upon the job requirement.
• Worked on Hue interface for querying the data
• Automating system tasks using Puppet.
• Created Hive tables to store the processed results in a tabular format.
• Utilized cluster co-ordination services through ZooKeeper.
• Moved Relational Database data using Sqoop into Hive Dynamic partition tables using staging tables.
• Populated HDFS with huge amounts of data using Apache Kafka
• Involved in collecting metrics for Hadoop clusters using Ganglia and Nagios
• Configuring Sqoop and Exporting/Importing data into HDFS
• Configured NameNode high availability and NameNode federation.
• Experienced in loading data from UNIX local file system to HDFS.
• Managing and scheduling Jobs on aHadoopcluster using Oozie.
• Configured NameNode high availability and NameNode federation.
• Use of Sqoop to import and export data from HDFS to Relational database and vice-versa.
• Data analysis in running Hive queries.
• Involved in Installing Cloudera Manager,Hadoop, Zookeeper, HBASE, HIVE, PIG etc
• Involved in configuring Quorum base HA for NameNode and made the cluster more resilient
• Integrated Kerberos intoHadoopto make cluster more strong and secure from unauthorized users
• Fine Tuned JobTracker by changing few properties mapred-site.xml
• Fine TunedHadoopcluster by setting proper number of map and reduced slots for the TaskTrackers.

Environment: Hadoop, HDFS, MapReduce, Yarn, Hive, Pig, Sqoop, Oozie, Flume, Zookeeper, kafka, Ubuntu

Confidential, Hoffman Estates, IL

Hadoop Admin

Responsibilities:

• Installed, Configured and Maintained Apache Hadoop clusters for application development and Hadoop tools like Pig, Zookeeper and Sqoop
• Wrote Pig scripts to load and aggregate the data.
• Worked on analyzing Hadoop cluster using different big data analytic tools including Pig, Hbase database and Sqoop
• Extensively involved in Installation and configuration of Cloudera distribution Hadoop NameNode, Secondary NameNode, JobTracker, TaskTrackers and DataNodes.
• Importing And Exporting Data from MySQL/Oracle to HiveQL using SQOOP.
• Worked on installing cluster, adding and removing of DataNodes
• Responsible for operational support of Production system
• Load and transform large sets of structured, semi structured and unstructured data
• Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
• Wrote shell scripts for rolling day-to-day processes and it is automated.
• Installed and configured Hive.
• Configured various property files like core-site.xml, hdfs-site.xml, mapred-site.xml andhadoop-env.xml based upon the job requirement.
• Troubleshoot Map/Reduce jobs.
• Wrote theshellscriptsto monitor the health check ofHadoopdaemon services and respond accordingly to any warning or failure conditions.
• Loading log data directly into HDFS using Flume
• Experienced on loading and transforming of large sets of structured, semi structured and unstructured data.
• Balanced cluster after adding/removing nodes or major data cleanup
• Created and modified scripts (mainly bash) to accommodate the administration of daily duties.
• Generate datasets and load to Hadoop ecosystem
• Involved in creating Hive tables, loading with data and writing Hive queries which will invoke and run MapReduce jobs in the backend.
• Cluster co-ordination services through ZooKeeper.
• Used Hive and Pig to analyze data from HDFS
• Wrote Pig scripts to load and aggregate the data
• Used Sqoop to import the data into SQL Database.
• Used Java to develop User Defined Functions (UDF) for Pig Scripts.

Environment: Hadoop, HDFS, MapReduce, Hive, Pig, Sqoop, Flume, Zookeeper, Oozie

Confidential, Overland Park, KS

Java Developer

Responsibilities:

• Created sequence diagrams, collaboration diagrams, class diagrams, Use Cases and activity diagrams using Rational Rose for the Configuration, Cache & logging Services.
• Designed and developed the project using MVC design pattern.
• Developed front end using Struts framework.
• Implementing Struts based framework to present the data to the user.
• Created the web UI using Struts, JSP, Servlets and Custom tags.
• Used OOAD for building applications.
• Coded different action classes in struts responsible for maintaining deployment descriptors like struts-config and web.xml using XML.
• Designed and Developed front-end using struts framework. Used JSP, JavaScript, Custom Tag libraries and Validations provided by struts framework. .

Environment: Core Java, JSP, Servlets, Struts, Junit, HTML, JavaScript, NetBeans IDE, Rational Rose, Apache Tomcat 5.x, and Oracle10g

Confidential 

Java Developer

Responsibilities:

• Developed various use cases, sequence diagrams using Rational Rose.
• Used OOAD for building Applications
• Involved in writing of JSP pages for the presentation of data
• Created HTML, CSS, and DHTML pages for front end, used JavaScript for client side validation
• Used JDBC for database connectivity to communicate with the back-end databases.
• Developed HTML reports for various modules as per the requirement.
• Involved in Normalization of database and created stored procedures using PL/SQL.

Environment: JSP, Servlets, HTML, DHTML, CSS, JavaScript, JDBC, Eclipse IDE, UML, Rational Rose