SUMMARY

• Having 7 plus years of professional IT experience which includes 3 plus years of experience in Hadoop Development and Administration using Cloudera (CDH) and Horton works(HDP) Distributions.
• Over Three plus years of experience in design, development, maintenance and support of Big Data using Hadoop(Cloudera and Hortonworks) Ecosystem tools like HDFS, Hive, Pig, Sqoop, Flume, Zookeeper, MapReduce, Spark and Oozie.
• Knowledge of multiple distributions/platforms (Cloudera, Hortonworks).
• Experienced in installation, configuration, supporting and monitoring 100+node Hadoop cluster using Cloudera manager and Hortonworks Ambari distributions.
• Have experience in installing, configuring, performance tuning and administrating Hadoop cluster for major Hadoop distributions like CDH 4, HDP 2.2.0.
• Experienced in mapping Hadoop clusters and services using Cloudera manager and Hortonworks ambari.
• Good understanding/knowledge of Hadoop architecture and various components such as HDFS, Yarn, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming.
• Strong working experience wif ingestion, storage, querying, processing and analysis of Big data.
• Experience in installation, configuration, supporting and managing Hadoop clusters.
• Expertise in writing Hadoop Jobs for analyzing data using Hive and Pig.
• Loaded streaming log data from various web servers into HDFS using Flume.
• Successfully loaded files to hive and HDFS from Oracle and SQL Server, Netezaa, Flat Files using SQOOP and FTP/SFTP.
• Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems (RDBMS) and vice - versa.
• Extensive experience wif SQL, PL/SQL and database concepts
• Knowledge of job workflow scheduling and monitoring tools like Oozie and Azkaban.
• Good understanding of NoSQL databases.
• Having experience on creating databases, tables and views in HIVEQL, IMPALA and PIG LATIN.
• Load and transform large sets of structured, semi-structured and unstructured data using Hadoop ecosystem components.
• Experience in working wif different data sources like Flat files, XML files
• Worked wif project documentation and also documented other application related issues, bugs on internal wiki website.
• Experience in trouble shooting failed mission critical production systems under extreme pressure conditions and time constraints.
• Full life cycle experience, involved in requirement analysis, design, development, testing, deployment and support.
• Experience in analyzing logs for troubleshooting java application issues from server side.
• A very good team player wif teh ability to work independently wif minimal supervision.

TECHNICAL SKILLS

• Application Development
• Object Oriented Programming (OOP)
• Big Data / Hadoop
• Hadoop(Cloudera, HDP)
• HDFS
• Map Reduce, YARN
• Sqoop, Hive, Pig
• Flume, Impala
• Oozie, Zookeeper
• Spark, Tez, Ambari
• Splunk
• Shell Script, Python
• PERL, PUPPET
• JBoss/WildFly,
• Websphere, Weblogic
• Apache Http
• Tomcat
• J2SE, J2EE
• XML Web Services SOAP/REST
• Eclipse, NetBeans, IntelliJ IDE
• HTML/CSS, Java Script
• Oracle 10g, 11i
• DB2, MySQL, MS Access
• SQL Server, Netezza
• Unix, Linux
• AIX, Solaris, Windows
• IP Center, JIRA
• Wily Introscope
• Nagios, UC4, Git
• Maven, ANT, Jenkin
• Subversion, Tortoise
• Azkaban, Filezilla
• Ambari
• Cloudera Manager

PROFESSIONAL EXPERIENCE

Confidential, Foster City, CA

Hadoop Administrator

Responsibilities:

• Performed Hadoop clusters administration through Cloudera Manager.
• Worked on Hadoop clusters capacity planning and management.
• Monitoring and Debugging Hadoop jobs/Applications running in production.
• Worked on Providing User support and application support on Hadoop Infrastructure.
• Monitored already configured cluster of 54 nodes.
• Installed and configured Hadoop components Hive, Impala, Pig.
• Communicating wif teh development teams and attending daily meetings.
• Addressing and Troubleshooting issues on a daily basis.
• Working wif data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos TEMPprincipals and testing HDFS, Hive.
• Cluster maintenance as well as creation and removal of nodes.
• Monitor Hadoop cluster connectivity and security.
• Manage and review Hadoop log files.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
• Installed Oozie workflow engine to run multiple Hive and pig jobs.
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
• Supported in setting up QA environment and updating configurations for implementing scripts wif Pig and Sqoop.
• Diligently teaming wif teh infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability.

Environment: Cloudera Hadoop, HDFS, Pig, Hive, Sqoop, Shell Scripting, Core Java, Oracle 11g, Linux, UNIX.

Confidential, Atlanta, GA

Hadoop Developer &Administrator

Responsibilities:

• Data Ingestion has done from teh different sources like Oracle DB, Netezza and flat files.
• Written a master script to download files from FTP source, uncompress them into staging and do a HDFS put.
• Analyzing teh data and using PIG, HIVE for teh loading of teh data into HDFS.
• Vast use of Shell scripting for teh loading of data into HDFS.
• Developed customized flume agents to consume live network data and persist into HDFS.
• Designed hive partitions that get created on a daily basis using Oozie workflows.
• Designed PIG scripts to filter network data based on configuration settings in a MySQL database.
• Developed Oozie workflows to look for configuration changes and rewrite partitions for hive tables.
• Written a Java code for Schema Analysis Engine to analyze teh data from different sources and create Hive QL scripts.
• Created Hive pre and partition tables for different sources manually and using Schema Analysis Engine.
• Created hive tables for various types of SERDE format.
• Worked on HCatalog which allows PIG and Map Reduce to take advantage of teh SerDE data format transformation definitions that write for HIVE.
• Data Ingestion has done in three layers wif different transformations using Hive and Pig Scripts.
• Written managed and external tables using Hive for different sources manually and using Schema Analysis Engine.
• Written Insert Hive QL scripts and Sqoop Jobs.
• Worked on different file formats (orc file, rc file, sequence file, text file) and different Compression Codec’s ( gzip, snappy).
• Worked on both Cloudera and Hortonworks distribution.
• Validated teh data in Production and QA after upgrading teh Cloudera version of Hadoop Environment.
• Importing and exporting data into HDFS and Hive using Sqoop.
• HiveQL scripts to create, load, and query tables in aHive.
• Installed and configured Hive and also written Hive UDFs.
• Created and executed Azkaban flow for different sources in DEV, QA and PROD.
• Worked on PIG Latin Scripts and UDF's while ingestion, querying, processing and analysis of Data.
• Validated daily volume metrics in DEV, QA and PROD for different sources.
• Experienced in managing and reviewing Hadoop log files.
• Responsible for naming conventions of column and table names in Hive and Pig Scripts.
• Load and transform large sets of structured, semi structured and unstructured data.
• Responsible to manage teh data coming from different sources.
• Created and maintained Technical documentation for launching Hadoop clusters and for executing Hive queries and Pig Scripts.
• Worked wif application teams to install operating system, Hadoop updates, patches, version upgrades as required.

Environment: Hortonworks/Cloudera Hadoop, HDFS, Pig, Hive, Sqoop, Shell Scripting, Core Java, Netezza, Oracle 11g, Linux, UNIX.

Confidential, Chicago, IL

Big Data Developer &Administrator

Responsibilities:

• Customized flume agents to consume live network data and persist into HDFS.
• Managing and reviewing Hadoop log files.
• Extracting files through Sqoop and place in HDFS and processed.
• Running Hadoop streaming jobs to process terabytes of xml format data.
• Loading and transforming large sets of structured, semi structured and unstructured data.
• Responsible to manage data coming from different sources.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Hive QL scripts to analyze customer data to determine patient’s health patterns.
• Hive QL scripts to create, load, and query tables in a Hive.
• Hive QL scripts to perform Sentiment Analysis (analyzed customer's comments and product ratings).
• Installed and configured Hive and also written Hive UDFs.
• Designed hive partitions that get created on a daily basis using oozie workflows.
• Utilized Apache Hadoop environment by Cloudera works.
• Experienced in defining job flows
• Implemented automated system used to set infrastructure leveraged by different users of combine site in Java/Scala.
• Experienced in managing and reviewing Hadoop log files.
• Load and transform large sets of structured, semi structured and unstructured data
• Responsible to manage data coming from different sources.
• Supported Map Reduce Programs those are running on teh cluster.
• Involved in loading data from UNIX file system to HDFS.
• Loading and transforming large sets of structured, semi structured and unstructured data.
• Developed Oozie workflows to look for configuration changes and rewrite partitions for hive tables.
• Worked on using technologies like Oracle Weblogic, Redhat Jboss, WildFly, IBM Websphere administration for various clients.

Environment: Java, Hadoop, Hive, Pig, JDBC, UNIX, HTML, CSS, XML, Oracle Weblogic, Redhat Jboss, WildFly, IBM Websphere.

Confidential

Java Developer

Responsibilities:

• Involved in teh design of teh applications using J2EE. This architecture employs a Model/View/Controller (MVC) design pattern.
• Developed code for presentation layer using MVC architecture using Struts framework that uses Servlets and JSP.
• Implemented Action Form class, Action class and Action Mapping for separating teh logic from teh presentation using Struts.
• Developed presentation layer using HTML, CSS, JSP and JavaScript.
• Developed teh halper classes for better data exchange between teh MVC layers.
• Coordinating wif offshore team, Trouble shootings and solved issues.
• Implemented exception mechanism and used Struts error message mechanism.
• Involved in writing SQL scripts.
• Eclipse is used as an IDE for development.
• Developed ANT Script to compile teh Java files and to build teh jars and wars.
• Implemented MVC architecture usingStruts 1.1in terms ofJSPandServlets.
• Written JavaScript for validation of page data in teh JSP pages.
• Good working knowledge on monitoring and troubleshooting.

Environment: Java, Servlets, JDBC, HTML, CSS, JavaScript, SQL Server, IBM Websphere, JBoss.