SUMMARY

• 7+ Years of experience in IT industry includes AWS Hadoop Bigdata consultant in Banking, Telecom and financial clients
• Having 4+ years of comprehensive experience as a Hadoop (HDFS, MAPREDUCE, HIVE, PIG, SQOOP, FLUME, SPARK,
• KAFKA, KAFKA, ZOOKEEPER, AVRO, OOZIE, HBASE) Hadoop Consultant & Administrator.
• In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, JobTracker,
• TaskTracker, NameNode, DataNode and MapReduce concepts.
• Well versed in installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster.
• Hands on experience on major components in Hadoop Ecosystem like Hadoop Map Reduce, HDFS, HIVE, PIG, HBase, Sqoop.
• Experience in NoSQL database like Cassandra.
• Experience in running workflow jobs with actions that run Hadoop Map/Reduce and Pig jobs.
• Experience in managing and reviewing Hadoop Log files.
• Experience in Hadoop administration activities such as installation and configuration of clusters using Apache and Cloudera.
• Good Knowledge in Amazon AWS concepts like EMR and EC2 web services which provides fast and efficient processing of Big Data.
• Experience in handling Hadoop Cluster and monitoring the cluster using Cloudera Manager, Ambari, Nagios and Ganglia
• Experience in Hadoop Shell commands, verifying managing and reviewing Hadoop Log files.
• Experience in performing major and minor upgrades of Hadoop clusters in Apache, and Cloudera distributions.
• Experience in deployment of Hadoop cluster using Puppet tool.

TECHNICAL SKILLS

Big Data Technologies: HDFS, Hive, MapReduce, Cassandra, Pig, Hcatalog, Phoenix, Falcon, Scoop, FlumeZookeeper, Mahout, Oozie, Avro, HBase, MapReduce, HDFS, Storm, CDH 5.3, CDH 5.5.

Programing Languages: C, Core Java, SQL, and PL/SQL

Data Ingestions And ETL Tools: Flume, Sqoop, storm, Kafka

BusinessIntelligenceTools: MSBI,Stack(SSIS,SSRS),Visual Studio 2013/2011/2008/2005

Databases: Oracle 11g, MySQL, MS SQL Server, Confidential DB2.

Relational Databases: MYSQL

NoSQL Databases: HBase, MongoDB,Cassandra.

Operating Systems: Linux, UNIX, MAC, Windows NT / 98 /2000/ XP, Windows 8.

Monitoring Tools: Cloudera Manager, Ambari, Nagios, Ganglia

Scripting Languages: Shell Scripting, Puppet, Scripting, Python, Bash, CSH.

PROFESSIONAL EXPERIENCE

Confidential, SCHAMBURG, IL

Hadoop Administrator

Responsibilities:

• Build and Support Hadoop Based EDW platform to support ETL process. Streamline data ingestion from multiple source systems into Datalake. Transform data. Create and support workflow in Talend TAC and Oozie, develop Pig, Hive QL, Spark QL, Spark Streaming scripts.
• Build Master Data Management (MDM) data sourcing data from datalake. Create data lineage using Talend TMM.
• Install and configure and manage hdp2.3.2/hdp2.5.2 hortonworks hadoop on premise cluster in Lower Level and Production, Disaster Recovery environments.
• Use Ambari, zookeeper, oozieFalcon to manage the cluster. Stabilize Bigdata/Hadoop cluster with regular patch updates and cluster upgrades. Build Operational process to manage Bigdata/hadoop clusters with agreed Service Level Agreements.
• Secure Bigdata/Hadoop cluster with robust security architecture using Kerberos, Active Directory, Ranger, Knox, Centrify. Automate hadoop cluster access controls through Ranger policies for the uptodate security. Provide granula security upto column level.
• Data governance from the TMM data lineage and Ranger based access controls on HDFS and Hive Metadata. End user security through Kerberos CLI tools, beeline and ambari views.
• Installed MySQL, Cassandra, and HBase.
• Prepare and execute contingency plan by building Disaster Recovery hadoop cluster and automate replication from Production with regular interval. Oozie, Falcon based replication.
• Build interactive web interface to the developers and end users of hadoop with Ambari view to run Pig jobs, hive jobs, View Tez jobs.

Confidential, Irvine, CA

Hadoop Administrator

Responsibilities:

• Installed, Configured and Maintained the Hadoop cluster for application development and Hadoop ecosystem components like Hive, Pig, HBase, Zookeeper and Sqoop.
• Worked on Hadoop Architecture and various components such as HDFS, Name Node, Data Node, Resource Manager,Node Manager and YARN / Map Reduce programming paradigm.
• Monitoring Hadoop Cluster through Cloudera Manager and Implementing alerts based on Error messages. Providing reports to management on Cluster Usage Metrics and Charge Back customers on their Usage.
• Extensively worked on commissioning and decommissioning of cluster nodes, replacing failed disks, file system integrity checks and maintaining cluster data replication.
• Assigning number of mappers and reducers to Map reduce cluster.
• Setting up HDFS Quotas to enforce the fair share of computing resources.
• Configuring and maintaining YARN Schedulers (Fair and Capacity).
• Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
• Setting up HBase cluster which includes master and region server configuration, High availability configuration, performance tuning and administration.
• Created user accounts and given users the access to the Hadoop cluster.
• Involved in loading data from UNIX file system to HDFS.
• Worked on ETL process and handled importing data from various data sources, performed transformations.
• Coordinate with QA team during testing phase.
• Provide application support to production support team.
• Implemented Hadoop stack and different Big Data analytic tools, migration from different databases to Hadoop.
• Monitored multiple Hadoop clusters environments using Ganglia and Nagios.
• Monitored workload, job performance and capacity planning. Environment: Cloudera, HDFS, Hive, Sqoop, Zookeeper and HBase, Unix Linux, Java, HDFS, Map Reduce, Pig, Hive, HBase, Flume, Sqoop, Shell Scripting.

Confidential, Chicago, IL

Linux / Hadoop Administrator

Responsibilities:

• Responsible for creating virtual environments for the rapid development.
• Responsible for handling the tickets raised by the end users which includes installation of packages, login issues, access issues User management like adding, modifying, deleting, grouping
• Responsible for preventive maintenance of the servers on monthly basis.
• Configuration of the RAID for the servers. Resource management using the Disk quotas.
• Responsible for change management release scheduled by service providers.
• Implemented and Configured High Availability Hadoop Cluster.
• In depth understanding of Hadoop Architecture and various components such as HDFS, Name node, Data node, Resource Manager, Node Manager and YARN / Map - Reduce programming paradigm.
• Involved in managing and reviewing Hadoop log files.
• Used Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Designed, recorded, and executed macros to automate data entry inputs. Formatted spreadsheets and workbooks for print, document reproduction, and presentations.
• Created HBase tables to store various data formats of data coming from different portfolios.
• Hive, HBase.
• Hands on experience working on Hadoop ecosystem components like HDFS, Map-Reduce, YARN, Zookeeper,Pig, Hive, Sqoop, Flume.
• Developed a data pipeline using Kafka and Storm to store data into HDFS.
• Collected, organized, and documented infrastructure project attributes, data, and project metrics.
• Developed project user guide documents which help in knowledge transfer to new testers and solution repository document which gives quick resolution of any issues occurred in the past thereby reducing the number of invalid defects.
• Identify repeated issues in production by analyzing production tickets after each release and strengthen the system testing process to arrest those issues moving to production to enhance customer satisfaction.
• Experience interpreting physical database models and understanding relational database concepts such as indexes, primary and foreign keys, and constraints using Oracle.
• Performed MySQL Replication setup and administration on Master-Slave and Master-Master.
• Developed stored procedures, triggers in MySQL for lowering traffic between servers & clients.
• Used MySQL workbench, query browser utilities
• Writing, optimizing, and troubleshooting dynamically created SQL within procedures.
• Managing Systems operations with final accountability for smooth installation, networking, and operation,troubleshooting of hardware and software in LINUX environment.
• Accomplished System/e-mail authentication using LDAP enterprise Database.
• Implemented a Database enabled Intranet web site using LINUX, Apache, MySQL Database backend.
• Installed Cent OS using Pre-Execution environment boot and Kick-start method on multiple servers.Monitoring System Metrics and logs for any problems.
• Running Cron-tab to back up Data. Applied Operating System updates, patches and configuration changes.
• Maintaining the MySQL server and Authentication to required users for Databases. Appropriately documented various Administrative & technical issues.

Environment: Red Hat Enterprise Linux 4.x,5.x, Ubuntu, Centos, Sun Solaris 8,9,10, VERITAS Cluster Server, Veritas Volume Manager, SLURM, Oracle 11G, HP UX, HP Blade, Confidential AIX, HP ProLiant DL 385, 585 WebLogic, Oracle RAC/ASM, MS Windows 2008 server.