PROFESSIONAL SUMMARY:

• Over 8 years of IT professional experience with 3+ years of experience as Hadoop Administrator.
• Experience with complete Software Design Life Cycle including design, development, testing and implementation of moderate to advanced complex systems.
• Good experience in installation, configuration and management of clusters in Cloudera (CDH4)& Hortonworks (HDP2.2) distributions using CM and Ambari 2.2.
• Experience in Hadoop architecture and its various components such as HDFS, Job Tracker, Task Tracker, NameNode, Data Node and MapReduce concepts.
• Good understanding and hands on experience of Hadoop Cluster capacity planning, performance tuning, cluster monitoring, troubleshooting.
• Hands on experience in installation of various Hadoop Ecosystem components like MapReduce, HDFS, Hive, HBase, and Hadoop Daemons.
• Experience in adding and removing nodes to an existing Hadoop cluster.
• Extensive experience on commissioning, decommissioning, balancing and managing nodes.
• Have configured various property files like core - site.xml, hdfs-site.xml, mapred-site.xml and hadoop-env.sh to set up required environment according to the job requirement.
• Good knowledge on analyzing log files for Hadoop ecosystem services and troubleshooting.
• Strong knowledge in Name node high availability and recovery of Name node metadata and data residing in the cluster.
• Experience in configuration and management of security for Hadoop cluster using Kerberos.
• Good knowledge in NOSQL databases like HBase,Cassandra.
• Experience in Ranger, Knox configuration to provide the security for Hadoop services.
• Designed and Developed the required data warehouse model using Star schema for the generalized model
• Used forward engineering approach for designing and creating data marts for OLAP model
• Experience in designing data models for databases and Data Warehouse/Data Mart/ODS for OLAP and OLTP environments.
• Having Strong Experience in LINUX/UNIX Administration, expertise in Red Hat Enterprise Linux 4, 5 and 6.
• Possesses excellent oral and written communication skills and works well in team environment.

TECHNICAL SKILLS:

Big Data Ecosystem: HDFS, HBase, MapReduce, Hive, Pig, Sqoop, Kafka, Flume, Oozie.

Database: SQL Server 2000/2005/2008 , MySQL, NoSQL Databases HBase, MongoDB.

Scripting Languages: UNIX shell scripting (bash), Python, SQL, PIG LATIN.

ETL tools: Talend Open Studio, ER studio

Network: Http/Https, TCP/IP, ssh, Ftp, Telnet.

Security: Kerberos Security.

Operating Systems: Unix, Linux (RHEL, Ubuntu, CentOS), Windows XP, Server 2000, 03, Server 2008.

PROFESSIONAL EXPERIENCE:

Confidential, Boca Raton, Florida

Hadoop Admin

Responsibilities:

• Installed and configured a Horton Works HDP 2.2 using Ambari and manually through command line.
• Worked on analyzing Hadoop cluster and different big data analytic tools including Apache Pig, Apache HBase and Apache Sqoop.
• Responsible for building scalable distributed data solutions using Hadoop.
• Worked on installing cluster, commissioning & decommissioning of Datanode, Namenode recovery.
• Created HBase tables to store variable data.
• Performed the data exchange between HDFS and different Web Applications and databases using Flume and Sqoop.
• Managing and reviewing Hadoop log files and debugging failed jobs.
• Implemented Kerberos Security Authentication protocol for the cluster.
• Worked on tuning the performance Pig queries.
• Wrote scripts to report the status of the cluster and scripts to manageApache HBase processes.
• Worked with Infrastructure teams to install operating system, Hadoop updates, patches as required.
• Backed up data on regular basis to a remote cluster using distcp.
• Responsible to manage data coming from different sources.
• Loaded the dataset into Hive for ETL Operation.
• Installation of Apachekafka and configuring it as per the requirements.
• Worked on logical data models and physical data models level using ER studio according to requirements.
• Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports.
• Implemented Fair scheduler to allocate fair amount of resources for scheduling small jobs.
• Involved in analyzing large amounts of data sets to determine optimal way to aggregate and report on it.

Environment: Hadoop HDFS, MapReduce, Hortonworks, Ambari, Hive, Pig, Kafka, Oozie, Sqoop, ER studio, Apache HBase.

Confidential, Hartford, CT

Hadoop Admin

Responsibilities:

• Experienced on setting up Horton works cluster and installing all the ecosystem components through Ambari and manually from command line.
• Cluster maintenance, Monitoring, commissioning and decommissioning of data nodes, troubleshooting, manage and review log files.
• Actively involved in installationperformance tuning, patching, regular backups, user account administration, upgrades and documentation.
• Installation of new components and removal of them through Ambari.
• Periodically reviewed Hadoop related logs and fixed errors.
• Commissioned new cluster nodes for increased capacity and decommissioned servers with hardware problems.
• Responsible for adding new eco system components, like flume, knox with required custom configurations based on the requirements and Hadoop daemons.
• Developed Python, Shell Scripts and Power shell for automation purpose.
• Implemented Kerberos Security Authentication protocol for existing cluster.
• Worked with Ranger, Knox configuration to provide centralized security to Hadoop services.
• Hands on experience with NoSQL databases like Hbase, Cassandra and MongoDB.
• Working experience on maintaining MySQL databases creation and setting up the users and maintain the backup of databases.
• Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes.
• Performing Linux systems administration on production and development servers (Red Hat Linux, CentOS and other UNIX utilities).

Environment: HDP 2.2, HDFS, Python, HBase, Kafka, Yarn,MongoDB,Hive, Oozie, Pig, Sqoop, Shell Scripting, MySQL, RHEL, CentOS, Ambari.

Confidential, Tarrytown, NY

Hadoop Admin

Responsibilities:

• Installed Cloudera CDH 4cluster (name node and data nodes).
• Monitored Hadoop cluster and file system usage on name and data nodes.
• Created user accounts and given users the access to the Hadoop cluster.
• Performed HDFS cluster support and maintenance tasks like adding and removing nodes.
• Lead various data conversion initiatives that included testing Hadoop SQL performance using bucketing and partitioning.
• Use of Sqoop to import and export data from HDFS to RDBMS and vice-versa.
• Balanced and tuned HDFS, Hive, Impala, MapReduce, and Oozie work flows.
• Configured Flume and Kafka to set up a live stream of data
• Experience in importing and exporting the logs using Flume.
• Enabled Sentry and Kerberos to ensure data protection.
• Experience with Sentry to enable role based authorizations for Hive
• Exported all log files generated from various sources to HDFS for further processing.
• Monitoring Hadoop Cluster through Cloudera Manager and Implementing alerts based on Error messages. Providing reports to management on Cluster Usage Metrics.
• Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive.
• Worked on YUM configuration and package installation through YUM.
• Performed installation of Apache web server for Ganglia to publish charts and graphs in the web console.
• Set up automated monitoring for Hadoop cluster using Ganglia, which helped figure out the load distribution, memory usage and provided an indication for more space.

Environment: HDFS, Map Reduce, Hive, Flume, Sqoop, Cloudera Manager, CDH4, HBase, Impala, Sentry, Splunk, Pig.

Confidential

Linux Administrator

Responsibilities:

• Provisioning, building and support of Linux servers both physical and virtual using VMware for production, QA and developers environment.
• Installed, configured and Administrated of all UNIX/LINUX servers, includes the design and selection of relevant hardware to support the installation/upgrades of RHEL, Cent OS, Ubuntu operating systems.
• Utilized bash and ksh shell scripting to automate daily system administration tasks
• Managed data backup of UNIX, Windows, Virtual servers, disk storage tier1 and tier2 backups.
• Ensured that various backup life cycle policies, and daily backup jobs are running, and failed jobs are fixed.
• Maintained configuration and security of the UNIX/LINUX operations systems with the enterprise's computing environment.
• Monitored system activities and fine-tuned system parameters and configurations to optimize performance and ensure security of systems.
• Responsible for maintenance of development tools and utilities and to maintain shell automation Scripts.
• Created and maintained detailed procedural documentation regarding operating system installation and configuration of software packages.
• Responsible for configuring real time backup of web servers. Log file was managed for troubleshooting and probable errors.
• Responsible for reviewing all open tickets, resolve and close any existing tickets.
• Performance Tuning, Client/Server Connectivity and Database Consistency Checks using different Utilities.
• Installation, Configuration, upgrade and administration of Ubuntu, RedHat Linux.
• Worked with File System includes UNIX file System and Network file system.

Environment: RHEL 4/5, Ubuntu, CentOS 4/5, SUSE Linux 10.1/10.3, VMware, Agile, Yum, Rpm .

Confidential

Linux Administrator

Responsibilities:

• Installation and configuration of Red Hat Linux, Solaris, Fedora and Cent OS on new server builds as well as during the upgrade situations.
• Log management like monitoring and cleaning the old log files.
• Administration of RHEL4.x, 5.x which includes installation, testing, tuning, upgrading and loading patches, troubleshooting both physical and virtual server issues.
• System audit report like no. of logins, success & failures, running cron jobs.
• System performance for hourly basis or daily basis.
• Remotely copying files using sftp, ftp, scp, winscp, and filezilla.
• Created user roles and groups for securing the resources using local operating System authentication.
• Experienced in tasks like managing User Accounts and Groups, managing Disks and File systems.
• Installing RedHat Linux using kick start and applying security polices for hardening the server based on company’s policies.
• Install and configure Instruction Detection System (IDS) like Tripwire, Snort, and Lids.
• Configuring & monitoring DHCP server.
• Taking backup using tar and recovering during the data loss
• Wrote bash scripts for job automation.
• Manage system installation, troubleshooting, maintenance, performance tuning, managing storage resources, network configuration to fit application and database requirements.
• Responsible for modifying and optimizing backup schedules and developing shell scripts for it.
• Performed regular installation of patches using RPM and YUM.
• Maintained LVM, VxVM and SVM filesystems along with NFS.

Environment: Linux (Red Hat Enterprise, CentOS), Windows 2000/NT, HP, IBM, Solaris, Cisco routers/switches, RPM, YUM, DHCP.