SUMMARY:

• 7 years of Total Information Technology experience with expertise in Administration and Operations experience, in Big Data and Cloud Computing Technologies
• Expertise in setting up fully distributed multi node Hadoop clusters, with Apache, Cloudera and MapR Hadoop.
• Expertise in AWS services such as EC2, Simple Storage Service (S3), Autoscaling, EBS, Glacier, VPC, ELB, RDS, IAM, Cloud Watch, and Redshift.
• Expertise in MIT kerberos and High Availability as well as Integration of Hadoop clusters.
• Experience in upgrading Hadoop clusters.
• Strong knowledge in installing, configuring and using ecosystem components like Hadoop MapReduce, Oozie, Hive, Sqoop, Pig, Flume, Zookeeper, Kafka, NameNode Recovery, HDFS High Availability, Spark.
• Experience in Hadoop Shell commands, verifying managing and reviewing Hadoop Log files.
• Experience in understanding the security requirements for Hadoop and integrate with Kerberos authentication and authorization infrastructure.
• Extensive experience on performing administration, configuration management, monitoring, debugging, and performance tuning in Hadoop Clusters.
• Expertise in writing pig scripts, SQL, HiveQL and CQL.
• Enabling/Disabling of Passive and Active check for Hosts and Service in Nagios.
• Good knowledge in installing, configuring & maintaining Chef server and workstation
• Expertise in provisioning clusters and building manifests files in puppet for any services.
• Excellent knowledge in Import/Export structured, un - structured data from various data sources such as RDBMS, Event logs, Message queues into HDFS, using a variety of tools such as Sqoop, Flume etc.
• Expertise in converting non kerberoized Hadoop cluster to Hadoop with kerberoized cluster
• Administration and Operations experience with Big Data and Cloud Computing Technologies
• Handling and Maintenance of AWS Architectures.
• Handling in setting up fully distributed multi node Hadoop clusters, with Apache and AWS EC2instances
• Handling in AWS services such as EC2, Simple Storage Service(S3), Auto scaling, EBS, ELB, RDS, IAM, Cloud Watch, Redshift & EMR
• Performing administration, configuration management, monitoring, debugging, and performance tuning in Hadoop Clusters.

TECHNICAL SKILLS:

Hadoop/Big Data: HDFS, HBase, Pig, Hive, Zookeeper, Sqoop, MapReduce, Flume, Spark, Oozie, Hortonworks and Cloudera

Languages: Unix Shell Script, JavaScript, Python, Java, Pig, MySQL, HiveQL, CSS, JavaScript, HTML

DevOps Tools: Chef, Puppet

Frameworks: Apache and Cloudera Hadoop, Amazon Web Services

Platforms: Ubuntu, Centos, Redhat, Amazon Linux

Web Servers: Apache/HTTPD, Nginx

Operating Systems: Windows, Macintosh, Ubuntu (Linux).

Monitoring Tools: Nagios, Pager Duty

RDBMS: MySQL, SQL Server

Data Warehousing: Hive

NoSQL Databases: MongoDB, HBase, Cassandra

Log Collector & Aggregation: Flume, Kafka

Source Control Tools: Github

Team Communication: Slack

Defect Tracking Tools: FogBugz, Jira

AWS Components: EC2, Simple Storage Service (S3), EBS, VPC, ELB, RDS, IAM, CloudWatch

Other Tools: S3Organizer, Sqoop, Oozie, Zookeeper, Hue, DBVisualizer, Google authentication to Services

PROFESSIONAL EXPERIENCE:

Confidential, San Ramon, CA

Hadoop Admin/DevOps Engineer

Responsibilities:

• Administration & Monitoring Hadoop & Cassandra Clusters on EC2
• Worked on Hadoop Upgradation from 4.5 to 5.2
• Monitor Hadoop cluster job performance and capacity planning
• Removing from monitoring of particular security group nodes in nagios in case of retirement
• Responsible for managing and scheduling jobs on Hadoop Cluster
• Replacement of Retired Hadoop slave nodes through AWS console and Nagios Repositories
• Performed dynamic updates of Hadoop Yarn and MapReduce memory settings
• Worked with DBA team to migrate Hive and Oozie metastore Database from MySQL to RDS
• Worked with fair and capacity schedulers, creating new queues, adding users to queue, Increase mapper and reducers capacity and also administer view and submit Mapreduce jobs
• Monitoring and Adding/Removing hosts and services to Nagios
• Managing the alert mechanisms through pager Duty
• Monitor Hadoop cluster connectivity and security as well manage and review Hadoop log files
• Attending to daily tickets reg. installations, rebooting, bring up services etc.
• Loaded data from Oracle, MS SQL Server, MySQL, Flat File database into HDFS, HIVE
• Handled data flows and backups to Amazon Simple Storage Service (S3)
• Handled all the service nodes and components on Amazon cloud
• Fixed Namenode partition failed, fsimage not rotated, MR job failed with too many fetch failures and troubleshooting common Hadoop cluster issues
• Implemented manifest files in puppet for automated orchestration of Hadoop and Cassandra clusters
• Maintaining Github repositories for Configuration Management
• Configured distributed monitoring system Ganglia for Hadoop clusters
• Managing cluster coordination services through Zoo Keeper
• Configured and deployed Namenode High Availability Hadoop cluster with SSL and kerberoized
• Deal with the several services restart and killing the process with Pid to clear the alert
• Monitoring Log files of several services, clear files incase of Diskspace issues on sharethis nodes
• Interacting with business users and compiling end user requirements for development process
• 24X7 production support for weekly schedule with Ops team
• Environment: CentOS, CDH4, Hive, Sqoop, Flume, Hbase, MySQL, Cassandra, Oozie, Puppet, Pager Duty, Nagios, AWS (S3, EC2, IAM, Cloud Watch, RDS, ELB, Auto Scaling, EBS, VPC, EMR, Github.

Confidential, SFO, CA

Hadoop Administrator/Support Engineer with DevOps Experience

Responsibilities:

• Maintaining Hadoop Cluster for getting insights data from every 6 months data
• Scheduling the Database Backups to AWS S3
• Providing data to Sales or Reporting team to their needs
• Migrating the reports as per the requirement
• Responsible for ongoing administration of data and analytics infrastructure
• Environment: CentOS, Python, Shell Script, Java, AWS (EC2, S3, Redshift, RDS, Cloud Watch), MySQL, Nagios, Kafka, Github, Jira, DBVisualizer.

Confidential

Responsibilities:

• Migrated the data from MySQL to HDFS using Sqoop.
• Wrote ETL scripts on Pig, Written udf's for pig.
• Installed and configured multiple Hadoop nodes
• Move the insight data into Teradata using scripts

Confidential

Hadoop/AWS Trainee

Responsibilities:

• Prepared detailed schema, and program specifications from which data base was modified
• Implemented Autoscaling and Elastic Load Balancer
• Trained on HDFS maintenance and administering it through Hadoop-Java API
• MySQL basic administration
• Nagio setup and creation of custom metrics in cloudwatch
• Utilized UNIX shell scripting/programming to build re-usable utilities
• Trained on AWS, Hadoop, Hive, NoSQL Databases, Cloud Operations, Implemented PoC projects