Hadoop Dev/Ops Engineer Resume Foster city, CA - Hire IT People

SUMMARY

5+ years of IT experience, which includes 3 years of Hadoop and 2 years of Linux Administration.
Worked on installation, configuration and maintenance of hadoop cluster.
Manage multipleinfrastructures on AWS, Rackspace,Azure and our own Datacenters.
Managed Hadoop clusters using Cloudera and Hortonworks.
Involved in collecting requirements from business users, designing and implementing data pipelines and ETL workflows end to end.
Experience in using Cloudera Manager and Ambari for Installation and management of Hadoop cluster.
Data migration from existing data stores to Hadoop.
Monitoring and support through Nagios and Ganglia.
Benchmarking Multiple Hadoop clusters to validate teh hardware before installation and after installation to tweak teh Configuration to provide better performance.
Importing and exporting data from different databases like MySQL, Oracle into HDFS and Hive using SQOOP.
Experience in writing scripts for automation.
Migrating applications from existing systems like MySQL, Oracle and Teradata to Hadoop.
Expertise wif Hadoop,Mapreduce, Pig, Sqoop, Oozie and Hive.
Experience in implementing security for Hadoop using Kerberos.
Developed and automated Hive queries on daily basis.
Extensive knowledge on Migration of applications from existing sources.

TECHNICAL SKILLS

Hadoop eco system components: Hadoop, Mapreduce,yarn,hive, pig, impala, sqoop,flume.

Hadoop Clusters: Cloudera and Hortonworks

Tools: Tableau, micro strategy integrations wif hive.

Cloud Environments: Openstack,AWS, Rackspace.

Programming Languages: JAVA, SQL, Unix Shell scripting, HTML.

Monitoring and Alerting: Nagios, Ganglia

Operating Systems: Linux Centos 5,6, Red hat 6.

PROFESSIONAL EXPERIENCE

Confidential, Foster city, CA

Hadoop Dev/Ops Engineer

Responsibilities:

Managing Hadoop clusters prodstaging,qa and dev.
Deploying 100+nodesHadoop Cluster wif Cloudera distribution of Hadoop.
Installed multiple Hadoop clusters using CM and automation scripts.
Monitoring and support through Nagios and Ganglia
Experienced in managing and reviewing Hadoop Log files.
Worked on High Availability for Name Node using ClouderaManager to avoid single point of failure
Build and manage development and testing environments, assisting developers in debugging application issues
Working experience in supporting and deploying in an AWS environment including teh following services: EC2, S3, EMR.
Migrated teh data from production cluster to DR cluster.
Benchmarking Hadoop cluster and tuning Hadoop configurations
Defining and documenting operational processes and monitoring and maintenance procedures
Experience working wif Hadoop Ecosystem (Hive, Pig, Oozie,Sqoop,)
Developing scripts for monitoring teh cluster and data loading.
Supporting users on day - to-day tickets and issues.
Contribute to teh creation and maintenance of system documentation
Deploying EMR cluster for users on demand in amazon elastic MapReduce.
Provided technical support for Level I-III issues via helpdesk and teh telephone.
Took Backup at regular intervals and planned wif a good disaster recovery plan.
Documented and maintain server, network, and support documentation including application diagrams.

Confidential, New Jersey

Hadoop Engineer:

Responsibilities:

Involved in collecting requirements from business users, designing and implementing data pipelines and ETL workflows.
Responsible for performance tuning, backup and recovery wif respect to Hadoop systems.
Involved in transferring data between RDBMS and HDFS using Sqoop.
Responsible for troubleshooting issues in teh execution of MapReduce jobs by inspecting and reviewing log files
Created internal and external Hive tables and defined static and dynamic partitions as per requirement for optimized performance
Effectively used Oozie to develop automatic workflows of Sqoop, MapReduce and Hive jobs
Used Tableau to visualize teh analyzed data
Conducted root cause analysis and worked wif Big Data Analysts, Designers and Scientists in troubleshooting map reduce job failures and issues wif Hive and Map Reduce.

Confidential

Linux/Database system Admin

Responsibilities:

Installing and maintaining teh Linux servers.
Installed Cent OS using Pre-Execution environment boot and Kick start method on multiple servers.
Update system as soon as new version of OS and application software comes out.
Setup securities for users and groups and firewall intrusion detection systems.
Creating new users, Resetting user passwords, Lock/Unlock user accounts.
Monitoring System Metrics and logs for any problems.
Running cron-tab jobs to back up MySQL data.
Involved in Adding, removing, updatinguser accountinformation, resettingpasswords etc.
Maintaining teh RDBMS server and Authentication to required users for databases