SUMMARY:

• A system engineer/architect with extensive experience in Linux, devOps, networking, data infrastructure, data analytics and Hadoop technologies.
• 11+ years of experience in system engineering, DBA/Linux admin and data analytics
• 3 years of experience with Hadoop ecosystem: installation, configuration, security and administration of different Hadoop components
• HDP Certified Administrator (HDPCA)
• Hands - on Experience with setting, configuring & monitoring of Hadoop cluster using Hort onworks (Ambari) and Cloudera (Cloudera Manager and Navigator)
• Security of data at rest and in motion using Ranger, Navigator Encrypt, SSL/TLS. Access Control using ACLs, Knox, Apache Sentry. Authentication using Kerberos
• Hands-on experience in Linux administration (CentOS, Red Hat), including installing, upgrading and configuring Linux based operating systems, for several functions
• Worked with Puppet Enterprise tool for patch updates, new user credentials. Experienced with Docker, Chef, and Jenkins.
• Automated regular jobs using Shell Scripting (bash, ksh)

TECHNICAL SKILLS:

RHEL: CentOS 5.0/6.0/7.0

Hadoop Ecosystems: HDFS, YARN, MapReduce, Spark, Hive, HBase, Zookeeper, Pig, Sqoop, Kafka, Oozie, Knox, Ranger, Sentry, Kerberos

Hadoop Monitoring: Ambari, Cloudera manager, Ganglia, Nagios

Web Cloud: Amazon EC2, MS Azure, Rackspace, Cloudbreak

Storage: SAN, iSCSI, SNAP, NAS, AWS EBS/S3/Redshift

RDBMS: Oracle, MS SQL, MySQL 5.1/5.5/6, AWS RDS

Automation: Chef, Puppet, Jenkins, Bash, ksh

WORK EXPERIENCE:

System Engineer

Confidential, San Francisco, CA

Responsibilities:

• Worked on setting up a securehadoop cluster with sparkSQL on AWS EMR
• Installed and configured Hive2, SparkSQL, druid, on the cluster
• Installed and tested Kerberos security on the cluster. Created detailed wiki for security implementation for ad-hoc user access
• Worked on cluster performance monitoring and tuning/optimization, particularly for Spark. Managed upto 20 small clusters (4-10 nodes) spun up on-demand for specific tasks

System Engineer

Confidential, Chicago, IL

Responsibilities:

• Worked on CDH 5.6 cluster setup and configuration using Cloudera manager
• Installed and configured Hive, Impala, Spark, Sqoop, Hbase, sentry and other ecosystem tools on the cluster
• Worked on cluster performance monitoring and tuning/optimization for various ecosystem tools using Cloudera manager.
• Helped data analyst to benchmark SQL like tools on the cluster. Comparison of Hive, Impala, SparkSQL performance on different datasets.

System Engineer

Confidential, Houston,TX

Responsibilities:

• Worked on installation and upgrade of 150 nodes Hortonworks HDP 2.3 cluster using Ambari
• Installed and configured different Hadoop ecosystem components including Zookeper, Hbase, Sqoop, Hive, Spark, Kafka, Oozie, Knox, Kerberos etc
• Setup, configuration and management of security for Hadoop cluster using Kerberos and integration with LDAP/AD at an enterprise level
• Configured API layer security using Knox gateway
• Configured and deployed Ranger and Ranger KMS (key management) for auditing.
• Performed cluster management, configuration, performance monitoring, and troubleshooting using Hortonworks Ambari
• Worked on setting up Falcon, and worked with Data Engineering team to setup data piplelines, troubleshoot them, and migrate across environments
• Provided support as Spark administrator, Helped users with Spark tuning and debugging. Familiar with spark architecture and processing framework
• Performed routine Linux administration tasks, including networking, shell scripting
• Assembled Puppet Master, and agents. Managing changes through Puppet master server on its clients. Used automation, configuration management and deployment tools such as Chef, Puppet and Ansible
• Set up users and groups, ACL file permissions, assigned users and groups quota
• Performed a Major upgrade in production environment from HDP 2.2 to HDP 2.3
• Worked on data migration from cluster to cluster using Distcp
• Setup Cloudbreak Deployer and tested cluster setup using Ambari Blueprints
• Worked with developers in designing scalable, supportable infrastructure

Sr. System Engineer

Confidential, Houston, TX

Responsibilities:

• Established a new 20 node Hortonworks Hadoop cluster with Ambari
• Analyzed business requirements and provided appropriate tools and capacity planning for cluster expansion
• Installed and configured different Hadoop ecosystem components including Hive, Hue,Rangeretc. using Ambari
• Setup user level security using Ranger, configured ACLs for users and groups
• Performed Hadoop performance tuning, ecosystem and OS optimization
• Provided support for day-to-day activities in Hadoop environment

System/Product Engineer

Confidential, Houston, TX

Responsibilities:

• Deployed 20 node Hadoop cluster using Cloudera distribution and Cloudera manager
• Involved in Hadoop cluster maintenance activities - commissioning and decommissioning nodes, upgrades in Hadoop cluster components
• Cloudera security and data encryption: Key management using Gazzang zTrustee (now called Cloudera Navigator Key Trustee Server)
• Deployed and implemented HBase for data logging and analytics
• Installed and configured Sqoop. Imported data frequently to HDFS using Sqoop.
• Installed and configured hive. Used HiveQL and Pig for map-reduce jobs
• Installed and generated data analytics using RHadoop
• Managed and administered Linux network for in-house data warehouse. Managed user accounts, permissions, files, processes, data and media for the server

System Engineer

Confidential, Houston, TX

Responsibilities:

• Engineering Applications: Performed preliminary and detail design of various offshore and subsea systems for data logging, data collection and storage systems
• Worked as package engineer for communication systems. Developed package execution plan. Created project and functional specifications
• Worked on Confidential control design philosophies. Created PID and reports.
• Generated detail design specifications, PID, HAZOP, HAZID, layouts, solid models and drawings for Confidential systems
• Created procurement specifications and packages, Co-ordinated RFQ process and performed technical evaluations to select vendors.
• Corporate Applications: Legacy application CIS ( Corporate Intranet System) written in VBScript and ASP.NET. Managed reports implemented in Microsoft's SSRS (SQL Server Reporting Services). Manage and maintain company's source control and server (TFS).

Staff Engineer

Confidential, Houston, TX

Responsibilities:

• Worked on many aerospace and oilfield modules analysis using Ansys and Nastran codes. Used C and Confidential code to run on Linux solver server
• Configured solver server for multiple users on Linux. Managed user accounts, permissions, files, processes, data and media for the server.
• Setup Linux server authentication with Active Directory using Winbind and Samba.
• Optimized solver process parameters, OS configuration, and terminal resources for optimum runtime. Managed process scheduling and priorities.
• Generated Ansys models for modules. Setup optimization runs on the Linux server. Performed design, modeling, analysis and reports.
• Designed and analyzed a metering skid for using Ansys for optimum performance. Generated reports with recommendations
• Implemented Sharepoint 2007 for centralized document management
• Windows administration for Active Directory, MS Access, custom VBA apps on MS SQL Server

Systems Engineer

Confidential, Houston, TX

Responsibilities:

• Developed and deployed port programs in C and Confidential for communications, data logging and analysis
• Worked on in-line testing of ID (interface detector), and calibration meter. Integrated sensors and streamlined continuous reading for data analytics
• Designed a fluid level monitoring system consisting IDs. Configured ports and data communication hardware. Created port programs, communication BOM, PID and reports.
• Supported applications on MS SQL Server on Windows and Linux servers. Applications used MS Visual Studio, Visual Basic and SQL server