SUMMARY

• 6+ years of professional IT experience, this includes 4 years of proven experience in Hadoop, Kafka Administration on Cloudera (CDH), Hortonworks (HDP) Distributions and in AWS, Confluent Cloud and on - prem environment’s..
• Experience in Implementing High Availability of Name Node and Hadoop Cluster capacity planning, Experience in benchmarking, performing backup and disaster recovery of Name Node metadata and important and sensitive data residing on cluster.
• Proficient with Shell and scripting languages.
• Configured Elastic Load Balancing (ELB) for routing traffic between zones, and used Route53 with failover and latency options for high availability and fault tolerance.
• Configured Elastic Search for log collections and Prometheus & Cloud watch for metric collections
• Branching, Tagging, Release Activities on Version Control Tools: GitHub.
• Team Player and self-starter possessing effective communication, motivation and organizational skills combined with attention to detail and business process improvements, hard worker with ability to meet deadlines on or ahead of schedules.

TECHNICAL SKILLS

Operating Systems: UNIX, Linux, Windows XP, Windows Vista, Windows 2003 Server

Servers: Web logic server, WebSphere and Jboss.

Programming Languages: Java, Shell Scripting and Python.

Tools: Chef, Puppet, Ruby, terraform, Jenkins and GitHub.

Database: Oracle.

Processes: Incident Management, Release Management, Change Management

Monitoring Tools: Confluent Control Center (C3), Sysdig, RTView, Grafana, Prometheus, Splunk, AEMT, Tivoli and ELK-logging

PROFESSIONAL EXPERIENCE

Kafka Adminstrator

Confidential, Pleasanton, California

Responsibilities:

• As a Kafka Administrator, was responsible for assisting with the design, architecture, implementation, and on-going support for the Kafka application teams.
• I have experience with Kafka installing from Scratch to Prod environments of all the components in both the Confluent and open-source..
• Managed large-scale multi-nodes Kafka cluster environments.
• Handled all Kafka environment builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring.
• Performed high-level, day-to-day operational maintenance, support, and upgrades for the Kafka Cluster.
• Created of key performance metrics, measuring the utilization, performance and overall health of the cluster.
• Research and tested out the different automated approaches for system administration tasks.
• Provided guidance in the creation and modification of standards and procedures.
• Proactively monitored and setup alerting mechanism for Kafka Cluster and supporting hardware to ensure system health and maximum availability.
• Had a deep understanding of the data components - including Cassandra, ElasticSearch, Kafka, Zookeeper, Hadoop, and Spark, and used that understanding to operate and automate properly configured clusters.
• Worked with Engineering to roll out new products and features.
• Develop infrastructure services to support the Kaiser engineering team’s pursuit of a full devops model.
• Work closely with Engineering and Customer Support to troubleshoot time-sensitive Production issues, regardless of when they happen.
• Making sure of critical business data safe, secure, and available.
• Part of Designing, implementing, maintaining, prompting standardized Kafka connector templates
• Experience with production and non-production Kafka platform support
• Supported all stages of the software development life cycle
• Implemented both architectural and infrastructure changes
• Responsible for Rotational on-call responsibilities for Kafka platform and other critical systems
• Assisting in defining principle level guidance for technology use, operational configuration for development and production utilization
• Worked closely with offshore team and external vendors
• Continuously Coordinated with other infrastructure teams, development and application teams
• Implemented Kafka platform connector templates.
• Advised application teams regarding the Kafka solutions and resolved Kafka platform incidents
• Did some POC work for Solace and MQ using Hermes JMS and SolAdmin.
• Also, do support, install and configure the MQ and solace systems.
• Worked on disk space issues in both the non-prod/prod environments by monitoring how fast the disk space will reach to max peak load, and review what is being logged created a long-term fix for this issue (Minimize Info, Debug, Fatal Logs, and Audit Logs).
• Using ansible tower installed and configured the Kafka and other components.
• Working closely with Vendor, in case if any issues comes up.
• Being a Middleware consultant, responsible for Governance, administrative and support for the app teams.

Environment: built: Deployed Confluent Kafka in both IBM Click-to-Cloud in various environments like DEV, QA, UAT, Production and DR.

Kafka Application Engineer

Confidential

Responsibilities:

• Lead and involved while doing the capacity planning, architecture and hardware/software procurement for the Kafka installations.
• Built the PROD cluster in Stretch cluster mode between two data centers which helpful for maximum HA.
• Installed and developed different POC's for different application/infrastructure teams both in Apache Kafka and Confluent open source for multiple clients.
• Installing, monitoring and maintenance of the clusters for the entire environments and support available 24/7.
• Installed both the single node-single broker and multi-node multi broker clusters and encrypted with SSL/TLS, authenticate with SASL/PLAINTEXT, SASL/SCRAM and SASL/GSSAPI (Kerberos) and exclusively 2-way SSL.
• Integrated topic-level security using ACLs and the cluster full up and running for 24/7.
• Performing rolling restart at the time of software upgrade or linux patching activity scheduled.
• Do have experience on installing and running single node Kafka in PKS as well.
• Installed and configured different monitoring tools like Confluent Control center, RTView and Sysdig (for Infrastructure health checks and app team’s data flow).
• Also successfully integrated the logging effort of Kafka to Splunk and ELK.
• Responsible for installing and support for the components like Kafka Connect, Schema-registry and KSQL.
• Do have experience replicating data between two different data centers in a distributed mode using Kafka Connect.
• Supported and worked with the Docker team to install both open source and confluent Kafka single node and enabled security in the DEV environment.
• Installed open source tool “Kafka Tool” for DEV environment and help the application teams to check their consumer lags and monitoring Kafka metrics like adding/viewing the topics, Partitions etc. This is just for POC purposes but no live environment is using it though.
• Successfully generated consumer group lags from Kafka using their API.
• Successfully did set up a no authentication Kafka listener in parallel with Kerberos (SASL) Listener. In addition, I tested non-authenticated user (Anonymous user) in parallel with Kerberos user.
• Installed Ranger in all environments for Second Level of security in Kafka Broker.
• Involved in Data Ingestion Process to Production cluster.
• Installed Docker for utilizing ELK, Influx dB, and Kerberos.
• Good experience in documenting and implementing best practices and optimizing Kafka, Zookeeper and JVM.
• Designed and implemented by configuring Topics in new Kafka cluster in all environment.
• While adding the new node, using the rebalancer migrated partition’s across the cluster successfully.
• Implemented Kafka security features using SSL and without Kerberos. Further, with finer grain security. I set up Kerberos to have users and groups this will enable more advanced security features.
• Experience on DR capabilities of the cluster.
• Created an automated scripts and deployed in enterprise Docker available in the bank and using the postman service able to create/verify the topic(s), ACL(s), logging and Kafka connect status.
• Integrated all Kafka environment clusters with different monitoring tools like Confluent control center, Sysdig, RTView and open source Kafka manager.
• Responsible for weekend changes, upgrades with the confluent Kafka software in rolling restart fashion and Linux patches.

Environment: built: Deployed Confluent Kafka on various environments like POC/Sandbox, INT, SYS, UAT and Production environments.

Big Data Engineer - Hadoop/Cloudera Adminstrator

Confidential, Salt Lake City,Utah

Responsibilities:

• Primary tasks and responsibilities center on O&M support of a Secure (Keberized) Cloudera distribution of Hadoop systems.
• Installing and Configuring Systems for use with Cloudera distribution of Hadoop (consideration given to other variants of Hadoop such as Apache, MapR, Hortonworks, Pivotal, etc.)
• Administering and Maintaining Cloudera Hadoop Clusters Provision physical Linux systems, patch, and maintain them.
• Primarily using Cloudera Manager but some command-line.
• Providing expertise in provisioning physical systems for use in Hadoop.
• Perform Tuning and Increase Operational efficiency on a continuous basis.
• Management and support of Hadoop Services including HDFS, Hive, Impala, and SPARK.
• Person will be responsible to Perform Hadoop Administration on Production Hadoop clusters.
• Monitor health of the platforms and Generate Performance Reports and Monitor and provide continuous improvements.
• Experience in working with cloud infrastructure like Amazon Web Services (AWS) and Rackspace.
• Working closely with development, engineering and operation teams, jointly work on key deliverables ensuring production scalability and stability.
• Develop and enhance platform best practices.
• Ensure the Hadoop platform can effectively meet performance & SLA requirements
• Responsible for support of Hadoop Production environment which includes Hive, YARN, Spark, Impala, Kafka, SOLR, Oozie, Sentry, Encryption, Hbase, etc.
• Perform optimization, capacity planning of a large multi-tenant cluster.
• Worked end to end with platform, Infrastructure and application teams and supported them by 24/7.

Environment: built: Deployed Open source Apache Kafka in Cloudera services provider of environments like SYS, UAT and PROD.

Hadoop and Linux/Unix Administrator

Confidential

Responsibilities:

• Involved in the installation of CDH5 and up-gradation from CDH4 to CDH5
• Cloudera Manager Up gradation from 5.3. to 5.5 version
• Created POC on Hortonworks and suggested the best practice in terms HDP, HDF platform
• Set up Hortonworks Infrastructure from configuring clusters to Node
• Installed Ambari server on the clouds
• Setup security using Kerberos and AD on Hortonworks clusters/Cloudera CDH
• Extensive experience in cluster planning, installing, configuring and administrating Hadoop cluster for major Hadoop distribution’s like Cloudera and Hortonworks.
• Installing, Upgrading and Managing Hadoop Cluster on Hortonworks
• Hands on experience using Cloudera and Hortonworks Hadoop Distributions.
• Responsible for implementation and support of the Enterprise Hadoop environment.
• Responsible for building scalable distributed data solutions using Hadoop.
• Used Scala functional programming concepts to develop business logic.
• Spark scripts by using Scala shell commands as per the requirement.
• Processing the schema oriented and non-schema oriented data using Scala and Spark.
• Developed and designed system to collect data from multiple portal using Kafka and then process it using spark.
• Integrated LDAP Configuration this includes integrating LDAP for securing Ambari servers and manage authorization and securing with permissions against users and Groups.
• Installed and configured Ambari Log Search under the hood it will required a SOLR instance that can collect and index all cluster-generated logs in real time and display them in one interface.
• Installed Ansible 2.3.0 in Production Environment.
• Implemented KNOX, RANGER, Spark and Smart Sense in Hadoop cluster.
• Installed HDP 2.6 in all environments.
• Worked on Micro Strategy report development, analysis, providing mentoring, guidance and troubleshooting to analysis team members in solving complex reporting and analytical problems.
• Extensively used filters, facts, Consolidations, Transformations and Custom Groups to generate reports for Business analysis.
• Leveraged with the design and development of MicroStrategy dashboards and interactive documents using Micro Strategy web and mobile.
• Extracted data from SQL Server 2008 into data marts, views, and/or flat files for Tableau workbook consumption using T-SQL. Partitioned and queried the data in Hive for further analysis by the BI team.
• Managed Tableau extracts on Tableau Server and administered Tableau Server.
• Extensively worked in data Extraction, Transformation and Loading data using BTEQ, Fast load, Multiload from Oracle to Teradata
• Extensively used the Teradata fast load/Multiload utilities to load data into tables
• Used Teradata SQL Assistant to build the SQL queries
• Did data reconciliation in various source systems and in Teradata.
• Involved in writing complex SQL queries using correlated sub queries, joins, and recursive queries.
• Worked extensively on date manipulations in Teradata.
• Tested and Performed enterprise wide installation, configuration and support for hadoop using MapR Distribution.
• Setting up cluster and installing all the ecosystem components through MapR and manually through command line in Lab cluster
• Set up automated processes to archive/clean the unwanted data on the cluster, in particular on Name node and Secondary name node.
• Involved in estimation and setting-up Hadoop Cluster in Linux.
• Extracted the data from oracle using sql scripts, loaded into teradata using fast/multi load, and transformed according to business transformation rules to insert/update the data in data marts.
• Installation and configuration, Hadoop Cluster and Maintenance, Cluster Monitoring, Troubleshooting and certifying environments for production readiness.
• Experience in Implementing Hadoop Cluster Capacity Planning.
• Experience installing, upgrading and configuring RedHat Linux 4.x, 5.x, 6.x using kick start servers and Interactive Installation.
• Responsible for creating and managing user accounts, security, rights, disk space and process monitoring in Solaris, CentOS and Redhat Linux.
• Performed administration and monitored job processes using associated commands.
• Manages systems routine backup, scheduling jobs and enabling cron jobs.
• Maintaining and troubleshooting network connectivity.
• Manages Patches configuration, version control, service pack and reviews connectivity issues regarding security problem.
• Configures DNS, NFS, FTP, remote access, and security management, Server hardening.
• Install the upgrades and manages packages via RPM and YUM package management.
• Logical Volume Management maintenance.
• Experience administering, installing, configuring and maintaining Linux
• Oversee their Linux systems - installing, monitoring and fine-tuning them and sorting out any system or network problems
• Support users on everything from setting up new accounts to updating the DNS, and troubleshoot email environments and apache configurations
• Build new servers, set up test servers for new applications and develop new Linux-based architecture.
• Learned out third-party products, keep on top of advances in technology, and make sure we always work in the most effective ways.