We provide IT Staff Augmentation Services!

Associate Data Technologies Engineer Resume

5.00/5 (Submit Your Rating)

Minnetonka, MN

SUMMARY

  • Over 6+ years of experience in IT industry which includes 3 year of experience in AWS cloud infrastructure, Elasticsearch ELK, Cassandra, KAFKA and Chef.
  • Hands on expertise in Writing Chef Cookbooks to modularize and optimize end - product configuration. Developed Chef for server provisioning and automating infrastructure
  • Involved in implementing Chef Recipes for Deployment on build on internal Data Centre Servers. Also re-used and modified same Chef Recipes to create a Deployment directly into Amazon EC2 instances. Sound knowledge on real time data streaming solutions using Apache Spark Streaming, Kafka.
  • Experience in working on version control systems like Subversion and GIT Stash used Source code management client tools like SVN, GIT, GitHub, etc.
  • Implemented Continuous Integration & Continuous Deployment using various CI Tools like Jenkins. working experience in using JIRA, Service Now as issue tracking and ticketing tool.
  • Versatile team player with excellent analytical, communication & interpersonal skills with ability to quickly adapt to new technologies and project environments.
  • Excellent interpersonal and communication skills, creative, research-minded, technically competent and result-oriented with problem solving and leadership skills.
  • Comprehensive knowledge of Software Development Life Cycle and experience with the software development methodologies like Agile and Waterfall.
  • Completely automated Cassandra cluster deployments on AWS using Terraform templates, Cloud formation templates and puppet.
  • Administered Cassandra cluster using Datastax OpsCenter and monitored CPU usage, memory usage and health of nodes in the cluster.
  • Experienced in provisioning and managing multi-datacenter Cassandra cluster on public cloud environment Amazon Web Services (AWS) - EC2.
  • Imported data from various resources to the Cassandra cluster using Java APIs.
  • Strong understanding of internal processes of NoSQL approach.
  • Optimized the Cassandra cluster by making changes in Cassandra properties and Linux (Red Hat) OS configurations.
  • Working closely with Datastax to resolve issues on cluster using ticketing mechanism.
  • Configured Performance Tuning and Monitoring for Cassandra Read and Write processes for fast I/O operations and low latency time.
  • Performed Stress and Performance testing to benchmark the cluster
  • Administered Cassandra cluster using Datastax OpsCenter and monitored CPU usage, memory usage and health of nodes in the cluster.
  • Configured accordingly to achieve maximum throughput and execution time based on the benchmarking results.
  • Configured, Documented and Demonstrated inter node communication between Cassandra nodes and client using SSL encryption
  • Experienced in storing the analyzed results into the Cassandra cluster.
  • Developed toolset to automatically create ASM disks/diskgroups, RAC setup, and DB creations.
  • Developed toolset to deploy Oracle Dataguard automatically.
  • Implemented Transparent Data Encryption (TDE) for Oracle databases at tablespace level for encrypting the data at rest. Implemented Oracle listener SSL for encrypting data at transit.
  • Implemented Goldengate trail file encryption for securely transferring the GG trail files across data center.

TECHNICAL SKILLS

Operating Systems: Centos, Linux, Ubuntu, Unix and Windows

Methodologies: Agile-Scrum, Waterfall

Build Tools: Maven, ANT

Source Control Tools: Subversion, GIT

CI/CD Tools: Jenkins, Chef, Puppet

CM Tools: Chef, Puppet

Programming Language: Python, Bash, Shell Scripting

Data Base: Oracle, My SQL, MongoDB, Cassandra

Cloud: Amazon web Services (AWS), Microsoft Azure

PROFESSIONAL EXPERIENCE

Confidential, Minnetonka, MN

Associate Data Technologies Engineer

Responsibilities:

  • Identifying optimal hardware configuration for Distributed Data technologies like Elasticsearch RELK stack, Cassandra and Kafka.
  • Maintaining and developing the Elasticsearch Clusters and managing the ELK Stack.
  • Also involved in Installing Opendistro and Perftop in new version of Elasticsearch.
  • Creating the Templates for the indices in Elasticsearch. Automating Retentions in Nonprod.
  • Managing ILM in Elasticsearch V7 clusters. And automating the scripts using curator.
  • Identifying software settings for performant implementation for ELK Stack.
  • Maintaining different versions for non-production/production deployment 20%. And Automating scripts by using Rundeck and curator.
  • Performing Stress and Performance testing to benchmark of the cluster.
  • Administered Cassandra cluster using Datastax OpsCenter and monitored CPU usage, memory usage and health of nodes in the cluster.
  • Configured accordingly to achieve maximum throughput and execution time based on the benchmarking results.
  • Develop and test Chef automation scripts to run the configurations in Elasticsearch.
  • Deploy and maintain the automation scripts and Upgrade Path.
  • Identifying upgrade path for major version changes in Elasticsearch, Cassandra and kafka.
  • Using Datos for Backup and Restoring the data.
  • Using Reaper for scheduling the Repairs in Cassandra.
  • Conducting feasibility of upgrade path to major version in Elasticsearch, Cassandra and kafka.
  • Implementing fixes to production issues and identify root cause to
  • Proactively identifying performance issues and create and implement solutions and Evaluate Emerging Technologies.
  • Evaluating new technologies in marketplace and see the fit for technology gaps in current practices like Perftop for analyzing the metrics in Elasticsearch Open distro.
  • Conducting POC of new technologies to understand the capabilities and implications of deployment of technologies.
  • Developed Lag Checker in Kafka to get the Lag in Kafka Pipeline.
  • Managing Kafka Manager and Solving the issues in Kafka.
  • Involved in ELK Upgrade and developed the Logstash pipeline. And Created 3 different pipeline.
  • Worked on installing Datos in Cassandra to store the data in Cloud in S3 bucket.
  • Involved in implementing the Reaper to schedule the repairs on the Cassandra.
  • Working closely with Datastax to resolve issues on cluster using ticketing mechanism.
  • Configured Performance Tuning and Monitoring for Cassandra Read and Write processes for fast I/O operations and low latency time.
  • Performed Stress and Performance testing to benchmark the cluster
  • Administered Cassandra cluster using Datastax OpsCenter and monitored CPU usage, memory usage and health of nodes in the cluster.

Environment: Elasticsearch, Cassandra, Kafka, Jenkins, Git, Seyogit, Docker, Curator, Chef, Rundeck, Datastax, Datos, Reaper, Kafka Manager, Lagchecker, DataStax Enterprise 5.1, Cassandra 3.1.2, Linux-CentOS, MySQL, Windows Server- Manager, VS Code 1.17, Shell scripting, Devcenter, OpsCenter.

Confidential, San Francisco, CA

Technology Engineer

Responsibilities:

  • Involved in capacity planning and requirements gathering for multi datacenter Cassandra cluster.
  • Involved in the process of designing Cassandra Architecture.
  • Involved in NoSQL database design, integration and implementation.
  • Installed, Configured, Tested Datastax Enterprise Cassandra multi-node cluster which has 4 Datacenters and 5 nodes each.
  • Installed and configured Cassandra cluster and CQL on the cluster.
  • Involved in the process of data mover for disaster recovery platforms Backup and recovery.
  • Involved in database deployments, capacity planning, monitoring multi datacenters, performance tuning, and troubleshooting.
  • Knowledge on set up Cassandra wide monitoring scripts and alerting system.
  • Knowledge on bootstrapping, removing, replicating the nodes in Cassandra and Solr clusters.
  • Experienced in upgrading the existing Cassandra cluster to latest releases.
  • Experience in Apache Spark with Scala
  • Experienced in provisioning and managing multi-datacenter Cassandra cluster on public cloud environment Amazon Web Services(AWS) - EC2.
  • Imported data from various resources to the Cassandra cluster using Java APIs.
  • Strong understanding of internal processes of NoSQL approach.
  • Optimized the Cassandra cluster by making changes in Cassandra properties and Linux (Red Hat) OS configurations.
  • Working closely with Datastax to resolve issues on cluster using ticketing mechanism.
  • Configured Performance Tuning and Monitoring for Cassandra Read and Write processes for fast I/O operations and low latency time.
  • Performed Stress and Performance testing to benchmark the cluster
  • Administered Cassandra cluster using Datastax OpsCenter and monitored CPU usage, memory usage and health of nodes in the cluster.
  • Configured accordingly to achieve maximum throughput and execution time based on the benchmarking results.
  • Configured, Documented and Demonstrated inter node communication between Cassandra nodes and client using SSL encryption
  • Experienced in storing the analyzed results into the Cassandra cluster.
  • Used Github version control for tagging the new versions.
  • Involved in the Migration of data from one database to another database.
  • Knowledge on applying updates and maintenance patches for the existing clusters.
  • Scheduled repair and cleanup process in production environment.

Environment: Cassandra, Jenkins, Git, Seyogit, Docker, Curator, Rundeck, Datastax, Datos, Reaper, chef, Java, DataStax Enterprise 5.1, Cassandra 3.1.2, Linux-CentOS, MySQL, Windows Server-Manager, VS Code 1.17, Shell scripting, Devcenter, OpsCenter.

Confidential

Software Developer

Responsibilities:

  • Completely automated Cassandra cluster deployments on AWS using Terraform templates, Cloud formation templates and puppet.
  • Involved in data modeling for applications on Cassandra and migrating databases from Oracle to Cassandra.
  • Written CQL and python Scripts on DataStax for Service implementation
  • Cassandra Cluster planning which includes Data sizing estimation, and identify hardware requirements based on the estimated data size and transaction volume. Cassandra deployments in Single, multi datacenter with spark and solr enabled.
  • Designing, creating, executing and monitoring spark job on cassandra cluster for validation, loading from Oracle database.
  • Good experience in using DataStax Enterprise (DSE) and OpsCenter for cluster administration and monitoring
  • Working closely with Application team to resolve issues related to spark, cql and loading issues, Writing CQL queries as per business requirements.
  • Analyze Solr indexing requirements and evaluate the impact to overall system.
  • Query tuning & performance tuning on cluster & suggesting best practice for developers.
  • Ingestion into Kafka using Spark on Cassandra for Micro batching.
  • Working closely with Cassandra loading activity on history load and incremental loads from Oracle Databases and resolving loading issues and tuning the loader for optimal performance.
  • COE and given training on DataStax Caasandra, data Modeling and Spark, Solr.
  • Involved in defining and documenting best practices for Cassandra, migrating application to Cassandra database from the legacy platform for Choice, upgraded Cassandra from 2.0 to 2.2.x
  • Architected the database migration from managed services to Confidential owned data center in Virginia.
  • Completely automated Oracle installation for standalone and RAC with patch bundles using Oracle cloning.
  • Developed toolset to automatically create ASM disks/diskgroups, RAC setup, and DB creations.
  • Developed toolset to deploy Oracle Dataguard automatically.
  • Implemented Transparent Data Encryption (TDE) for Oracle databases at tablespace level for encrypting the data at rest. Implemented Oracle listener SSL for encrypting data at transit.
  • Implemented Goldengate trail file encryption for securely transferring the GG trail files across datacenters.
  • Architected the migration plan for SAP database migrations from Oracle 11.x on HP-UX to Oracle 12c on OEL 7.x.
  • Developed a fully automated toolset for migrating underlying storage for Oracle ASM diskgroups.
  • Designed and developed the backup toolset for Oracle database backups, the tool is a fully customizable using config file, centralised scheduling, uses Oracle catalog, web based reporting and centralised logging, error reporting, backup performance monitoring, daily backup status reporting. Some of the multi terabyte DB’s backed from DR using the DG replicated data.
  • RMAN backups are automatically synced to AWS S3 bucket for long term retention and automatically expire after retention period.
  • Implemented EMC DD boost for Oracle RMAN backups.
  • Architected and implemented Delphix VDB’s for deploying 600 databases in dev.
  • Scripted a toolset that real time alerts for any file changes across the database landscape, the alerting includes details on what changed and who changed, and where they logged in from.
  • Implemented Oracle auditing across the Oracle DB environments, also written some customized triggers that audit/alert based on certain business requirements.
  • Written a php/python centric web based tool that can be used by business to upload an encrypted zip file, that gets populated to Oracle database once the user has been authenticated across the corp ldap servers.
  • Written Oracle triggers/procedures that captures the change data, convert to Json and transfers to NOSQL database.
  • Written php, shell scripts that gathers the usage/stats data through REST calls from EMC XtremIO clusters and publishes as webpages
  • Written web based tool to selectively reset application passwords, as well as in bulk.
  • Written a modular web based tool for managing and monitoring Oracle RAC scan-ip’s, RMAN backup’s, ASM DG usage, DB growth patterns, tablespace monitoring, filesystem, inode monitoring, db logins monitoring, DG lag monitoring, RMAN backups to S3, db lock monitoring and few others.

Environment: Elasticsearch, Cassandra, Jenkins, Git, Seyogit, Docker, Curator, Rundeck, Datastax, Datos, Reaper, chef, Java, DataStax Enterprise 5.1, Cassandra 3.1.2, Linux-CentOS, MySQL, Windows Server- Manager, VS Code 1.17, Shell scripting, Devcenter, OpsCenter.

Confidential

Software Developer

Responsibilities:

  • Planning, deploying, monitoring, and maintaining Amazon AWS cloud infrastructure consisting of multiple EC2 nodes and VMWare Vm's as required in the environment.
  • Defined AWS Security Groups which acted as virtual firewalls that controlled the traffic allowed reaching one or more AWS EC2 instances. Created monitors, alarms and notifications for EC2 hosts using CloudWatch.
  • Worked on operational support activities to ensure availability of customer websites hosted on AWS cloud infrastructure using Virtual private cloud (VPC) and public cloud.
  • Well Versed with Configuring Access for inbound and outbound traffic RDS DB services, DynamoDB tables, EBS volumes to set alarms for notifications or automated actions.
  • Created and Maintained Chef Recipes and Cookbooks to simplify and expedite deployment of applications and mitigate user error.
  • Installed and used Chef Server Enterprise on premise, workstation and bootstrapped the nodes using knife and automated by writing ruby scripts in Chef Recipes and Cookbooks with test-kitchen/chef spec.
  • Written Chef Cookbooks, recipes using ruby to automate installation of Middleware Infrastructure like Apache Tomcat, JDK, and configuration tasks for new environments.
  • Deployment and implementation of Chef for infrastructure as code initiative.
  • Implemented Chef Server and components installations, including cert imports, increase chef license, creating admins and users.
  • Involved in chef infra maintenance including backup/monitoring/security fixes.
  • Implemented Splunk infrastructure and used Splunk to capture and analyze data from various layers load balancers, web servers, and application servers.
  • Configuring Chef to build up services and applications on the instances once they have been configured using cloud formation.
  • Designing and implementing fully automated server build, management, monitoring and deployment solutions spanning multiple platforms, tools and technologies including Jenkins.

Environment: Jenkins, Git, Seyogit, Docker, Curator, Rundeck, Datastax, Datos, Reaper, chef, Java, DataStax Enterprise 5.1, Cassandra 3.1.2, Linux-CentOS, MySQL, Windows Server-Manager, VS Code 1.17, Shell scripting, Devcenter, OpsCenter.

We'd love your feedback!