We provide IT Staff Augmentation Services!

Big Data Engineer Resume

Bethesda, MarylanD

TECHNICAL EXPERTISE IN THE FOLLOWING AREAS:

R, Hadoop Ecosystem 3.0: Pig, Map Reduce, Scoop, Hive, HBase, Couchbase 4.5, 6.0, Oracle 12.2, PL/SQL, Pearl, JAVA Custom Connectors, NoSQL, JSON, Python, BSON, UNIX / Linux shell scripting, Linux 7.5, Solaris 11.1,, MongoDB 4.0, DB2 LUW V11, SQL Server 2017, MySQL 8.0.12 - Replication

PROFESSIONAL EXPERIENCE:

Confidential, Bethesda, Maryland

Big Data Engineer

Responsibilities:

  • Assisted in designing, development and architecture of Hadoop and HBase systems.
  • Coordinated with technical teams for installation of Hadoop and third related applications on systems.
  • Formulated procedures for planning and execution of system upgrades for all existing Hadoop clusters.
  • Supported technical team members for automation, installation and configuration tasks.
  • Suggested improvement processes for all process automation scripts and tasks.
  • Provided technical assistance for configuration, administration and monitoring of Hadoop clusters.
  • Conducted detailed analysis of system and application architecture components as per functional requirements.
  • Participated in evaluation and selection of new technologies to support system efficiency
  • Assisted in creation of ETL processes for transformation of data sources from existing RDBMS systems.
  • Designed and developed scalable and custom Hadoop solutions as per dynamic data needs.
  • Coordinated with technical team for production deployment of software applications for maintenance.
  • Provided operational support services relating to Hadoop infrastructure and application installation.
  • Supported technical team members in management and review of Hadoop log files and data backups.
  • Participated in development and execution of system and disaster recovery processes.
  • Formulated procedures for installation of Hadoop patches, updates and version upgrades.
  • Automated processes for troubleshooting, resolution and tuning of Hadoop clusters.

Confidential, Gaithersburg, MD

Hadoop/Couchbase Administrator

Responsibilities:

  • Managed several Hadoop and Couchbase clusters in development and production environments across data centers at Iron Mountain, Phoenix Arizona, Ashburn VA and Frederick Md.
  • Developed Map Reduce programs for generation of reports supporting Business Intelligence
  • Business intelligence support via import / export of column data into HDFS/Hive Partitioned Tables - Hadoop via Sqoop
  • Built and configured log data loading into HDFS using Flume.
  • Performed Importing and exporting data into HDFS and Hive using Sqoop.
  • Managed cluster coordination services through Zoo Keeper.
  • Provisioning, installing, configuring, monitoring, and maintaining HDFS
  • Supported code/design analysis, strategy development and project planning.
  • Created software design documents for modification of current OLTP NED system for refactoring of several Oracle databases objects to support functionality modifications.
  • Performed in depth analysis of legacy system residing as a sub-module to the NED system.
  • Built Hadoop clusters on HDP 2.2 and HDP 2.3
  • Designed and implemented 3 node Couchbase clusters across data centers
  • Worked with engineering software developers to investigate problems and make changes to the Hadoop environment and associated applications.
  • Performance tuning on Hadoop. Couchbase and Oracle 12.2 RAC
  • Expertise in recommending hardware configuration for Hadoop cluster
  • Installing, Upgrading and Managing Hadoop Cluster on Hortonworks
  • Trouble shooting many cloud related issues such as Data Node down, Network failure and data block missing.
  • Major Upgrade from HDP 2.2 to HDP 2.3.
  • Managing and reviewing Hadoop and HBase log files
  • Built and configured log data loading into HDFS using Flume.
  • Performed Importing and exporting data into HDFS and Hive using Sqoop.
  • Managed cluster coordination services through Zoo Keeper.
  • Provisioning, installing, configuring, monitoring, and maintaining HDFS, Yarn, HBase, Flume, Sqoop, Oozie, Pig and Hive
  • Practiced recovering from node failures and troubleshooting common Hadoop cluster issues.
  • Scripting Hadoop package installation and configuration to support fully-automated deployments.
  • Supporting Hadoop developers and assisting in optimization of map reduce jobs, Pig Latin scripts, Hive Scripts, and HBase ingest required.
  • Performed HDFS cluster support and maintenance tasks like Adding and Removing Nodes without any effect to running jobs and data.
  • Oversaw the design, construction and implementation of an Oracle 12.2.0.1 Real Application Cluster (RAC). This project included the construction of a Proof of Concept 2 node cluster, two - 2 node clusters in development and test, and a production 2 node cluster.

Confidential, Frederick, Md

Big Data Engineer / Database Manager

Responsibilities:

  • Developed Map reduce program to extract and transform the data sets and resultant dataset were loaded to Cassandra and vice versa using kafka 2.0.x.
  • Used Spark API 1.4.x over Cloudera Hadoop YARN 2.5.2 to perform analytics on data in Hive.
  • Exploring with the Spark 1.4.x, improving the performance and optimization of the existing algorithms in Hadoop 2.5.2 using Spark Context, SparkSQL, Data Frames.
  • Implemented Batch processing of data sources using Apache Spark 1.4.x.
  • Developed analytical components using Spark 1.4.x, Scala 2.10.x and Spark Stream.
  • Imported data from different sources like HDFS/Hbase 0.94.27 to Spark RDD.
  • Developed Spark scripts by using Scala Shell commands as per the requirement.
  • Developed numerous MapReduce jobs in Scala 2.10.x for Data Cleansing and Analyzing Data in Impala 2.1.0.
  • Loaded and extracted the data using Sqoop 1.4.6 from Oracle 12.1.0.1 into HDFS.
  • Senior Manager / Database developer in charge of monitoring database activity conducted by several contracting companies responsible for maintaining existing database systems and implementing new systems. Oversee a vast field of players including Accenture, Atos, Atlas, Starwood, IBM, Linux and Oracle to evaluate database design, day-to-day activity and various other tasks associated with database systems.
  • Databases under my supervision included SQLServer, Couchbase 4.5, DB2 LUW 10.5, MySQL 5.7, Oracle 12.2.0.1
  • Oversaw the design, construction and implementation of an Oracle 12.2.0.1 Real Application Cluster (RAC). This project included the construction of a Proof of Concept 2 node cluster, two - 2 node clusters in development and test, and a production 2 node cluster.
  • Upgraded 30+ Oracle Databases from 11.2.0.4 to 12.1.0.2.
  • Designed and implemented Disaster Recovery Site using Oracle Data Guard
  • Designed, developed and implemented Korn shell / Perl programs to provide real time monitoring of resource intensive SQL, alert log scanning and database logon attempts
  • Created security profiles for users, developed and maintained procedures and processes to provide proactive database security
  • Utilized AWR reports for review of SQL plans. Coded SQL / PL/SQL / Perl programs for study of optimizer plan baselines and general database performance
  • Monitored, optimized, and allocated physical and virtual storage for database systems via NetApp SAN, implemented NetApp block level snapshot backups
  • Supported 50-person offshore development team during migration of applications from Web Based to Mobile Based platform.
  • Developed database architectures, coding standards, and quality assurance policies and procedures
  • Designed and implemented RMAN backup at master and disaster recovery sites.
  • Assisted in developing the Support Services allocation model in the Hyperion Profitability and Cost Management tool

Confidential

Database Manager / Developer

Responsibilities:

  • Upgraded 9.2.0 databases to 10.2.0 on Solaris 9, Solaris 10
  • Designed and implemented Data Guard for disaster recovery and developed Disaster Recovery Plan
  • Developed Operations and Database Users Manuals and Backup/Recovery Guides covering all aspects of database maintenance and usage.
  • Provided long-term strategic goals for OLTP and Data Warehouse consisting of several terabytes of data and the on-line processing of 30 million transactions per month.
  • Supported classified Security and Risk Assessment System at TSA client site in Pentagon City
  • Provide 24/7 support for multiple Internet/Intranet RAC enabled 9i databases supporting 11i/9ias applications for the Transportation Safety Administration (TSA)
  • Developed RMAN backup and recovery programs
  • Evaluated database performance via Statspack and implemented changes as needed
  • Maintained multiple Apache Web Servers in complex multi-network environment.
  • Provided general database design, implementation, and administration to assure highest reliability and availability of IT services for all enterprise databases

Hire Now