We provide IT Staff Augmentation Services!

Hadoop Architect Resume

3.00/5 (Submit Your Rating)

Richardson, TX

SUMMARY:

  • Highly - adaptable Systems Engineer skilled at solving systems-level programming problems. Candidates’ experience includes creation of an Apache Hadoop (Big Data) implementation on a fully-distributed cluster install using HDFS, MapReduce and its various sub-projects, I.E., HBase, Pig, Hive and Oozie.
  • Expert with YARN, Drill, HDFS and MapReduce in a Linux OS.
  • Later performed AWS Hadoop (Big Data) testing using Cloudera Hadoop distribution.
  • Experienced also includes database installations, implementations and modifications of all aspects of RDBMS. An influential team leader.
  • An Electrical Engineer with extensive communication systems integration and Java and Python development background.
  • Earned an MSEE with an emphasis in communications and computer programming. Proven ability to work well under high volume conditions while maintaining exceptional attention to detail.
  • All under an agile environment.
  • Possess excellent multitasking, team-building and interpersonal skills.
  • I am an experienced Hadoop (Big Data) Architect, certified MapR Hadoop Administrator, and familiar with Apache Hadoop Spark, Drill, Hive, Kafka, YARN, if not the entire Hadoop stack.

CORE STRENGTHS:

  • Object-oriented Analysis, Design and Programming
  • Proficient in Hadoop technology including HDFS, MapReduce2, Spark, Kafka and YARN Framework
  • Data Modeling, Database Design and Developer
  • Proficient in MapR & Cloudera distributions Hadoop technology
  • Troubleshooting and Problem Resolution
  • Project Team Leader

TECHNICAL SKILLS:

LANGUAGES: Python, PERL, Java, C/C++, ADA, SQL, PL/SQL

PLATFORMS: Windows XP, Sun Solaris UNIX, LINUX, CENTOS LINUX

DATABASES: Oracle, MySQL, SAP, MS-SQL Server, MS-ACCESS

TOOLS: DOORS, Matlab, LabView, MS-Excel, Regular expression

ADDITIONAL: OOP, UML

PROFESSIONAL EXPERIENCE:

Confidential, Richardson, TX

Hadoop Architect

Responsibilities:

  • Architect new HDP cluster for processing streaming Rx details from stores using Kafka, Spark, Hive feeding into repository HDFS.
  • Multiplex my time performing production support sustaining tasks.
  • Utilized Infoworks to ingest data from EDW.

Confidential, Marlborough, MA

MapR Architect

Responsibilities:

  • Provided leadership architect an enterprise data warehouse (EDW) offload to a MapR distribution Hadoop solution.
  • Identified shortcomings in the EDW system and provided options to accommodate ingestion of data sets earmarked once for the EDW over to a new MapR Hadoop environment.
  • Utilized Sqoop and Python for offloading data from EDW.

Confidential, Plano, TX

Cloudera Hadoop Architecture/Administrator

Responsibilities:

  • Involved in designing and implementing creative solutions in bring Cloudera Big Data DEV, UAT, & PROD clusters On-line and providing enhancements.
  • Originally, I do all my solution architect in Cloud VM sandbox prior to going live, to verify steps are appropriate to achieve correct outcome.
  • Became a Cloudera Hadoop (Big Data) Architect and Administrator.
  • Established test reporting standards using Bash scripts to extract select fields from log files and import into Hive table for use as Hadoop stack tool test results.
  • Am familiar with using Puppet.
  • Designed a Hadoop (Big Data) stack Interoperability suite to certify that Hadoop tools played well together as well as they interface to external components (RDBMS, DMXh, SFTP, etc). Worked on Hadoop tools ranging from batch (data at rest) to NRT, near real time (eg. Spark, Scala, Kafka). Architect the entire operation.

Confidential, McLean, Virginia

MapR Hadoop Administrator/Engineer

Responsibilities:

  • Created a 4 node MapR Hadoop cluster installation at client site in Palo Alto CA.
  • Cluster included MapR-DB and OpenTSDB services.
  • Brought up numerous MapR hadoop clusters in VM environment.
  • Performed data ingestion using Hadoop distcp, Java, Python and Maven.
  • Used OpenTSDB to monitor 20 to 30 misc time series sensor values.
  • Assisted with development of a MapR-DB Garbage Collection prototype as reader/write solution for another client.
  • Development included using the MapR-DB service to create a semi-distributed table using the Hbase Shell along with using MapR C-APIs.

Confidential, Clearwater, FL

Requirements/Design/Validation Engineer

Responsibilities:

  • Wrote/upgraded requirements for a Remote Interface Unit (RIU) for Confidential used on the P-8 Poseidon
  • Used Matlab for accuracy/verification of software, validated (FQT) requirements for software integrated into the RIU.

Confidential, Largo, Florida

Systems Engineer

Responsibilities:

  • Created an Apache Hadoop (Big Data) fully distributed POC cluster install using HDFS, MapReduce and its various sub-projects, Pig, Hive, HBase and Oozie.
  • Sub-Expert with HDFS and MapReduce in a Linux OS. Effort involved installing Hadoop and configuring Hadoop environment using .bashrc & hadoop-env.sh.
  • Configured the Hadoop framework (provide JDK and SSH password-less keys from NameNode to all slave nodes).
  • Configured the NameNode (create the masters and slaves files).
  • Modified core, hdfs and mapred-site.xml files used to configure the nodes.
  • Then format NameNode and verified the configuration.
  • Troubleshot the cluster using log files and the node web user interface to correct errors, and finally shut down each node (decommissioned the DataNodes as well for practice).
  • Used Fs shell, fsck and dfsadmin tools to interact with HDFS. Throughout effort performed unit testing on MapReduce portion under Eclipse IDE to eliminate bugs before entering job onto cluster. Once bugs were eliminated put job onto cluster to verify it worked. Made liberal use of the web user interface to verify all nodes' status in the cluster.
  • A key producer in designing a 'user friendly' Database using ACCESS to replace the ESS log books in ESS Lab.
  • Worked on the ‘next generation’ CEC SDP-S terminal. Responsible for evaluating CEC CCAs that comprise the CEC (Cooperative Engagement Capability) terminal. Ensured that CCAs respective VIDs (Vendor Item Drawing) content met CEC requirements. Interfaced with vendors from UK and Canada to discuss flaws that existed in VIDs and assisted on remedies for corrective action.
  • Successfully developed, implemented, and lead CEP (Cooperative Engagement Processor) FQT (Formal Qualification Testing) of latest upgrade of CEP software in embedded multiprocessor communication system (St. Petersburg, FL). Supervised five Engineer testers whose responsibility was to create and run tests to simulate multi platform CEP activity and ensure the CEP software was meeting system specifications.
  • Conducted daily ‘hot wash’ with testers, customer, and quality engineers present to discuss day’s activities and what testing was planned for the next day.
  • Involved in the integration of various newly acquired businesses (APEX in Tewksbury, MA). Used SAP business application software as the underlying foundation to achieve a successful integration (ETL) into data warehousing their business data of these legacy companies into one Raytheon.
  • Responsible for the design and implementation of the SIAP (Single Integrated Air Picture) database for use in the CEC program. Designed the Oracle instance and schema for this research project using a Solaris UNIX box.
  • Designed and developed software routines for a spread-spectrum frequency hopping communications system (CEC). Designed link budget models for this spread-spectrum frequency hopping communications system using MS-Excel and Matlab. Target was VXWorks OS and development was on a Solaris UNIX OS. Used Labview to control an antenna pedestal feeding INS input.

Confidential, Vandenberg AFB, California

Systems Engineer

Responsibilities:

  • Contributed and completed a retrofit of a ship-based radar system into a land-based tracking radar in Saipan.
  • Range of engineering functions include redesign of the Rf component chain to minimize VSWR and maximize output power.
  • Wrote software for a real-time data handling application.

Confidential, Rolling Meadows, Illinois

Engineer Specialist

Responsibilities:

  • Project leader in design and development of RF ATE station for testing of receiver front-end of AN/ALQ-162 airborne radar set.
  • Effort included RF system analysis, defining test plan, writing major software device drivers, power and I/O Interface circuitry design and procurement of all components and document required to complete objective.
  • Actively participated and directed four engineers and technical support team using minimal resources and under critical time constraints.
  • Responsible for the development and implementation of ATE test stations/fixtures used in the production testing of hybrid circuits and major receiver assemblies of the AN/ALQ-135 surveillance radar system.
  • Led technical team in development of test plan of select RF assemblies. Work included creating test methodology, circuit design, writing software, creating TPSs (Test Program Set), integrating HW/SW and delivering final products to customer.
  • Software developer of exception processing routines for embedded multiprocessor communications system (CEC) using C/C ++ programming language. Developed system test requirements and conducted network engineering testing during field support evaluations. Due to the success of this work, the customer - Confidential - requested additional projects from team.
  • Designed and developed software routines for spread-spectrum frequency hopping communications system. Used ADA programming language to generate embedded software for target multiprocessor in Wind River OS environment. Sun UNIX OS was the development environment.
  • Preform use case analysis to identify classes. This complex effort was recognized by the Confidential command as very successful and launched a Navy program that ultimately became 'defacto' in Navy ship protocol.
  • Developed software for real-time data handling applications using C++. This led to a successful major installation of radar system retrofit in Saipan.

Data Modeling & Database Design and Developer

Confidential

Responsibilities:

  • Designed and installed Oracle instance on a UNIX box as research project (SIAP); used real communication system in combat environment simulation. This effort brought additional research work to team.
  • Designed link budget models of spread-spectrum frequency hopping communications system. Used MS-Excel as link budget tool. Developed spreadsheets for various link components (i.e., receiver, path loss, transmitter, etc.). Created macros to automate and perform 'what if' analysis.
  • Designed and implemented the SIAP (Single Integrated Air Picture) database. Performed Solaris UNIX OS Installation and Oracle 11g installation. Designed Oracle instance and schema for research project. Responsible for system administration duties of UNIX OS and Oracle database.
  • Team member in integration of various acquired businesses (e.g., Hughes, Texas Instruments, Chrysler Defense, E-Systems) into the Raytheon organization (APEX). Used SAP business application software as underlying foundation to achieve successful integration of these companies into one Raytheon. Developer in ‘contracts to cash’ team that wrote software that scrubbed through legacy databases and integrated them to obtain a clean, common Raytheon database.
  • Created an Apache Hadoop (Big Data) fully distributed cluster install using HDFS, MapReduce, Java and its various sub-projects, Pig, Hive and HBase.
  • I am an experienced Hadoop (Big Data) Architect, certified MapR Hadoop Administrator, and familiar with Apache Hadoop Spark, Drill, Hive, Impala, Hbase if not the entire Hadoop stack.
  • Architect, developed, and designed Hadoop framework install for a distributed node environment.

We'd love your feedback!