We provide IT Staff Augmentation Services!

Hadoop Engineer Resume

0/5 (Submit Your Rating)

CA

PROFESSIONAL SUMMARY:

  • 6 years experience in Software Development Life Cycle (SDLC) and Software Engineering including Requirement Gathering, Analyzing, Designing, Implementing, Testing, Support and Maintenance
  • Have experience with Hadoop Ecosystem including HDFS, MapReduce, PIG, HIVE and HBase.
  • Have experience in installing, configuring and administrating Hadoop cluster for major Hadoop distributions.
  • Have experience in working with Hadoop in stand - alone, pseudo and distributed modes.
  • Experience in importing and exporting data from different RDBMS like MySql, Oracle and SQL Server into HDFS and Hive using Sqoop.
  • Experience in developing custom Map-Reduce programs using Apache Hadoop to perform Data Transformation and analysis as per requirement.
  • Strong competency in HIVE Schema design, Data imports and Analysis.
  • Hands-on experience in writing Pig Latin scripts, working with grunt shell and job scheduling with Oozie.
  • Experience in Design & Development, tuning and maintenance of NoSQL databases.
  • Experience in working with NoSQL databases like HBase.
  • Strong understanding of Data Modeling and experience with Data Cleansing, Data Profiling and Data analysis.
  • Excellent experience in ETL analysis, designing, developing, testing and implementing ETL processes including performance tuning and query optimizing of databases.
  • Excellent experience in extracting source data from Sequential files, XML files, Excel files, transforming and loading it into the target data warehouse.
  • Expertise in Java/J2EE technologies such as Core Java, JDBC, HTML, JavaScript, JSON
  • Have noledge of Python programming.
  • Proficiency in programming with different IDE's like Eclipse, NetBeans and Canopy.
  • Involved in database design, creating Tables, Views, Stored Procedures, Functions, Triggers and Indexes.
  • Have hands on experience on various DB platforms like Oracle, MySQL, DB2 and MS SQL Server.
  • Experience in deploying applications in heterogeneous application servers TOMCAT and WebLogic.
  • Good understanding of service oriented architecture (SOA) and web services like XML, XSD, XSDL, SOAP
  • Experience in object oriented analysis and design (OOAD), unified modeling language (UML) and Agile Methodologies.

TECHNICAL SKILLS:

Hadoop Ecosystem: HDFS, MapReduce, Hive, Pig, Zookeeper, Sqoop, Oozie, Flume

Languages: C, Java, PigLatin, Shell Scripting

Operating Systems: Windows Variants,UNIX, LINUX

Database: Oracle 11g/10g/9i, MySQL, Postgresql

IDE Tools: Eclipse, Net Beans, SQL Developer, MS Visual Studio

Version Control: Git

Software Tools: MS Office Suite(Word, Excel, Project),MS Visio, Rational Rose, Apache Maven

Web Technologies: HTML, CSS, XML, JSP

Monitoring Tools: Ganglia, Nagios and Cloudera Manager, Ops Center

Application Servers: Apache Tomcat

WORK EXPERIENCE:

Hadoop Engineer

Confidential, CA.

Responsibilities:

  • Having 7 plus years of professional IT experience which includes 2.5 years of proven experience inHadoopAdministration using Cloudera(CDH) and Hortonworks(HDP) Distributions and, 4.5 years in oracle Database Administration.
  • Knowledge of multiple distributions/platforms (Apache,Cloudera,Hortonworks).
  • Experienced in installation, configuration, supporting and monitoring 100+nodeHadoopcluster using cloudera manager and Hortononwworks Ambari distributions.
  • Have experience in installing. configuring, performance tuning and administratingHadoopcluster for majorHadoopdistributions like CDH 4, HDP 2.2.0
  • Experienced in mappingHadoopclusters and services using cloudera manager and Hortonworks ambari.
  • Experience in upgrading HortonworksHadoopHDP2.2.0 and Mapreduce 2.0 with YARN in Multi Clustered Node environment.
  • Extensive noledge onHadoopHDFS architecture and MRv1, MRv2 (YARN) framework.
  • Experienced in optimizing Hbase running on multi node cluster.
  • Imported logs from web servers with Flume to ingest the data into HDFS.
  • Created Hive tables and loaded the data into tables and query data using HQL.
  • Manage & review ofHadooplog files.
  • Installed and configured Spark on multimode environment.
  • Experienced in Rebalancing a HDFS Cluster
  • Extensive noledge in using job scheduling and monitoring tools like Oozie and Zookeeper.
  • Performance Metrics and Reporting. Fine tuning of Apache.
  • 4.5 years of experience as Oracle DBA (12C, 11G and 10G) on various platforms SUN Solaris, RHEL, Oracle Enterprise Linux and Windows
  • Installed and Tested databases on 12C to help Migrate from 11G to 12C.
  • Specialized in Migration of Oracle databases from Oracle 8i, 9i, 10G to the latest releases - Oracle 12C and 11G.
  • Strong Troubleshooting skills, understanding RAC Approach.
  • Proficient in Oracle DBA - Installation of Oracle RAC, ASM and disaster recovery data guard setup and Streams configuration. Designing & Testing of database, administration, backup & recovery strategy implementation according to the volatility of the data, user management, performance monitoring/tuning and database security.
  • Experience on Oracle Identity Management Tools.
  • Exposure to Basic Architecture of 11G OAS, Installation, up gradation, Cloning, Patch Application of databases of all versions.
  • Excellent interpersonal, collaboration, and problem solving skills

Big Data Engineer

Confidential, CA.

Responsibilities:

  • Installed and configured various components ofHadoopecosystem and maintained their integrity
  • ManagedHadoopclusters: setup, install, monitor, maintain.
  • Planning for production cluster hardware and software installation on production cluster and communicating with multiple teams to get it done.
  • Designed, configured and managed the backup and disaster recovery for HDFS data.
  • Commissioned DataNodes when data grew and decommissioned when the hardware degraded.
  • Migrated data across clusters using DISTCP.
  • Experience in collecting metrics forHadoopclusters using Ganglia and Ambari.
  • Experience in creating shell scripts for detecting and alerting problems system.
  • Monitored multiplehadoopclusters environments using Ganglia and Nagios. Monitored workload, job performance and capacity planning using Ambari.
  • Performed data analytics in Hive and tan exported dis metrics back to Oracle Database using Sqoop.
  • Designed workflow by scheduling Hive processes for Log file data which is streamed into HDFS usingFlume.
  • Conducting root cause analysis and resolve production problems and data issues.
  • Installed and configured Hive, Pig, Sqoop and Oozie on the HDP 2.2.0 cluster.
  • Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Namenode utilizing zookeeper services.
  • Implemented HDFS snapshot feature.
  • Performed a Major upgrade in production environment from HDP 1.3 to HDP 2.2.0
  • Worked with big data developers, designers and scientists in troubleshooting map reduce job failures and issues with Hive, Pig and Flume.
  • Configured custom interceptors in Flume agents for replicating and multiplexing data into multiple sinks.
  • Administrating Tableau Server backing up the reports and providing privileges to users.
  • Worked on Tableau for generating reports on HDFS data.
  • Installed Ambari on existingHadoopcluster.

Environment: Hortonworks Data Platform (HDP1.3 & HDP 2.2.0), Hive, Pig Latin Ambari, Cassandra, Tableau and ParAccel

Linux Administrator

Confidential

Responsibilities:

  • Installation, configuration, maintenance, administration, and support on Solaris / Redhat Linux.
  • Responsible for maintaining the integrity and security of the enterprise UNIX (Linux /Solaris) servers and systems.
  • Installation and configuration of HA environment using Sun or VERITAS Cluster.
  • Image machines using Jumpstart /Kickstart to install Solaris 10 and Red Hat Enterprise Linux.
  • Installation and configuration of Solaris Zones.
  • Maintains a disaster recovery plan. Creates backup capabilities adequate for the recovery of data and understands concepts and processes of replication for disaster recovery.
  • Maintains DNS, NFS, and DHCP, printing, mail, web, and FTP services for the enterprise.
  • Manages UNIX account maintenance including additions, changes, and removals.
  • User administration for all the NIS users.
  • Forecast storage needs. Work with the site management to determine future disk requirements.
  • Works with the application teams and other IT department personnel to coordinate system software changes and support application changes
  • Unix - to-Windows interoperability and configurations.
  • Debug and correct installed system software as required.
  • Automating system tasks using Puppet.
  • Configuring NFS, NIS, DNS, Auto-mounter and disk Space management on SUN servers.
  • Troubleshooting issues related to DNS, NIS, NFS, DHCP, SENDMAIL on Linux and Solaris Operating Systems.
  • Working noledge on the TCP/IP protocols RSH, SSH, RCP, SCP.
  • Operating system support - Windows RHEL4/5/6, SLES10/11.
  • Install and Configure Virtual Machines (VM), VM tools on ESX boxes from the GUI and console.

Environment: Solaris, Redhat, SENDMAIL, VERITAS Volume Manager, Sun and VERITAS Clusters, Shell Scripting

Confidential

Java Developer

Responsibilities:

  • Installation, Configuration, upgradation and administration of Sun Solaris, Red - Hat Linux.
  • User account management and support.
  • Jumpstart &Kick-start OS integration, DDNS, DHCP, SMTP, Samba, NFS, FTP, SSH, LDAP integration.
  • Responsible for configuring and managing Squid server in Linux.
  • Configuration and Administration of NIS environment.
  • Managing file systems and disk management using Solstice Disksuite.
  • Trouble shooting the system and end user issues.
  • Responsible for configuring real time backup of web servers.
  • Log file was managed for troubleshooting and probable errors.
  • Responsible for reviewing all open tickets, resolve and close any existing tickets.
  • Document solutions for any issues that have not been discovered previously.

Environment: Sun Solaris 2.6/7, SUN Ultra Enterprise … SUN Ultra … Windows NT 4.0, RHEL 3.x.

We'd love your feedback!