Hadoop Engineer Resume
CA
PROFESSIONAL SUMMARY:
- 6 years experience in Software Development Life Cycle (SDLC) and Software Engineering including Requirement Gathering, Analyzing, Designing, Implementing, Testing, Support and Maintenance
- Have experience with Hadoop Ecosystem including HDFS, MapReduce, PIG, HIVE and HBase.
- Have experience in installing, configuring and administrating Hadoop cluster for major Hadoop distributions.
- Have experience in working with Hadoop in stand - alone, pseudo and distributed modes.
- Experience in importing and exporting data from different RDBMS like MySql, Oracle and SQL Server into HDFS and Hive using Sqoop.
- Experience in developing custom Map-Reduce programs using Apache Hadoop to perform Data Transformation and analysis as per requirement.
- Strong competency in HIVE Schema design, Data imports and Analysis.
- Hands-on experience in writing Pig Latin scripts, working with grunt shell and job scheduling with Oozie.
- Experience in Design & Development, tuning and maintenance of NoSQL databases.
- Experience in working with NoSQL databases like HBase.
- Strong understanding of Data Modeling and experience with Data Cleansing, Data Profiling and Data analysis.
- Excellent experience in ETL analysis, designing, developing, testing and implementing ETL processes including performance tuning and query optimizing of databases.
- Excellent experience in extracting source data from Sequential files, XML files, Excel files, transforming and loading it into the target data warehouse.
- Expertise in Java/J2EE technologies such as Core Java, JDBC, HTML, JavaScript, JSON
- Have noledge of Python programming.
- Proficiency in programming with different IDE's like Eclipse, NetBeans and Canopy.
- Involved in database design, creating Tables, Views, Stored Procedures, Functions, Triggers and Indexes.
- Have hands on experience on various DB platforms like Oracle, MySQL, DB2 and MS SQL Server.
- Experience in deploying applications in heterogeneous application servers TOMCAT and WebLogic.
- Good understanding of service oriented architecture (SOA) and web services like XML, XSD, XSDL, SOAP
- Experience in object oriented analysis and design (OOAD), unified modeling language (UML) and Agile Methodologies.
TECHNICAL SKILLS:
Hadoop Ecosystem: HDFS, MapReduce, Hive, Pig, Zookeeper, Sqoop, Oozie, Flume
Languages: C, Java, PigLatin, Shell Scripting
Operating Systems: Windows Variants,UNIX, LINUX
Database: Oracle 11g/10g/9i, MySQL, Postgresql
IDE Tools: Eclipse, Net Beans, SQL Developer, MS Visual Studio
Version Control: Git
Software Tools: MS Office Suite(Word, Excel, Project),MS Visio, Rational Rose, Apache Maven
Web Technologies: HTML, CSS, XML, JSP
Monitoring Tools: Ganglia, Nagios and Cloudera Manager, Ops Center
Application Servers: Apache Tomcat
WORK EXPERIENCE:
Hadoop Engineer
Confidential, CA.
Responsibilities:
- Having 7 plus years of professional IT experience which includes 2.5 years of proven experience inHadoopAdministration using Cloudera(CDH) and Hortonworks(HDP) Distributions and, 4.5 years in oracle Database Administration.
- Knowledge of multiple distributions/platforms (Apache,Cloudera,Hortonworks).
- Experienced in installation, configuration, supporting and monitoring 100+nodeHadoopcluster using cloudera manager and Hortononwworks Ambari distributions.
- Have experience in installing. configuring, performance tuning and administratingHadoopcluster for majorHadoopdistributions like CDH 4, HDP 2.2.0
- Experienced in mappingHadoopclusters and services using cloudera manager and Hortonworks ambari.
- Experience in upgrading HortonworksHadoopHDP2.2.0 and Mapreduce 2.0 with YARN in Multi Clustered Node environment.
- Extensive noledge onHadoopHDFS architecture and MRv1, MRv2 (YARN) framework.
- Experienced in optimizing Hbase running on multi node cluster.
- Imported logs from web servers with Flume to ingest the data into HDFS.
- Created Hive tables and loaded the data into tables and query data using HQL.
- Manage & review ofHadooplog files.
- Installed and configured Spark on multimode environment.
- Experienced in Rebalancing a HDFS Cluster
- Extensive noledge in using job scheduling and monitoring tools like Oozie and Zookeeper.
- Performance Metrics and Reporting. Fine tuning of Apache.
- 4.5 years of experience as Oracle DBA (12C, 11G and 10G) on various platforms SUN Solaris, RHEL, Oracle Enterprise Linux and Windows
- Installed and Tested databases on 12C to help Migrate from 11G to 12C.
- Specialized in Migration of Oracle databases from Oracle 8i, 9i, 10G to the latest releases - Oracle 12C and 11G.
- Strong Troubleshooting skills, understanding RAC Approach.
- Proficient in Oracle DBA - Installation of Oracle RAC, ASM and disaster recovery data guard setup and Streams configuration. Designing & Testing of database, administration, backup & recovery strategy implementation according to the volatility of the data, user management, performance monitoring/tuning and database security.
- Experience on Oracle Identity Management Tools.
- Exposure to Basic Architecture of 11G OAS, Installation, up gradation, Cloning, Patch Application of databases of all versions.
- Excellent interpersonal, collaboration, and problem solving skills
Big Data Engineer
Confidential, CA.
Responsibilities:
- Installed and configured various components ofHadoopecosystem and maintained their integrity
- ManagedHadoopclusters: setup, install, monitor, maintain.
- Planning for production cluster hardware and software installation on production cluster and communicating with multiple teams to get it done.
- Designed, configured and managed the backup and disaster recovery for HDFS data.
- Commissioned DataNodes when data grew and decommissioned when the hardware degraded.
- Migrated data across clusters using DISTCP.
- Experience in collecting metrics forHadoopclusters using Ganglia and Ambari.
- Experience in creating shell scripts for detecting and alerting problems system.
- Monitored multiplehadoopclusters environments using Ganglia and Nagios. Monitored workload, job performance and capacity planning using Ambari.
- Performed data analytics in Hive and tan exported dis metrics back to Oracle Database using Sqoop.
- Designed workflow by scheduling Hive processes for Log file data which is streamed into HDFS usingFlume.
- Conducting root cause analysis and resolve production problems and data issues.
- Installed and configured Hive, Pig, Sqoop and Oozie on the HDP 2.2.0 cluster.
- Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Namenode utilizing zookeeper services.
- Implemented HDFS snapshot feature.
- Performed a Major upgrade in production environment from HDP 1.3 to HDP 2.2.0
- Worked with big data developers, designers and scientists in troubleshooting map reduce job failures and issues with Hive, Pig and Flume.
- Configured custom interceptors in Flume agents for replicating and multiplexing data into multiple sinks.
- Administrating Tableau Server backing up the reports and providing privileges to users.
- Worked on Tableau for generating reports on HDFS data.
- Installed Ambari on existingHadoopcluster.
Environment: Hortonworks Data Platform (HDP1.3 & HDP 2.2.0), Hive, Pig Latin Ambari, Cassandra, Tableau and ParAccel
Linux Administrator
Confidential
Responsibilities:
- Installation, configuration, maintenance, administration, and support on Solaris / Redhat Linux.
- Responsible for maintaining the integrity and security of the enterprise UNIX (Linux /Solaris) servers and systems.
- Installation and configuration of HA environment using Sun or VERITAS Cluster.
- Image machines using Jumpstart /Kickstart to install Solaris 10 and Red Hat Enterprise Linux.
- Installation and configuration of Solaris Zones.
- Maintains a disaster recovery plan. Creates backup capabilities adequate for the recovery of data and understands concepts and processes of replication for disaster recovery.
- Maintains DNS, NFS, and DHCP, printing, mail, web, and FTP services for the enterprise.
- Manages UNIX account maintenance including additions, changes, and removals.
- User administration for all the NIS users.
- Forecast storage needs. Work with the site management to determine future disk requirements.
- Works with the application teams and other IT department personnel to coordinate system software changes and support application changes
- Unix - to-Windows interoperability and configurations.
- Debug and correct installed system software as required.
- Automating system tasks using Puppet.
- Configuring NFS, NIS, DNS, Auto-mounter and disk Space management on SUN servers.
- Troubleshooting issues related to DNS, NIS, NFS, DHCP, SENDMAIL on Linux and Solaris Operating Systems.
- Working noledge on the TCP/IP protocols RSH, SSH, RCP, SCP.
- Operating system support - Windows RHEL4/5/6, SLES10/11.
- Install and Configure Virtual Machines (VM), VM tools on ESX boxes from the GUI and console.
Environment: Solaris, Redhat, SENDMAIL, VERITAS Volume Manager, Sun and VERITAS Clusters, Shell Scripting
Confidential
Java Developer
Responsibilities:
- Installation, Configuration, upgradation and administration of Sun Solaris, Red - Hat Linux.
- User account management and support.
- Jumpstart &Kick-start OS integration, DDNS, DHCP, SMTP, Samba, NFS, FTP, SSH, LDAP integration.
- Responsible for configuring and managing Squid server in Linux.
- Configuration and Administration of NIS environment.
- Managing file systems and disk management using Solstice Disksuite.
- Trouble shooting the system and end user issues.
- Responsible for configuring real time backup of web servers.
- Log file was managed for troubleshooting and probable errors.
- Responsible for reviewing all open tickets, resolve and close any existing tickets.
- Document solutions for any issues that have not been discovered previously.
Environment: Sun Solaris 2.6/7, SUN Ultra Enterprise … SUN Ultra … Windows NT 4.0, RHEL 3.x.