Hadoop Administrator Resume
SUMMARY
- Over 10 years of experience with designing, development, maintenance and testing of business applications with an exposure to different Software Systems and Platforms. (4 years Hadoop, 3 years Java/J2EE Technologies and 4 years IBM Mainframe.)
- Very well understanding of Hadoop architecture and Map Reduce (MR1) & YARN Frameworks.
- Hands - on experience with Hadoop Cluster administration, configuration management, monitoring, debugging, and performance tuning.
- Hands on experience in installing, configuring and using Hadoop ecosystem components like HDFS, MAPREDUCE, YARN, Pig, Hive, HCatalog, Hbase, Zoo Keeper, Oozie, Sqoop, Flume, Giraph, PIVOTAL HAWQ, HDB 2.0, PXF, MYSQL, WANdisco Fusion
- Experience in setup, configuration and management of security for hadoop clusters using Kerberos and integration with LDAP/AD at an Enterprise level
- Experience in setup,configuration and management of JAAS security for hadoop web console using SPNEGO and integration with LDAP/AD at an Enterprise level
- Experience in setup,configuration and management of Apache Sentry for Role-based authorization and privilege validation for Hive and Impala Services.
- Experience in setup,configuration and management of Apache Ranger for Role-based authorization and privilege validation for HDFS
- Configured Apache Ranger and Ambari for AD Login
- Experience in validating WANdisco Fusion active active replication solution for Hadoop DR
- Expertise in developing MapReduce jobs in Java
- Background with traditional databases such as Oracle, SQL Server, ETL tools / processes and data warehousing architectures
- Hands on experience in all aspects of software development life cycle (SDLC).
- Hands on experience in different software development approaches such as Waterfall & Agile iterative models.
- Expertise with full Project Life Cycle and Project Management, Standards and Configuration Management, QA Management, Team Management, Documentation, Implementation, Coding etc.
- Expertise in the areas include excellent communication skills, ability to work as part of a team and a clear understanding of business rules.
- Strong interpersonal skills with the experience of working in a multi-cultural environment.
TECHNICAL SKILLS
Operating System: Red Ha Linux, Sun/OS (UNIX), Unix, DOS/VSE, VM/CMS, Windows
Database: DB2 (UDB), IMS, PC Focus 5.0, ms Access, SQL Server, Oracle, DBase IV (Home Usage)
Languages: Cobol, Cobol II, MF Cobol II Workbench, Focus (Reports),Java, HTML, DHTML, XML, XSL, Basic, Pascal, Cobol, MS/C++, PL/SQL
Hardware: IBM ES/9000, IBM 308X, IBM 43XX, IBM PS/2,PC Workstations, Sun Workstations
OLTP: CICS/ESA, IMS DC, MFS
Windows (GUI): OS/2 PRES MGR, MOTIF, X-Windows, MS/Windows, Visual Basic, Power Builder
Other: SQL Loader, SQL*Forms 4.5, SQL*Report, SQL*Plus, SQL*Net, WordPerfect, MS/Excel, MS/Office
Software/Ancillary: AWT (Abstract Window Toolkit), Applets, JFC/ Swing, Java Scripts, Servlets,JDBC/ODBC, RMI, Multi Threading, Java Beans, JBuilder, Visual Age, Web Sphere, Web Logic, Rational Rose98,TSO/ISPF,OS/JCL,VSAM,ISPFDialogManager,AUTOSYS, Checkpoint/Restart,BTS,Platinum,Changeman, REXX, CLIST, Endeavor, QMF, Librarian, CCF/MMF, UNIX,Panavalet, Endevor, AUTOSYS, Data AGER,SyncSort,AnalystDataWorkbench (ADW),File-Aid,CA-7,XPEDITER,IEFCoolGEN,Gentran,LotusNotes,Excel,MS-Word, Oracle Enterprise Service Bus (ESB), Web Services, SOA(Service Oriented Architecture), WSDL(WizDel),TOAD,AMDOCS CES 8.1, HADOOP,HDFS,MapRedue, YARN,Pig, Hive, HBase,HCatalog, Flume, Sqoop, ZooKeeper, Oozie, Ganglia, Nagios, WANdisco Fusion, DconE PIVOTAL HAWQ 1.3, HDB 2.0, PXF, MYSQL, Kafka, Spark, Solr, Impala, Hue, Storm
PROFESSIONAL EXPERIENCE
Hadoop Administrator
Confidential
Responsibilities:
- Involved in installation, configuration, supporting and managing Hadoop clusters.
- Involved in Hadoop Cluster administration that includes commissioning & decommissioning of datanode,capacity planning, slots cofiguration,performance tunning, cluster monitoring and troubleshooting.
- Involved in installing Hadoop ecosystem components
- Configured Kerberos RPC and SPNEGO security authentication for hadoop clusters
- Configured JAAS LDAP Authorization Login Module for hadoop web console
- Configured Apache Sentry for Role-based authorization and privilege validation for Hive and Impala Services
- Configured Apache Ranger for Role-based authorization and privilege validation for HDFS
- Configured Apache Ranger and Ambari for AD Login
- Configured Capacity Scheduler to provide service-level agreements for multiple users of a cluster.
- Worked with application teams to install Hadoop updates, patches, version upgrades as required.
- Involved in Cluster coordination services through Zookeeper.
- Manage and review data backups and log files.
- Installed Oozie workflow engine to run multiple Hive and Pig jobs
- Managing nodes on Hadoop cluster connectivity and security.
- Execute system and disaster recovery processes as required
- Involved in implementation of WANdisco Fusion Active Active replication solution for DR
- Exposure to HDFS Transparent Encryption, HDFS Federation, HADOOP KMS
- Involved in HDFS File system management and monitoring
Environment: Java (JDK 1.7), Python 2.6, HDFS, MAPREDUCE,YARN,Pig, Hive, Hcatalog, HBase, Flume, Sqoop, OoozieZooKeeper, Giraph1.0.0, PIVOTAL HAWQ 1.3, HDB 2.0, PXF, MYSQL,WANdisco Fusion 2.9, RHEL 6.6.
Hadoop Developer
Confidential
Responsibilities:
- Reviewed Project Artifacts to develop business workflows
- Developed MapReduce jobs in java for data cleaning and preprocessing.
- Experienced in defining job flows
- Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
- Importing data into HDFS using Flume and Sqoop
- Analyzing data with Hive, Pig and Hadoop Streaming.
- Used Pig as ETL tool to do transformations, event joins, filter bot traffic and some pre-aggregations before storing the data onto HDFS
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
Environment: Java (JDK 1.6), Eclipse Helios 3.6, MapReduce, HDFS, Pig, Hive, Hcatalog, HBase, Flume, Sqoop, OoozieZooKeeper, Ubantu 11.04, RHEL 6.
Java Developer
Confidential
Responsibilities:
- Involved in development and testing of User Interface screens
- Involved in server side coding using Servlets and JSPs for dynamic content display.
- Developed javascript functions for validations.
- Involved in coding Session beans that contain business logic.
- Involved in writing SQL to get data for complex views.
Environment: JAVA, JDK1.6, J2EE, EJB, WEB SPHERE, ORACLE 8I, INFORMIX, JDBC,JFC/SWINGVISUAL AGE JAVA, UNIX, JBUILDER,MAPINFO, CCC-HARVEST,RATIONAL ROSE 98
Mainframe Programmer
Confidential
Responsibilities:
- Responsible for Interacting with Business partners in requirements gathering.
- Responsible for writing detailed design specification document and implementing all business rules as per design standards.
- Involved in Design, Develop and deployment of application architecture,use cases, flowcharts, application flow, prototypes and pseudo code.
- Involved in writing DB2 stored procedures
- Involved in writing Cobol Programs using CICS, DB2
- Responsible for production support and resolving the issues as needed.
Environment: IBM ES/9000, VS COBOL II, JCL, DB2, IMS DB, VSAM, MVS/ESA,MS OFFICE, XPEDITORFILE-AID, TICTO, DATAAGER,SUPERC, CHANGEMAN
