Hadoop Admin Resume
CA
SUMMARY
- 8 years of experience with proven expertise in system development activities including requirement analysis, design, implementation and supporting with emphasis on Object Oriented, Java/J2EE, SQL and Hadoop(HDFS, Map Reduce, Pig, Hive, Impala, Hbase and Sqoop) technologies.
- Experienced in installing, configuring, and administratingHadoopcluster of majorHadoopdistributions.
- Good knowledge on Oozie, Flume and Zookeeper
- Good knowledge on data analysis with R.
- Familiarity with networks: TCP/ IP Firewall, DNS.
- Assisted in designing, development and architecture of Hadoop systems.
- Coordinated with technical teams for installation of Hadoop and third party related applications on systems
- Working experience with large scale Hadoop environments build and support including design, configuration, installation, performance tuning and monitoring.
- Good Knowledge in Hadoop, Pig, Hive, Sqoop, Yarn and designing and implementing Map/Reduce jobs to support distributed data processing and process large data sets utilizing the Hadoop cluster.
- Good knowledge on Apache Hadoop Cluster planning which includes choosing the Hardware and operating systems to host an Apache Hadoop cluster.
- Expertise in developing applications using J2EE Architectures / frameworks like Struts, Spring Framework and SDP (Qwest Communications) Framework.
- Excellent Java development skills using J2EE, J2SE, Servlets, JSP, EJB, JDBC.
- Experience in database design using PL/SQL to write Stored Procedures, Functions, Triggers and strong experience in writing complex queries for Oracle 8i/9i/11g.
- Proficient in writing PL/SQL code (Stored Procedures, Packages, Triggers etc.,) in retrieving data efficiently from database.
- Worked on different OS like UNIX/Linux, Windows XP, and Windows Sound Relational Database concepts and extensively worked with ORACLE, DB2. Very good in writing complex SQL queries and PL/SQL procedures.
- Implemented Java Webservices (JAX - WS and JAX-RS).
- Experience in Agile Engineering practices.
- Good understanding of Scrum methodologies, Test Driven Development and continuous integration
TECHNICAL SKILLS
Bigdata: Hadoop (MapReduce, HBASE, HDFS, Sqoop, Hive, ozie, flume, PIG, Zookeeper).
Distributions: Cloudera, Apache Hadoop.
NoSQL Data Stores: HBASE, CouchDB
Scripts: Pig.
Programming Languages: J2SE, J2EE, SQL, PIG, PL/SQL.
Java Technologies: JDBC.
Frame Works: Jakarta Struts 1.1, Struts 2.0, spring, hibernate.LDAP.
Databases: Oracle 11g/8i/9i, MY SQL, MS SQL server.
IDE’s & Utilities: Eclipse, JCreator and NetBeans.
Web Dev. Technologies: HTML, XML.
Protocols: TCP/IP, UDP, HTTP and HTTPS.
Operating Systems: Linux, Unix, MacOS, WINDOWS 98/00/NT/XP/7/8.
PROFESSIONAL EXPERIENCE
Confidential, CA
Hadoop Admin
Responsibilities:
- Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database and Sqoop.
- Responsible for building scalable distributed data solutions using Hadoop.
- Implemented nine nodes CDH4 Hadoop cluster on Ubuntu LINUX.
- Involved in loading data from LINUX file system to HDFS.
- Worked on installing cluster, commissioning & decommissioning of datanode, namenode recovery, capacity planning, and slots configuration.
- Supported technical team members for automation, installation and configuration tasks.
- Provided technical assistance for configuration, administration and monitoring of Hadoop clusters.
- Implemented a script to transmit sysprin information from Oracle to Hbase using Sqoop.
- Implemented test scripts to support test driven development and continuous integration.
- Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
- Assisted in designing and development of technical architecture, requirements and statistical models using R.
- Coordinated with end users for designing and implementation of analytics solutions using R as per project proposals.
- Responsible to manage data coming from different sources.
- Involved in loading data from UNIX file system to HDFS.
- Load and transform large sets of structured, semi structured and unstructured data
- Cluster coordination services through Zookeeper.
- Experience in managing and reviewing Hadoop log files.
- Installed Oozie workflow engine to run multiple Hive and pig jobs..
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
Environment: Hadoop, HDFS, Pig, Hive, Sqoop, HBase, Shell Scripting, Ubuntu, Linux Red Hat.
Confidential, Sunnyvale CA
Hadoop Admin
Responsibilities:
- Gathered the business requirements from the Business Partners and Subject Matter Experts.
- Involved in installing CDH3 Hadoop Ecosystem components.
- Responsible to manage data coming from different sources.
- Supported Map Reduce Programs those are running on the cluster.
- Assisted in designing, development and architecture of Hadoop systems.
- Coordinated with technical teams for installation of Hadoop and third party related applications on systems
- Involved in HDFS maintenance and loading of structured and unstructured data.
- Installed and configured Pig and also written PigLatin scripts.
- Imported data using Sqoop to load data from MySQL to HDFS on regular basis.
- Developed Scripts and Batch Job to schedule various Hadoop Program.
- Utilized Agile Scrum Methodology to help manage and organize a team of 4 developers with regular code review sessions.
- Weekly meetings with technical collaborators and active participation in code review sessions with senior and junior developers.
Environment: Java Hadoop, MapReduce, HDFS, Hive, Java (jdk1.7), Pig, Linux, XML, MySQL, MySQL Workbench.
Confidential, Salt Lake City, Utah
HADOOP Admin/Developer
Responsibilities:
- Worked on analyzing Hadoop stack and different big data analytic tools including Pig and Hive, Hbase database and Sqoop.
- Implemented nine nodes CDH3 Hadoop cluster on Redhat Hadoop.
- Worked on installing cluster, commissioning & decommissioning of datanodes, namenode recovery, capacity planning, JVM tuning, map and slots configuration.
- Implemented a script to transmit sysprin information from Oracle to Hbase using Sqoop.
- Worked on Hive for exposing data for further analysis and for generating transforming files from different analytical formats to text files.
- Implemented test scripts to support test driven development and continuous integration.
- Worked on tuning the performance of Hive and Pig queries.
- Work with various project teams to identify opportunities to optimize client infrastructure.
- Involved in various POC activity using technology like Map reduce, Hive, Pig, and Oozie.
Environment: Hadoop 1x, HDFS, Hive, Pig, Sqoop, Hbase, Shell Scripting, Ubuntu, Linux RedHat.
Confidential, Westport, Connecticut
Java/J2EE Developer
Responsibilities:
- Responsible for understanding the scope of the project and requirement gathering.
- Developed the web tier using JSP, Struts MVC to show account details and summary.
- Created and maintained the configuration of the Spring Application Framework (IoC).
- Implemented various design patterns - Singleton, Business Delegate, Value Object and Spring DAO.
- Used Spring JDBC to write some DAO classes to interact with the database to access account information.
- Mapped business objects to database using Hibernate.
- Involved in writing Spring Configuration XML files that contains declarations and other dependent objects declaration.
- Used Tomcat web server for development purpose.
- Used Oracle as Database and used Toad for queries execution and also Involved in writing SQL scripts, PL/SQL code for procedures and functions.
- Used CVS, Perforce as configuration management tool for code versioning and release.
- Developed application using Eclipse and used build and deploy tool as Maven.
- Used Log4J to print the logging, debugging, warning, info on the server console.
Environment: Java1.5, J2EE Servlet, JSP, XML, Spring 3.0, Struts 1.1, Hibernate 3.0, Design Patterns, Log4j, CVS, Maven, Eclipse, Apache Tomcat 6, and Oracle 11g.
Confidential, San Fransisco, CA
Java/J2EE Developer
Responsibilities:
- Used HTML, JavaScript, Servlets, and JSP in the presentation tier. Developed business delegates to communicate with the Enterprise Java Beans (EJB) and web services residing on the business tier.
- Responsible and active in the analysis, definition, design, implementation, management, and deployment of full software development life cycle of project.
- Development of the application that was used to create dynamic JSPs, given input from a database table containing information about the contents of the JSP being developed.
- Wrote Servlets programming and JSP scripting for the communication between web browser and server.
- Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC
- Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
- Coded different deployment descriptors using XML. Generated Jar files are deployed on WebLogic Server.
- Involved in the development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.
- Developed an API to write XML documents from a database. Utilized XML and XSL Transformation for dynamic web-content and database connectivity.
- Involved in the development of presentation layer and GUI framework in JSP. Client Side validations were done using JavaScript.
Environment: J2EE, Java 1.4, XML, JAXB, JNDI, SOAP, WSAD 5.1, Websphere Application Server, Linux, SQL Server 2000, Rational Rose, and Oracle 9i
Confidential
Java/J2EE Developer
Responsibilities:
- Used Hibernate ORM tool as persistence Layer - using the database and configuration data to provide persistence services (and persistent objects) to the application.
- Responsible for developing DAO layer using Spring MVC and configuration XML’s for Hibernate and to also manage CRUD operations (insert, update, and delete).
- Implemented Dependency injection of spring frame work.
- Developed and implemented the DAO and service classes.
- Developed reusable services using BPEL to transfer data.
- Participated in Analysis, interface design and development of JSP.
- Configured log4j to enable/disable logging in application.
- Developed Rich user interface using HTML, JSP, AJAX, JSTL, Java Script, JQuery and CSS.
- Implemented PL/SQL queries, Procedures to perform data base operations.
- Wrote UNIX Shell scripts and used UNIX environment to deploy the EAR and read the logs.
- Implemented Log4j for logging purpose in the application.
- Involved in code deployment activities for different environments.
- Implemented agile development methodology.
Environment: Java, Jest, SOA Suite 10g (BPEL), Struts, Spring, Hibernate, Web services(JAX-WS), JMS, EJB, Web logic 10.1 Server, JDeveloper, Sql Developer, HTML, LDAP, Maven, XML, CSS, JavaScript, JSON, SQL, PL/SQL, Oracle, CVS and UNIX/Linux.
