Hadoop Developer Resume
San Ramon, CA
SUMMARY
- Over 7+ years of professional IT experience and over 2 Years of Big Data Ecosystem experience in ingestion, storage, querying, processing and analysis of bid data.
- In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, JobTracker, TaskTracker, NameNode, DataNode and MapReduce concepts
- Experience with configuration of Hadoop Ecosystem components: Hive, HBase, Pig, Sqoop, Mahout, Zookeeper
- Experience in building, maintaining multiple Hadoop clusters (prod, dev etc.,) of different sizes and configuration and setting up the rack topology for large clusters
- Experience in optimization of Map reduce algorithm using combiners and partitioners to deliver the best results
- Proficient in using Cloudera Manager, an end to end tool to manage Hadoop operations
- Followed Test driven development of Agile, Water Fall and RUP Methodology to produce high quality software
- Expertise in design and development of various web and enterprise applications using various technologies like JSP, Servlets, Struts, Hibernate, Spring, JDBC, EJB, JMS, JSF, XML, IBM MQ Series, AJAX, SOAP and Web Services.
- Proficient in Working with Various IDE tools including Eclipse Galileo, IBM Rational Application Developer (RAD), IBM Web
- Sphere Studio Application Developer (WSAD), and IBM Visual Age for Java (VAJ).
- Expertise in developing distributing business applications using EJB implementing Session beans for business logic, Entity beans for persistence logic and Message driven beans for asynchronous communication
- Good working experience on Application Servers like WebSphere and WebLogic with IDEs like WSAD5.1, RAD6.0, RAD7.0, Eclipse and JBuilder.
- Experience in Database design, Entity relationships, Database analysis, Programming SQL, Stored procedure’s PL/ SQL, Packages and Triggers in Oracle and SQL Server on Windows and UNIX.
- Basic MySQL administration skills
- Strong SQL skills & working knowledge with Teradata, IBM DB2
- Worked on different OS like UNIX/Linux, Windows NT, Windows XP, and Windows 2K.
TECHNICAL SKILLS
Big Data: HDFS, Hive, Pig, HBase, Sqoop, mahout, Hadoop components (JT, TT, ZK) Oracle LDAP, Kickstart, Datameer
Languages: Java, C/C++, Python, XML Shell scripting, Python, COBOL
Web Technologies: HTML, XML, JavaScript, J query, AJAX
Frame works: Spring, Hibernate, Struts
Servers: IBM WebSphere, WebLogic, Tomcat, and Redhat Satellite Server
IDE Tools: IBM WebSphere Studio Application Developer (WSAD), My Eclipse, Eclipse.
Version Control: CVS, Tortoise SVN
Database: Oracle, DB2, MS - SQL Server, MySQL, MS-Access
Operating Systems: Windows 95/98/2000/XP/Vista/7, Macintosh, Unix.
Monitoring & Reporting: Nagios, Ganglia, Custom shell scripts
Databases/ETL: MySQL, Teradata v 12.x, DB2, Informatica v 8.x, Teradata ETL Utilities, DB2
IBM Mainframes: JCL, IBM DB2, BMC utilities for DB2, FILEAID
PROFESSIONAL EXPERIENCE
Confidential, San Ramon, CA
Hadoop Developer
Responsibilities:
- Installation and administration of RHEL 4.0/5.0, Solaris 9/10, AIX 5.3/6.1 and Suse 10.x.
- Configured kickstart server and updating/applying patches to the servers using Red hat Satellite server.
- Planning and implementing system upgrades including hardware, operating system and periodical patch upgrades.
- Applied appropriate support packages/patches to maintain system integrity.
- Performed capacity analysis, monitored and controlled disk space usage on systems.
- Monitored system activities and fine-tuned system parameters and configurations to optimize performance and ensure security of systems.
- Responsible for architecting Hadoop clusters with CDH3.
- Experience in Installation and configuration of Cloudera distribution Hadoop 2, 3, NameNode, Secondary NameNode, JobTracker, TaskTrackers and DataNodes.
- Strong Experience in Installation and configuration of Hadoop ecosystem like HBase, Flume, Pig, Sqoop.
- Expertise in Hadoop cluster task like Adding and Removing Nodes without any effect to running jobs and data.
- Install, Configure and Manage of Hadoop Cluster spanning multiple racks using automated tools like puppet
- DRBD implementation of NameNode Replication to avoid single point of failure.
- Manage and review Hadoop Log files.
- Set up automated 24x7x365 monitoring and escalation infrastructure for Hadoop cluster using Nagios and Ganglia
- Load log data into HDFS using Flume.
- Scheduling cron jobs for file system check using fsck and also running the balancer for uniform load on the nodes
- Data scrubbing and processing with Oozie.
- Perform Infrastructure services (DCHP, PXE, DNS, KICKSTART, and NFS).
- Support data analysis projects using Elastic Map Reduce on the Amazon Web Services (AWS) cloud.
- Provide Technical support for the Research in Information Technology program
- Manage and upgrade Linux and OS X server systems.
- Install and upgrade Plone, Zope and Apache web servers.
- Responsible for installation, configurations and management for Linux Systems
- Jumpstarting Solaris 9, 10 on Sun servers, configuring servers. Configured/Maintained Zones, LDOMs, ZFS file system Sun Solaris 10 Servers.
- Management and monitoring of various services using Service Management Facility and Service Administration.
- Installing, configuring VERITAS Netback up 6.5 on Linux, Solaris servers, creating backup polices.
- Designing Firewall rules for new servers to enable communication with application, Oracle 10g servers.
- Working on Volume management, Disk Management, software RAID solutions using VERITAS Volume manager & Solaris Volume Manager. File system Tuning and growing using VERITAS File System (VxFS), coordinated with SAN Team for storage allocation and Disk Dynamic Multi path.
- Worked on resolving production issues and documenting Root Cause Analysis and updating the tickets using BMC Remedy.
- Configuring, adding Oracle ASM disks to database server. Creating, managing raw volumes using LVM.
- Organized various meetings with teams as mentioned above for tracking their requirements as inputs to get this done and designed the plan/steps involved and took user approvals.
- Setting up network environments using TCP/IP, NIS, NFS, DNS, SNMP agents, DHCP and Proxy.
- Installation, maintenance and administration of Oracle and Db2 Server on Sun Servers.
Environment: Solaris 2.9 / 2.10, Red Hat Linux AS/EL 4.0, AIX 5.2, 5.3, Sun E10k, E25K, E4500, SunFire V440/880, DMX 3 & DMX4, CDH3, Cloudera Manager, HBase, flume, pig, JDK 1.5,1.6 21,24,VxFS 4.1, VxVM 4.1, SVM.
Confidential, Los Angeles, CA
Hadoop Developer
Responsibilities:
- Worked extensively in creating MapReduce jobs to power data for search and aggregation
- Designed a data warehouse using Hive
- Worked extensively with Sqoop for importing metadata from Oracle
- Extensively used Pig for data cleansing
- Created partitioned tables in Hive
- Worked with business teams and created Hive queries for ad hoc access.
- Evaluated usage of Oozie for Workflow Orchestration
- Mentored analyst and test team for writing Hive Queries
- Gained very good business knowledge on health insurance, claim processing, fraud suspect identification, appeals process etc.
Environment: Hadoop, MapReduce, HDFS, Hive, Java (jdk1.6), Hadoop distribution of Hortonworks, Oozie, Oracle 11g/10g
Confidential, Chesapeake, VA
Hadoop Developer
Responsibilities:
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS
- Developed the Pig UDF’S to pre-process the data for analysis
- Develop Hive queries for the analysts
- Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig
- Generated reports using Pentaho report designer
- Cluster co-ordination services through ZooKeeper
- Job management using Fair Scheduler
- Managed and reviewed Hadoop log files
Environment: Hadoop, MapReduce, HDFS, Hive, Java (jdk1.6), Pig, Oozie, Pentaho
Confidential, Waukegan, IL
Java/J2EE Developer
Responsibilities:
- Utilized Agile Methodologies to manage full life-cycle development of the project.
- Implemented MVC design pattern using Struts Framework.
- Form classes of Struts Framework to write the routing logic and to call different services.
- Created tile definitions, Struts-config files, validation files and resource bundles for all modules using Struts framework.
- Developed web application using JSP custom tag libraries, Struts Action classes and Action. Designed Java Servlets and Objects using J2EE standards.
- Used JSP for presentation layer, developed high performance object/relational persistence and query service forentire applicationutilizingHibernate.
- Developed the XML Schema and Web services for the data maintenance and structures.
- Developed the application using Java Beans, Servlets and EJB’s.
- Created Stateless Session EJB’s for retrieving data and Entity Beans for maintaining User Profile.
- Used WebSphere Application Server and RAD to develop and deploy the application.
- Worked with various Style Sheets like Cascading Style Sheets (CSS).
- Designed database and created tables, written the complex SQL Queries and stored procedures as per the requirements.
- Involved in coding for JUnit Test cases, ANT for building the application.
Environment: Java/J2EE, Oracle 10g, SQL, PL/SQL, JSP, EJB, Struts, Hibernate, WebLogic 8.0, HTML, AJAX, Java Script, JDBC, XML, JMS, XSLT, UML, JUnit, log4j, MyEclipse 6.0
Confidential, Jefferson City, MO
Java/J2EE Developer
Responsibilities:
- Responsible for understanding the scope of the project and requirement gathering.
- Developed the web tier using JSP, Struts MVC to show account details and summary.
- Created and maintained the configuration of the Spring Application Framework (IoC).
- Implemented various design patterns - Singleton, Business Delegate, Value Object and Spring DAO.
- Used Spring JDBC to write some DAO classes to interact with the database to access account information.
- Mapped business objects to database using Hibernate.
- Involved in writing Spring Configuration XML files that contains declarations and other dependent objects declaration.
- Used Tomcat web server for development purpose.
- Involved in creation of Test Cases for JUnit Testing.
- Used Oracle as Database and used Toad for queries execution and also Involved in writing SQL scripts, PL/SQL code for procedures and functions.
- Used CVS, Perforce as configuration management tool for code versioning and release.
- Developed application using Eclipse and used build and deploy tool as Maven.
- Used Log4J to print the logging, debugging, warning, info on the server console.
Environment: Java1.5, J2EE Servlet, JSP, XML, Spring 3.0, Struts 1.1, Hibernate 3.0, Design Patterns, Log4j, CVS, Maven, Eclipse, Apache Tomcat 6, and Oracle 11g.
Confidential, Minnetonka, MN
Java Developer
Responsibilities:
- Coded the business methods according to the IBM Rational Rose UML model.
- Extensively used Core Java, Servlets, JSP and XML.
- Used Struts 1.2 in presentation tier.
- Generated the Hibernate XML and Java Mappings for the schemas
- Used DB2 Database to store the system data
- Used Rational Application Developer (RAD) as Integrated Development Environment (IDE).
- Used unit testing for all the components using JUnit .
- Used Apache log 4j Logging framework for logging of trace and Auditing.
- Used Asynchronous JavaScript and XML (AJAX) for better and faster interactive Front-End.
- Used IBM Web-Sphere as the Application Server.
- Used IBM Rational Clearcase as the version controller.
Environment: s: Java 1.6, Servlets, JSP, Struts1.2, IBM Rational Application Developer (RAD) 6, Websphere 6.0, iText, AJAX, Rational Clearcase, Rational Rose, Oracle 9i, log4j.
Confidential
Junior Java Developer
Responsibilities:
- Involved in Analysis, Design, Implementation, and Testing of the project.
- Implemented the presentation layer with HTML, XHTML, JavaScript, and CSS.
- Developed web components using JSP, Servlets and JDBC.
- Implemented database using SQL Server
- Designed Tables and indexes
- Wrote complex T-SQL and Stored Procedures.
- Involved in fixing defects and unit testing with test cases using JUnit.
- Developed user and technical documentation.
Environment: s: Java, JSP, Servlets, JDBC, JavaScript, CSS, MySQL, JUnit, Eclipse, JBoss.
