Hadoop Developer Resume
Plano, TexaS
PROFESSIONAL SUMMARY:
- Over 8 years of professional IT experience which includes more than 3 years of experience in Big data ecosystem related technologies.
- Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, NameNode, Data Node and MapReduce programming paradigm.
- Hands on experience in installing, configuring, and using Hadoop ecosystem components like Hadoop MapReduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, Zookeeper and Flume.
- Good Exposure on Apache Hadoop Map Reduce programming, PIG Scripting and Distribute Application and HDFS.
- Good Knowledge on Hadoop Cluster architecture and monitoring the cluster.
- In - depth understanding of Data Structure and Algorithms.
- Experience in managing and reviewing Hadoop log files.
- Experience in NoSQL database MongoDB and Cassandra.
- Implemented by setting up standards and processes for Hadoop based application design and implementation.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice-versa.
- Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
- Experience in Amazon AWS cloud services (EC2, EBS, S3).
- Experience in managing Hadoop clusters using the Cloudera Manager Tool.
- Very good experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications.
- Extensive experience working with Oracle, DB2, SQL Server and My SQL database.
- Hands on experience in VPN, Putty, WinSCP, VNCviewer, etc.
- Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
- Experience in Java, JSP, Servlets, EJB, WebLogic, WebSphere, Hibernate, Spring, JBoss, JDBC, RMI, Java Script, Ajax, Jquery, XML, and HTML
- Ability to adapt to evolving technology, strong sense of responsibility and .
TECHNICAL SKILLS:
HADOOP/BIG DATA: HDFS, MapReduce, Hive, Pig, HBase, Sqoop, Flume, Oozie, Zoo keeper, Apache Cassandra, PowerPivot, Datameer, Pentaho, IBM BigInsights, Netezza.
DATABASE: Oracle 10g, DB2, MySQL, MongoDB, CouchDB
WEB TECHNOLOGIES: MQseries, struts, Junit, ODBC, JDBC, XML, XSL,XSD, CSS, JavaScript,Hibernate, Spring, Ajax,Jquery,JSP, Servlets, Java Swings, Java Beans, EJB, MVC, JNDI, JMS, Java Mail, HTMLOPERATING SYSTEMS: Windows, Linux, UNIX
PROFESSIONAL EXPERIENCE:
Confidential, Plano, Texas
Hadoop Developer
Responsibilities:
- Installed and configured Hive, Pig, Sqoop and Oozie workflow engine (developed Sqoop, Hive and pig actions).
- Configured MySQL database to store hive metadata.
- Extracted data from Oracle database to HDFS using Sqoop.
- Migrated all the Net flow data from Netezza to HDFS for IP analytics purpose.
- Handled importing of data from various data sources, performed transformations using hive. (External tables, partitioning).
- Performed analysis of data using Pig scripting.
- Used Pig as ETL tool to do transformations, event joins, filter and some pre-aggregations
- Converted SQL scripts into hive to get better performance.
- Worked on Sequence files, RC files, Map side joins, bucketing, petitioning for Hive performance enhancement and storage improvement.
- Implemented Hive Generic UDF's to implement business logic.
- Involved in loading data into HBase using HBase Shell, HBase Client API, Pig and Sqoop.
- Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (like, Pig, Hive, and Sqoop) as well as system specific jobs (such as Perl and shell script).
- Automated all the jobs, for pulling netflow data from relational databases to load data into Hive tables, using Oozie workflows and enabled email alerts on any failure cases.
- Wrote the shell scripts to monitor the health check of Hadoop daemon services and respond accordingly to any warning or failure conditions.
- Worked on IBM Big Insights installation.
Environment: RHEL, Hive, Pig, Sqoop, Oozie, Netezza, Big Insights, Oracle, base, Perl
Confidential
Hadoop Developer
Responsibilities:
- Installed and configured Flume, Hive, Pig, Sqoop and Oozie on the Hadoop cluster.
- Launching and Setup of HADOOP/ HBASE Cluster, which includes configuring different components of HADOOP and HBASE Cluster.
- Experienced in loading data from the UNIX file system to HDFS.
- Created HBase tables to load large sets of structured, semi-structured and unstructured data coming from UNIX, NoSQL and a variety of portfolios.
- Worked on writing transformer/mapping Map-Reduce pipelines using Java.
- Involved in creating Hive tables, loading them with data and writing hive queries that will run internally in map reduce way.
- Migrated ETL jobs to Pig scripts do Transformations, even joins and some pre-aggregations before storing the data onto HDFS
- Worked on different file formats like Sequence files, XML files and Map files using Map Reduce Programs.
- Experienced in loading and transforming of large sets of structured, semi structured and unstructured data.
- Involved in creating Oozie workflow and Coordinator jobs to kick off the jobs on time for data availability.
- Used Flume to collect, aggregate, and store the web log data from different sources like web servers, network devices and pushed to HDFS.
- Scripting to deploy monitors, checks and critical sysadmin functions automation.
- Managing and scheduling Jobs on a Hadoop cluster.
- Performing tuning and troubleshooting of MapReduce jobs by analyzing and reviewing Hadoop log files.
- Involved in defining job flows, managing and reviewinglog files.
Environment: Map Reduce, HDFS, Hive, Java, SQL, Cloudera Manager, Pig, Sqoop, Oozie
Confidential
IT Analyst
Responsibilities:
- Involved in Analysis, Design, Development and Testing of application modules.
- Analyzed the complex relationship of the system and improve the performances of various screens.
- Developed various user interface screens using struts framework.
- Worked with Spring framework for dependency injection.
- Developed JSP pages, using Java Script, JQuery, and AJAX for client side validation and CSS for data formatting.
- Written domain, mapper and DTO classes and hbm.xml files to access data from DB2 tables.
- Developed various reports using Adobe APIs and Web services.
- Wrote test cases using Junit and coordinated with testing team for integration tests
- Fixed bugs, improved performance using root cause analysis in production support
Environment: JDK 1.4.2, Swings, EJB 1.3, XML, XML Spy, SQL, WinSQL, StarTeam, DB2, WSAD 5.1.2, Apache ant, Windows XP/7, Web services, Junit, Hyperion 8/9.3, Citrix, Mainframes, CVS, JNDI
Confidential
Java/J2EE Interface Developer
Responsibilities:
- Created Use case, Sequence diagrams, functional specifications and User Interface diagrams using Star UML.
- Involved in complete requirement analysis, design, coding and testing phases of the project.
- Participated in JAD meetings to gather the requirements and understand the End Users System.
- Developed user interfaces using JSP, HTML, XML and JavaScript.
- Generated XML Schemas and used XML Beans to parse XML files.
- Created Stored Procedures & Functions. Used JDBC to process database calls for DB2/AS400 and SQL Server databases.
- Developed the code which will create XML files and Flat files with the data retrieved from Databases and XML files.
- Created Data sources and Helper classes which will be utilized by all the interfaces to access the data and manipulate the data.
- Developed web application called iHUB (integration hub) to initiate all the interface processes using Struts Framework, JSP and HTML.
- Developed the interfaces using Eclipse 3.1.1 and JBoss 4.1 Involved in integrated testing, Bug fixing and in Production Support
Environment: Java 1.3, Servlets, JSPs, Java Mail API, Java Script, HTML, MySQL 2.1, Swing, Java Web Server 2.0, JBoss 2.0, RMI, Rational Rose, Red Hat Linux 7.1.
Confidential
Software Engineer
Responsibilities:
- Utilized the base UML methodologies and Use cases modeled by architects to develop the front-end interface. The class, sequence and state diagrams were developed using Rational Rose and Microsoft Visio.
- Designed application using MVC design pattern.
- Developed front-end user interface modules by using HTML, XML, Java AWT, and Swing.
- Front-end validations of user requests carried out using Java Script.
- Designed and developed the interacting JSPs and Servlets for modules like User Authentication and Summary Display.
- Designed and developed Entity/Session EJB components for the primary modules.
- Java Mail was used to notify the user of the status and completion of the request.
- Developed Stored Procedures on Oracle 8i.
- Implemented Queries using SQL (database triggers and functions).
- JDBC was used to interface the web-tier components on the J2EE server with the relational database.
Environment: Java1.3, EJB, Java Script, HTML, XML, Rational Rose, Microsoft Visio, Swings, JSP, Servlets, JNDI, JDBC, SQL, Oracle8i, Tomcat 3.1, Windows NT.