Hadoop Developer Resume
Dallas, TX
SUMMARY:
- Around 7.5 years of IT experience in software development and support with experience in developing strategicmethods for deploying Big Data technologies to efficiently solve Big Data processing requirement.
- Experience with distributed systems, large - scale non-relational data stores, RDBMS, NoSQL map-reducesystems, data modeling, database performance, and multi-terabyte data warehouses.
- Working experience in Hadoop framework, Hadoop Distributed File System and Parallel Processingimplementation.
- Hands-on experience with the overall Hadoop eco-system - HDFS, Map Reduce, Pig/Hive, Hbase.
- Working experience with large scale Hadoop environments build and support including design, configuration,installation, performance tuning and monitoring.
- Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, NameNode, Data Node and MapReduce programming paradigm.
- Hands on experience in application development using the technologies Java, RDBMS, Linux/Unix shellscripting and Linux internals.
- Experience in writing custom UDFs in java for Hive and Pig.
- Experience in writing custom partitioner and countersMap Reduce programs in java.
- Experience in installation, configuration and management of development, testing and production HadoopCluster.
- Performed Importing and exporting data into HDFS and Hive using Sqoop.
- Experience in working with Flume to load the log data from multiple sources directly into HDFS.
- Experience in designing both time driven and data driven automated workflows using Oozie.
- Experience in loading log data into HDFS using Flume.
- Experience in writing shell scripts.
- Experience working with JAVA, J2EE, JDBC, ODBC, JSP, Java Eclipse, Java Beans, EJB, Servlets, MSSQL Server.
- Experience in using IDEs like Eclipse and NetBeans.
- Experience working with spring.
- Development experience in Oracle.
- Experience with Client Sidedesigning and validations using HTML, DHTML and Java Script.
- Experience in User Interface Designing using HTML, DHTML CSS, JavaScript and Photoshop.
- Knowledge on various NoSQL databases like NeoDB, Cassandra andMongoDB.
- Expert in writingSQL queriesand database programming using PL/SQL.
- Quick learning skills and effective team spirit with good communication skills.
- Strong analytical and Problem solving skills.
TECHNICAL SKILLS:
Programming Languages: Java, C++, SQL, PL/SQL
Java Technologies: JDBC
Frame Works: Spring, LDAP,JUnit and JTest
Databases: Oracle 10g/9i, NO SQL (HBase)
IDE's &Utilities: Eclipse, NetBeans
Web Dev. Technologies: HTML, XML, CSS
Protocols: TCP/IP, HTTP and HTTPS.
Operating Systems: Linux, Mac, WINDOWS 98/00/NT/XP
Hadoop eco-system: Hadoop and MapReduce, Sqoop, PIG, Hive
PROFESSIONAL EXPERIENCE:
Confidential, Dallas,TX
Hadoop Developer
Responsibilities:
- Installed and configured HadoopMapReduce, HDFS, Developed multiple MapReduce jobs in java for datacleaning and preprocessing.
- Experience in installing, configuring and using Hadoop Ecosystem components.
- Experience in Importing and exporting data into HDFS and Hive using Sqoop.
- Experienced in defining job flows.
- Knowledge in performance troubleshooting and tuning Hadoop clusters.
- Experienced in managing and reviewing Hadoop log files.
- Participated in development/implementation of ClouderaHadoop environment.
- Load and transform large sets of structured, semi structured and unstructured data.
- Experience in working with various kinds of data sources such as MongoDb and Oracle.
- Successfully loaded files to Hive and HDFS from MongoDB.
- Installed oozie workflow engine to run multiple map-reduce programs which run independently with time anddata.
- Performed Data scrubbing and processing with Oozie.
- Responsible for managing data coming from different sources.
- Gained good experience with NOSQL database.
- Experience in working with Flume to load the log data from multiple sources directly into HDFS.
- Supported Map Reduce Programs those are running on the cluster.
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive UDFs.
- Involved in creating Hive tables, loading with data and writing hive queries, which will run internally in Map Reduce way.
- Worked in installing cluster, commissioning & decommissioning of Data Node, Name Node recovery, capacity planning and slots configuration.
- Implemented best income logic using Pig scripts.
- Load and transform large sets of structured, semi structured and unstructured data.
- Exported the analyzed data to the relational databases using Sqoopfor visualization and to generate reportsfor the BI team.
- Supported in setting up QA environment and updating configurations for implementing scripts with Pig andSqoop.
Environment: Hadoop, MapReduce, HDFS, Hive, Java, SQL, Datameer, PIG, Zookeeper, Sqoop, Oozie,MongoDb, CentOS, SOLR.
Confidential, Concord, NH
Hadoop Developer
Responsibilities:
- Replaced default Derby metadata storage system for Hive with MySQL system.
- Setup and benchmarked Hadoop/HBaseclusters for internal use.
- Involved in creating Hive tables, and loading and analyzing data using hive queries.
- Executed queries using Hive and developed Map Reduce jobs to analyze data.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Knowledge on reading data from Cassandra and also writing to it.
- Responsible in modification of API packages.
- Developed the Pig UDF's to preprocess the data for analysis.
- Extracted files from CouchDB through Sqoop and placed in HDFS and processed.
- Responsible for managing the log files. The log files were copied to S3 and later brought in the HDFS usingFlume.
- Migrated all the MapReduce results to Hbase for future reference.
- Developed Hive queries for the analysts.
- Utilized Apache Hadoop environment by Hortonworks.
- Involved in loading data from LINUX and UNIX file system to HDFS.
- Assisted in exporting analyzed data to relational databases using Sqoop.
- Supported in setting up QA environment and updating configurations for implementing scripts with Pig.
- Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hivequeries and Pig Scripts.
Environment: Core Java, Apache Hadoop (Horton works), HDFS, Pig, Hive, Hbase, Cassandra, Sqoop, Flume,Shell Scripting, My Sql, LINUX, UNIX.
Confidential - New York, NY
Hadoop Developer
Responsibilities:
- Involved in review of functional and nonfunctional requirements.
- Facilitated knowledge transfer sessions.
- Installed and configured Hadoop Map reduce, HDFS, Developed multiple Map Reduce jobs in java for datacleaning and preprocessing.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Experience in defining job flows.
- Experience in managing and reviewing Hadoop log files.
- Extracted files from RDBMS through Sqoop and placed in HDFS and processed.
- Experience in running Hadoop streaming jobs to process terabytes of xml format data.
- Got good experience with NOSQL database.
- Supported Map Reduce Programs those are running on the cluster.
- Involved in loading data from UNIX file system to HDFS.
- Involved in creating Hive tables, loading with data and writing hive queries which will run internally in MapReduce way.
- Replaced default Derby metadata storage system for Hive with MySQL system.
- Executed queries using Hive and developed MapReduce jobs to analyze data.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Developed the Pig UDF's to preprocess the data for analysis.
- Developed Hive queries for the analysts.
- Involved in loading data from LINUX and UNIX file system to HDFS.
- Supported in setting up QA environment and updating configurations for implementing scripts with Pig.
- Developed a custom File System plug in for Hadoop so it can access files on Data Platform. This pluginallows Hadoop MapReduce programs, HBase, Pig and Hive to work unmodified and access files directly.
- Designed and implemented Map Reduce based large scale parallel relation learning system
- Extracted feeds form social media sites such as Facebook, Twitter using Python scripts.
- Setup and benchmarked Hadoop/HBase clusters for internal use.
- Setup Hadoop cluster on Amazon EC2 using whirr for POC.
- Wrote recommendation engine using mahout.
Environment: Java, Eclipse, Oracle 10g, Sub Version, Hadoop, Hive, HBase, MapReduce, HDFS, Pig Hive,Cassandra, Java (JDK 1.6), Hadoop Distribution of Cloudera, MapReduce.
Confidential - Dallas, TX
Java/J2EE/ Hadoop Developer
Responsibilities:
- Involved in review of functional and non-functional requirements.
- Installed and configured Hadoop MapReduce and HDFS.
- Acquired good understanding and experience of NoSQL databases such as HBase and Cassandra.
- Installed and configured Hive and also implemented various business requirements by writing Hive UDFs.
- Extensively worked on user interface for few modules using HTML, JSP's, JavaScript, Python and Ajax.
- Generated Business Logic using servlets, Session beans and deployed them on Web logic server.
- Created complex SQL queries and stored procedures.
- Developed the XML schema and Web services for the data support and structures.
- Implemented the Web service client for login verification, credit reports and applicant information usingApache Axis 2 web service.
- Responsible for managing data coming from different sources.
- Used Hibernate ORM framework with spring framework for data persistence and transaction management.
- Used struts validation framework for form level validations.
- Wrote test cases in JUnit for unit testing of classes.
- Provided technical support for production environments resolving the issues, analyzing the defects, providingand implementing the solution defects.
- Built and deployed Java application into multiple UNIX based environments and produced both unit andfunctional test results along with release notes.
Environment: Hadoop, HBase, Hive, Java, Eclipse, J2EE 1.4, Struts 1.3, JSP, Servlets 2.5, WebSphere 6.1,HTML, XML, ANT 1.6, Python, JavaScript, Junit 3.8.
Confidential - Atlanta, GA
Java Developer
Responsibilities:
- Worked with sprint planning, sprint demo, status and daily standup meeting.
- Developed the application using Spring Web MVC framework.
- Worked with Spring Configuration files to add new content to the website.
- Worked on the Spring DAO module and ORM using Hibernate. Used Hibernate Template andHibernateDaoSupport for Spring-Hibernate Communication.
- Extensively used Spring's features such as Dependency Injection/Inversion of Control to allow loose couplingbetween business classes (POJOs)
- Used Hibernate framework to retrieve and update information and dependency Injection is achieved by SpringMVC Framework.
- Configured Association Mappings such as one-one and one-many in Hibernate.
- Worked with JavaScript calls as the Search is triggered through JS calls when a Search key is entered inthe Search window.
- Worked on analyzing other Search engines to make use of best practices.
- Collaborated with the Business team to fix defects.
- Worked on XML, XSL and XHTML files.
- Interacted with project management to understand, learn and to perform analysis of the Search Techniques.
- Used Ivy for dependency management.
- As part of the team to develop and maintain an advanced search engine, would be able to attain expertiseon a variety of new software technologies.
Environment: Java 1.6, J2EE, Eclipse SDK 3.3.2, Java Spring 3.x, JQuery, Oracle 10i, Hibernate, JPA, Json,Apache Ivy, SQL, Stored Procedures, Shell Scripting, JQuery, XML, HTML, Junit, TFS, Ant, Visual StudioPremium 2010, Rational Clearquest.
Confidential
Java Developer
Responsibilities:
- The project was the development of remote network administration application for the R&D department atDimensionality Software Services. It dealt with administering a Server system using a mobile device.
- It was developed using a proxy server as the Server system and a mobile simulator as the Client machine.
- Using this application a system administrator can perform Login, Logoff, Shutdown, Restart among otheroperations on the server system through a mobile device.
- As a developer was involved in all the development phases including gathering requirements, documentingthe requirements as Use case documents.
- Involved in developing code, utilizing the Object Oriented design principles, Unit and integration testing.
- As a Developer worked with the Lead in analyzing the requirements.
- Used knowledge of design tools for building use cases and class diagrams.
- Utilized programming skills for developing the application.
- Designed drop down menus and the Graphical user interface for the application using Java Swings.
- Implemented the Wireless Access Protocol (WAP) in Java (J2EE) to facilitate communication between themobile simulator and the Server system.
- Attained experience in working with and integrating multiple modules to build a synchronized application.
- Created stored procedures and triggers to access information.
Environment: Java 1.5, XML, JSP, Java Swings, Junit, MS SQL, Apache Tom-Cat Server, JDBC, Mobile-Simulators.
Confidential
Java Developer
Responsibilities:
- The project was the development of remote network administration application for the R&D department atDimensionality Software Services. It dealt with administering a Server system using a mobile device.
- It was developed using a proxy server as the Server system and a mobile simulator as the Client machine.
- Using this application a system administrator can perform Login, Logoff, Shutdown, Restart among otheroperations on the server system through a mobile device.
- As a developer was involved in all the development phases including gathering requirements, documentingthe requirements as Use case documents.
- Involved in developing code, utilizing the Object Oriented design principles, Unit and integration testing.
- As a Developer worked with the Lead in analyzing the requirements.
- Used knowledge of design tools for building use cases and class diagrams.
- Utilized programming skills for developing the application.
- Designed drop down menus and the Graphical user interface for the application using Java Swings.
- Implemented the Wireless Access Protocol (WAP) in Java (J2EE) to facilitate communication between themobile simulator and the Server system.
- Attained experience in working with and integrating multiple modules to build a synchronized application.
- Created stored procedures and triggers to access information.
Environment: Java 1.5, JSP, Java Swings, Apache Tom-Cat Server, Mobile-Simulators