Hadoop Consultant Resume
Sunnyvale, CA
SUMMARY
- Over all 8+ years of experience in IT industry, which includes 2+ years of experience in Big Data and Hadoop Ecosystem and its tools.
- 6 years of experience in JAVA programming wif various technologies like Java, J2EE, JavaScript and Data Structures.
- Extensive experience in Banking, Insurance, Financial, Retail and Telecom domains.
- Excellent understanding of Hadoop architecture and underlying framework including storage management.
- Experience in using various Hadoop infrastructures such as MapReduce, Pig, Hive, Zookeeper, HBase, Sqoop, Oozie, Flume and SOLR for data storage and analysis.
- Experience in developing custom UDFs for Pig and Hive to in corporate methods and functionality of Python/Java into PigLatin and HQL (HiveQL)
- Experience wif Oozie Scheduler in setting up workflow jobs wif Map/Reduce and Pig jobs.
- Knowledge of architecture and functionality of NOSQL DB like HBase.
- Experience in managing Hadoop clusters and services using Cloudera Manager.
- Experience in troubleshooting errors in HBase Shell/API, Pig, Hive and MapReduce.
- Experience in importing and exporting data between HDFS and Relational Database Management systems using Sqoop.
- Collected logs data from various sources and integrated in to HDFS using Flume.
- Assisted Deployment team in setting up Hadoop cluster and services.
- Hands - on experience in setting up Apache Hadoop and Cloudera CDH clusters on Ubuntu, Fedora and Windows (Cygwin) environments.
- In-depth noledge of modifications required in static IP (interfaces), hosts and bashrc files, setting up password-less SSH and Hadoop configuration for Cluster setup and maintenance.
- Excellent understanding of Virtualization, wif experience of setting up a POC multi-node virtual cluster by leveraging underlying Bridge Networking and NAT technologies.
- Experience in loading data to HDFS from UNIX (Ubuntu, Fedora, Centos) file system.
- Knowledge of project life cycle (design, development, testing and implementation) of Client Server and Web applications.
- Experience in writing batch scripts in Ubuntu/UNIX to automate sequential script entry.
- Knowledge of Hardware, Software, Networking and external tools including but not limited to Excel, Access and experience in utilizing their functionality as and when required to enhance productivity and ensure accuracy.
- Determined, committed and hardworking individual wif strong communication, interpersonal and organizational skills.
- Technology enthusiast, highly motivated and an avid blog reader, keeping track of latest advancements in hardware and software fields. Well versed in designing and implementing Map Reduce jobs using JAVA on Eclipse to solve real world scaling problems.
- In-depth understanding of Data Structure and Algorithm
- Solid understanding of teh high volume, high performance systems.
- Strong Java development skills using J2EE, J2SE, Servlets, JSP, EJB, JDBC.
- Fair amount of experience wif scripting in PERL and Java script and statistical programming in R.
- Good noledge in integration of various data sources like RDBMS, Spreadsheets, JSON and XML and flat files.
- Expertise noledge of UNIX and shell scripting.
- Have flair to adapt to new software applications and products, self-starter, have excellent communication skills and good understanding of business work flow.
TECHNICAL SKILLS
Big Data Technologies: Hadoop, HDFS, Hive, MapReduce, Pig, Sqoop, Flume, Zookeeper, Crunch, Oozie, Hue, Hadoop distribution of Cloudera CDH3, Crunch API, HCatalog and HBase
Scripting/ Web Languages: JavaScript, Perl, Python, HTML, XML, SQL, Shell Scripting, WSDL, XSL, JAX
Programming Languages: C, C++ and Java
Java/J2EE Technologies: Java, Java Beans, J2EE (JSP, Servlets, EJB), and JDBC.
Frameworks: Hibernate 2.x/3.x, spring 2.x/3.x, Struts 1.x/2.x
Databases/ RDBMS: MySQL, PL-SQL, PostgreSQL, MS-SQL Server 2005/2008,Oracle 9i/10g/11g.
Statistical Programming: Programming in R, SAS
Operating Systems: Unix, Windows XP/7/NT/8/2003, MS DOS, Mac
Software Life Cycles: SDLC, Waterfall and Agile models
Office Tools: MS-OFFICE - Excel, Word, and PowerPoint
Utilities/ Tools: Eclipse, Tomcat, NetBeans, TOAD, JUnit, SQL, SVN, Log4j, Tiles, Developer, SQL*PLUS, SOAP UI, ANT, Maven, Visio, Mule ESB and MRUnit
Cloud Platforms: Amazon EC2
TECHNICAL SKILLS
Confidential, Sunnyvale, CA
Hadoop Consultant
Responsibilities:
- Developing parser and loader map reduce application to retrieve data from HDFS and store to HBase and Hive.
- Importing teh data from teh MySQL into teh HDFS using Sqoop.
- Importing teh unstructured data into teh HDFS using Flume.
- Used Oozie to orchestrate teh map reduce jobs that extract teh data on a timely manner.
- Written Map Reduce java programs to analyze teh log data for large-scale data sets.
- Involved in using HBase Java API on Java application.
- Automated all teh jobs for extracting teh data from different Data Sources like MySQL to pushing teh result set data to Hadoop Distributed File System.
- Customize parser loader application of Data migration to HBase.
- Developed Pig Latin scripts to extract teh data from teh output files to load into HDFS.
- Developed custom UDFS and implemented Pig scripts.
- Implemented MapReduce jobs using Java API and PIG Latin as well HIVEQL
- Participated in teh setup and deployment of Hadoop cluster
- Hands on design and development of an application using Hive (UDF).
- Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
- Provide support data analysts in running Pig and Hive queries.
- Involved in HiveQL.
- Involved in Pig Latin.
- Importing and exporting Data from MySQL/Oracle to HiveQL Using SQOOP.
- Importing and exporting Data from MySQL/Oracle to HDFS.
- Configured HA cluster for both Manual failover and Automatic failover.
- Designed and built many applications to deal wif vast amounts of data flowing through multiple Hadoop clusters, using Pig Latin and Java-based map-reduce.
- Specifying teh cluster size, allocating Resource pool, Distribution of Hadoop by writing teh specification texts in JSON File format.
- Creates a SOLR schema from teh Indexer settings
- Implemented SOLR index cron jobs.
- Experience in writing SOLR queries for various search documents
- Responsible for defining teh data flow wifin Hadoop eco system and direct teh team in implement them.
- Exported teh result set from Hive to MySQL using Shell scripts.
- Developed HIVE queries for teh analysts.
Environment: Apache Hadoop, Hive, Hue Tool, Zookeeper, Map Reduce, Sqoop, crunch API, Pig 0.10 and 0.11, HCatalog, Unix, Java, JSP, Eclipse, Maven, SQL, HTML, XML, Oracle, SQL Server, MYSQL
Confidential, Orlando, FL
Hadoop Consultant
Responsibilities:
- Involved in teh Software development life cycle (SDLC) to develop teh application.
- Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database and Sqoop.
- Involved in loading data from LINUX file system to HDFS.
- Experience in managing and reviewing Hadoop log files.
- Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Implemented test scripts to support test driven development and continuous integration.
- Supported in setting up QA environment and updating configurations for implementing scripts wif Pig and Sqoop.
- Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleaning and preprocessing.
- Created Pig Latin scripts to sort, group, join and filter teh enterprise wise data.
- Involved in creating Hive tables, loading wif data and writing hive queries that will run internally in MapReduce way.
- Supported MapReduce Programs those are running on teh cluster.
- Analyzed large data sets by running Hive queries and Pig scripts.
- Worked on tuning teh performance Pig queries.
- Mentored analyst and test team for writing Hive Queries.
- Installed Oozie workflow engine to run multiple MapReduce jobs.
- Worked wif application teams to install operating system, Hadoop updates, patches, version upgrades as required.
Environment: Hadoop, HDFS, Map Reduce, Hive, Pig, Sqoop, Linux, Java, Oozie, Hbase.
Confidential, Pleasanton, CA
Hadoop Consultant
Responsibilities:
- Installed and configured Hadoop MapReduce, HDFS and developed multiple MapReduce jobs in Java for data cleansing and preprocessing.
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive UDFs.
- Evaluated business requirements and prepared detailed specifications that follow project guidelines required to develop written programs.
- Devised procedures that solve complex business problems wif due considerations for hardware/software capacity and limitations, operating times and desired results.
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
- Provided quick response to ad hoc internal and external client requests for data and experienced in creating ad hoc reports.
- Responsible for building scalable distributed data solutions using Hadoop.
- Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
- Worked hands on wif ETL process.
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, and loaded data into HDFS.
- Extracted teh data from Teradata into HDFS using Sqoop.
- Analyzed teh data by performing Hive queries and running Pig scripts.
- Exported teh patterns analyzed back into Teradata using Sqoop.
- Continuous monitoring and managing teh Hadoop cluster through Cloudera Manager.
- Installed Oozie workflow engine to run multiple Hive.
- Developed Hive queries to process teh data and generate teh data cubes for visualizing
Environment: Hadoop, MapReduce, HDFS, Hive, Oozie, Java (jdk1.6), Cloudera, NoSQL, Oracle 11g/10g, PL SQL, SQL*PLUS, Toad 9.6, Windows NT, UNIX Shell Scripting.
Confidential, Bloomington, IL
Sr. Java Developer
Responsibilities:
- Involved in Design, Developing, Testing, and Support of Application.
- Implemented User Interface in Model-View-Controller Architecture, which accomplishes a tight and neat co-ordination of Spring-MVC, JSP and Custom Taglibs.
- Developed user interfaces using JSP, HTML, CSS and JavaScript.
- Implemented Dependency Injection (IOC) feature of spring framework to inject beans into User Interface and AOP for Logging.
- Developed reusable and interoperable Web service modules based on SOA architecture using SOAP, JAX-RPC and Apache AXIS 2.0.
- Used various design patterns like Business Delegate, session facade, Service locator for developing teh n-tier architecture.
- Experience in Enterprise Service Bus tools such as Mule.
- Implemented MULE Enterprise Service Bus (Mule ESB) to send and receive Messages between different services and applications.
- Involved in writing JAXB code to marshall and unmarshall XML data that Mule delivers.
- Developed unit testing framework by JUnit test cases.
- Developed database access layer using Spring-JDBC wif postgres database.
- Consumed teh web services and developed WSDLs for teh web service by teh use java2wsdl eclipse plug in.
- Actively participated in code reviews and design discussions.
- Used SOAPUI for testing teh web service response.
- Created and published SOAP based web services using Apache Axis 2.0 framework tool. These web services will be used to upload teh documents from teh client to teh claim handler.
- Implemented teh logging mechanism using log4j framework.
- Loaded data using to database using ETL tools like SQL* loader and external tables to load data from data warehouse and various other database like SQL Server, DB2
- Used SVN version controlling system for teh source code and project management.
- Involved in requirement analysis, in design phase, good exposure to UML, OOAD.
Environment: JDK 1.6, J2EE, Eclipse, Tomcat, tcServer, JSP, JSF, Javascript, Spring, Spring MVC, Spring JDBC, Agile, JUnit, SVN, XML, Web Services, WSDL, SOAP, JAX-RPC, Postgres, SQL Server 2008, ETL, Mule ESB.
Confidential, North Carolina
Java Developer
Responsibilities:
- Involved in development of business domain concepts into Use Cases, Sequence Diagrams, Class Diagrams, Component Diagrams and Implementation Diagrams.
- Implemented various J2EE Design Patterns such as Model-View-Controller, Data Access Object, Business Delegate and Transfer Object.
- Responsible for analysis and design of teh application based on MVC Architecture, using open source Struts Framework.
- Involved in configuring Struts, Tiles and developing teh configuration files.
- Developed Struts Action classes and Validation classes using Struts controller component and Struts validation framework.
- Developed and deployed UI layer logics using JSP, XML, JavaScript, HTML /DHTML.
- Used Spring Framework and integrated it wif Struts.
- Involved in Configuring web.xml and struts-config.xml according to teh struts framework.
- Designed a lightweight model for teh product using Inversion of Control principal and implemented it successfully using Spring IOC Container.
- Used transaction interceptor provided by Spring for declarative Transaction Management.
- Teh dependencies between teh classes were managed by Spring using teh Dependency Injection to promote loose coupling between them.
- Provided connections using JDBC to teh database and developed SQL queries to manipulate teh data.
- Developed ANT script for auto generation and deployment of teh web service.
- Wrote stored procedure and used JAVA APIs to call these procedures.
- Developed various test cases such as unit tests, mock tests, and integration tests using teh JUNIT.
- Experience writing Stored Procedures, Functions and Packages
- Used log4j to perform logging in teh applications.
Environment: Java, J2EE, Struts MVC, Tiles, JDBC, JSP, JavaScript, HTML, Spring IOC, Spring AOP, JAX-WS, Ant, Oracle, JUNIT and Log4j, Eclipse
Confidential, AZ
J2EE/Java Developer
Responsibilities:
- Involved in teh designing of teh project using UML.
- Followed J2EE Specifications in teh project.
- Designed teh user interface pages in JSP.
- Used XML and XSL for mapping teh fields in database.
- Used JavaScript for client side validations.
- Created stored procedures and triggers that are required for project.
- Created functions and views in Oracle.
- Enhanced teh performance of teh whole application using teh stored procedures and prepared statements.
- Responsible for updating database tables and designing SQL queries using PL/SQL.
- Created bean classes for communicating wif database.
- Involved in documentation of teh module and project.
- Prepared test cases and test scenarios as per business requirements.
- Involved in bug fixing.
- Prepared coded applications for unit testing using JUnit.
Environment: Java, JSP, Servlets, J2EE, EJB 3, Java Beans, Oracle, HTML, DHTML, XML, XSL, JavaScript.
Confidential
J2EE/Java Developer
Responsibilities:
- Involved in Full Life Cycle Development in Distributed Environment Using Java and J2EE framework.
- Responsible for developing and modifying teh existing service layer based on teh business requirements.
- Involved in designing & developing web-services using SOAP and WSDL.
- Involved in database design.
- Created tables, stored procedures in SQL for data manipulation and retrieval, Database Modification using SQL, PL/SQL, Stored procedures, triggers, Views in Oracle 9i.
- Created User Interface using JSF.
- Involved in integration testing teh Business Logic layer and Data Access layer.
- Integrated JSF wif JSP and used JSF Custom Tag Libraries to display teh value of variables defined in configuration files.
- Used technologies like JSP, JSTL, JavaScript, HTML, XML and Tiles for Presentation tier
- Involved in JUnit testing of teh application using JUnit framework.
- Written Stored Procedures functions and views to retrieve teh data.
- Used Maven builds to wrap around Ant build scripts.
- CVS tool is used for version control of code and project documents.
- Responsible to mentor/work wif team members to make sure teh standards and guidelines are followed and delivery of tasks in time.
Environment: JQuery, JSP, Servlets, JSF, JDBC, HTML, JUnit, JavaScript, XML, SQL, Maven, Web Services, UML, Web Logic Workshop and CVS.
