Hadoop Developer Resume
Columbia, MD
OBJECTIVE
- Looking for a challenging position as Hadoop Developer where I can use my noledge, technical and analytical skills to contribute to projects that add value to teh organization.
SUMMARY
- Having total work experience of 9+ years in Information Technology with skills in analysis, design, development, testing and deploying various software applications, which include Web related and windows applications with emphasis on Object Oriented Programming.
- About 2 years of work experience on Big Data Analytics as Hadoop Developer.
- Experienced on major Hadoop ecosystem’s projects such as Pig, Hive, HBase and monitoring them with Cloudera Manager & Hortonworks.
- Extensive experience in developing Pig Latin Scripts and using Hive Query Language for data analytics
- Hands on experience working on NoSQL databases including HBase and its integration with Hadoop cluster
- Have hands on experience in writing Map Reduce jobs on Hadoop Ecosystem including Hive and Pig.
- Hands on experience in installing, configuring and using ecosystem components like Hadoop Map Reduce, HDFS, Pig, Hive, Sqoop.
- Handling and further processing schema oriented and non - schema oriented data using Pig.
- Read, processed and stored desperate data in parallel using Pig.
- Good noledge of database connectivity (JDBC) for databases like Oracle, DB2, SQL Server, My Sql and Netezza.
- Experienced in coding SQL, Procedures/Functions, Triggers and Packages on database (RDBMS) packages like Oracle.
- Developed stored procedures and queries using SQL.
- Worked on Agile methodology, SOA for many of teh applications.
- Excellent analytical, problem solving, communication and interpersonal skills with ability to interact with individuals Confidential all levels and can work as a part of a team as well as independently.
- In-depth understanding of Data Structure and Algorithms.
- Strong Communication skills of written, oral, interpersonal and presentation.
- Ability to perform Confidential a high level, meet deadlines, adaptable to ever changing priorities.
- Good noledge in using job scheduling and monitoring tools like Oozie and ZooKeeper
- Mentor team in UNIX and open source tools/platformsrevolving around Hadoop.
TECHNICAL SKILLS
Hadoop/Big Data: HDFS, Map Reduce, Hive, Pig, Sqoop, Zookeeper,Solr and Oozie.
No SQL Databases: HBase
Programming Languages: Java, PL/SQL, Pig Latin, Hive QL, Unix shell scripts
Operating Systems: UNIX, Windows, LINUX
Web technologies: JSP, JDBC
Databases: Oracle 9i/10g, Microsoft SQL Server and MySQL
Java IDE: Eclipse 3.x
Tools: TOAD, SQL Developer
PROFESSIONAL EXPERIENCE
Confidential, Columbia, MD
Hadoop Developer
Responsibilities:
- Developed Map-Reduce programs to parse teh raw data, populate staging tables and store teh refined data in partitioned tables in teh EDW.
- Developed teh Sqoop scripts in order to make teh interaction between Pig and MySQL Database.
- Created Hive queries that helped market analysts spot emerging trends by comparing fresh data with EDW reference tables and historical metrics.
- Created Hive tables to store teh processed results in a tabular format.
- Writing teh script files for processing data and loading to HDFS
- Writing CLI commands using HDFS.
- Developed teh UNIX shell scripts for creating teh reports from Hive data.
- Completely involved in teh requirement analysis phase.
- Enabled speedy reviews and first mover advantages by using Oozie to automate data loading into teh Hadoop Distributed File System and PIG to pre-process teh data.
- Provided design recommendations and thought leadership to sponsors/stakeholders that improved review processes and resolved technical problems.
- Managed and reviewed Hadoop log files.
- Tested raw data and executed performance scripts.
- Involved in gathering teh requirements, designing, development and testing
- Responsible for building scalable distributed data solutions using Hadoop
- Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
- Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
- Exported teh result set from Hive to Netezza using Shell scripts.
Environment: Hadoop 2.4, Hive 0.13,HDP 2.1, Map Reduce, Sqoop, Pig 0.10 and 0.11, JDK1.6,HDFS, Flume, Tidal, HBase, Zookeeper,Mahout, PL/SQL and SQL.
Confidential, Fremonth, CA
Hadoop Developer
Responsibilities:
- Installed and configured Hadoop Map Reduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
- Loaded teh customer profiles data, customer spending data, credit from legacy warehouses onto HDFS using Sqoop.
- Built data pipeline using Pig and Java Map Reduce to store onto HDFS.
- Used Oozie to orchestrate teh map reduce jobs that extract teh data on a timely manner.
- Applied transformations and filtered both traffic using Pig.
- Used Pattern matching algorithms to recognize teh customer across different sources and built risk profiles for each customer using Hive and stored teh results in HBase.
- Performed unit testing using MRUnit.
- Responsible for building scalable distributed data solutions using Hadoop
- Installed and configured Hive, Pig, Sqoop, Flume and Oozie on teh Hadoop cluster
- Setup and benchmarked Hadoop/HBase clusters for internal use
- Developed Simple to complex Map/reduce Jobs using Hive and Pig
- Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted teh data from MySQL into HDFS using Sqoop
- Analyzed teh data by performing Hive queries and running Pig scripts to study customer behavior
- Installed Oozie workflow engine to run multiple Hive and Pig jobs
- Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team
- Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
- Provide support data analysts in running Pig and Hive queries.
- Importing and exporting Data from Mysql/Oracle to HiveQL using SQOOP.
- Importing and exporting Data from Mysql/Oracle to HDFS using SQOOP.
- Responsible for defining teh data flow within Hadoop eco system and direct teh team in implement them.
- Exported teh result set from Hive to MySQL using Shell scripts.
- Developed HIVE queries for teh analysts.
Environment: Hadoop, Hive, Zookeeper, Map Reduce, Sqoop, Pig 0.10 and 0.11, JDK1.6,HDFS, Flume, Oozie, DB2, HBase, Mahout, PL/SQL and SQL.
Confidential, San Ramon, CA
Java/PLSQL Developer
Responsibilities:
- Responsible for gathering and analyzing requirements and converting them into technical specifications
- Used Rational Rose for creating sequence and class diagrams
- Developed presentation layer using JSP, Java, HTML and JavaScript
- Designed and developed Hibernate configuration and session-per-request design pattern for making database connectivity and accessing teh session for database transactions respectively. Used HQL and SQL for fetching and storing data in databases
- Participated in teh design and development of database schema and Entity-Relationship diagrams of teh backend Oracle database tables for teh application
- Implemented web services with Apache Axis
- Designed and Developed Stored Procedures, Triggers in Oracle to cater teh needs for teh entire application. Developed complex SQL queries for extracting data from teh database
- Designed and built SOAP web service interfaces implemented in Java
- Used Apache Ant for teh build process
- Used ClearCase for version control and ClearQuest for bug tracking
Environment: Java, JDK 1.5, Servlets, Hibernate, Oracle 10g, Eclipse, Web Services (SOAP), JavaScript, HTML, XML
Confidential, Waukegan, IL
Java developer
Responsibilities:
- Utilized Agile Methodologies to manage full life-cycle development of teh project.
- Implemented MVC design pattern using Struts Framework.
- Form classes of Struts Framework to write teh routing logic and to call different services.
- Created tile definitions, Struts-config files, validation files and resource bundles for all modules using Strutsframework.
- Developed web application using JSP custom tag libraries, Struts Action classes and Action.Designed Java Servlets and Objects using J2EE standards.
- Used JSP for presentation layer, developed high performance object/relational persistence and query service forentire applicationutilizingHibernate.
- Developed teh XML Schema and Web services for teh data maintenance and structures.
- Developed teh application using Java Beans, Servlets.
- Used WebSphere Application Server and RAD to develop and deploy teh application.
- Worked with various Style Sheets like Cascading Style Sheets (CSS).
- Designed database and created tables, written teh complex SQL Queries and storedprocedures as per teh requirements.
Environment: Java JDK 1.4, Oracle 10g, SQL, PL/SQL, JDBC, XML.
Confidential
Pl/Sql Developer
Responsibilities:
- Experience on Oracle SQL, PL/SQL and Perl development
- Lead experience giving recommendations and direction on development strategies, conducting code reviews, and mentoring junior developers, and setting Standards and establishing Best Practices.
- Good RDBMS understanding
- Expertise in tuning Oracle SQLs
- Good Understanding of Data warehouse concepts
- Knowledge of SDLC cycles
- Strong experience in Technical documentation, coding and testing
- Expertise in problem solving through debugging, research and investigation
- Familiar with Best practices, Standard Concepts.
- Good Communication skills
- Requirements analysis of teh inputs.
- Execution of teh required deliverables.
- Defect prevention activities.
- Creation of UTP, UTR.
Environment: SQL* PLUS, PL/SQL Developer 9.0, SQL-Loader, SQL Navigator, SSH, DB2, Perl, Windows 2000/2003/NT/XP, UNIX, C, C++, Java, TOAD, Eclipse, Notepad ++ 5.7, HP OpenView, WinZip, Oracle 10g/9i/8i, MS Office (Visio, Excel, Word, Access), MS SQL Server 2005/2000