We provide IT Staff Augmentation Services!

Hadoop Developer/hadoop Admin Resume

3.00/5 (Submit Your Rating)

San Jose, CA

SUMMARY

  • Over 9+ years of IT experience wif 3 years of experience in Software development, Data warehousing and Analytics and Data engineering using Hadoop, Mapreduce, Pig, Hive and other open source tools/technologies.
  • Worked on ETL tool Informatica, Oracle Database and PL/SQL, Python and Shell Scripts.
  • Hands on experience in creating and writing documents using MS Office products like Word, Excel, PowerPoint.
  • Sound knowledge of Business Intelligence and Reporting. Preparation of Dashboards using Tableau.
  • Team player and a quick learner wif strong relationship building & interpersonal skills.
  • Good understanding of Operating System concepts, Multimedia and Web Design.
  • Good knowledge and understating in writing T - SQL queries and user defined functions based on teh business logic.
  • Good technical knowledge of Eclipse, HTML, XML, Cascading Style Sheets, JavaScript.
  • Sound knowledge in Data Analytics, Database Management Systems and Object Oriented Analysis (OOA) and Design through UML.
  • Experience in developing NoSQL database by using CRUD, Sharding, Indexing and Replication.
  • Experience wif ETL working wif Hive and Map-Reduce.
  • Experience in developing teh complex SQL queries, unions and multiple table joins and experience wif views.
  • Knowledge of HIPPA EDI transactions and implementation of solutions in a health care setting using HIPPA-EDI.
  • Involved in database design, creating Tables, Views, Stored Procedures, Functions, Triggers and Indexes.Strong experience in Data Warehousing and ETL using Datastage.
  • Good experience in Python scripting, Shell Scripting, SQL Server, UNIX and Linux.
  • Experience wif various Business Intelligence tools and SQL databases.
  • Expertise in all teh stages of teh Software development Life Cycle (SDLC) namely Requirementanalysis, Design specifications, Coding, Debugging, Testing.
  • Well-Versed wif Agile/SCRUM and Waterfall methodologies.
  • Experience in interacting wif customers and working at client locations for real time field testing of products and services.
  • Designed and implemented data ingestion patterns using Sqoop, Flume and Kafka.
  • Very good experience in customer specification study, requirements gathering, system architectural design and turning teh requirements into final product.
  • Experience in installation, configuration, supporting and managing Hadoop clusters.
  • Extensively used ODI (Oracle Data Integrator) to perform ELT from heterogeneous sources using ODI tools - Security Manger, Topology Manager, Designer and Operator.
  • Implemented standards and processes for Hadoop based application design and implementation.
  • Expertise in all components of Hadoop Ecosystem- Hive, Hue, Pig, Sqoop, HBase, Flume, Zookeeper, Oozie, and Apache Spark.
  • Responsible for writing MapReduce programs using Java.
  • Logical Implementation and interaction wif HBase.
  • Developed MapReduce jobs to automate transfer of data from HBase.
  • Experience in developing Pig scripts and Hive Query Language.
  • Performed data analysis using Hive and Pig.
  • Used Hbase in accordance wif PIG/Hive as and when required for real time low latency queries.
  • Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie.
  • Expertise in Java/J2EE technologies such as Core Java, Struts, Hibernate, JDBC, JSP, JSTL, HTML, JavaScript, JSON.
  • Fluent wif teh core Java concepts like I/O, Multi-threading, Exceptions, RegEx, Collections, Data-structures and serialization.
  • Experience in Object Oriented Analysis Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.

TECHNICAL SKILLS

Applications: Tableau 9.0, MS Office, Excel, Word, PowerPoint, Eclipse.

Operating System: Windows 98/2000/XP/NT/Vista, Linux.

Languages: Python, PHP, C, JAVA 1.6, Scala, XML, CSS.

Internet Technologies: ASP.NET, HTML, JavaScript.

BI/ETL Tools: OBIEE 12c, 11g,Informatica 9.1, DataStage.

Hadoop Ecosystem: Hadoop MapReduce, Hive, Pig, HBase, HDFS, Zookeeper, Oozie, Sqoop.

Database: NoSQL, MySQL, Oracle 11g/10g.

PROFESSIONAL EXPERIENCE

Confidential, San Jose, CA

Hadoop Developer/Hadoop Admin

Responsibilities:

  • Coordinated wif business customers to gather business requirements.
  • Install and maintain teh Hadoop Cluster and Cloudera Manager Cluster, Hadoop ecosystem like HDFS, Yarn, Sqoop, Hive, HBase.
  • Importing and exporting data into HDFS from database and vice versa using Sqoop.
  • Responsible for managing data coming from different sources.
  • Worked on analyzing Hadoop cluster and different Big Data analytic tools including Pig, Hbase database and Sqoop.
  • Built various graphs for business decision making using Python matplotlib library.
  • Used Pandas library for statistical Analysis.
  • Good knowledge on Yarn and HDFS implementation in cluster for better performance.
  • Used Unit Test Python library for testing many Python programs and block of codes.
  • Working under UNIX environment in development of application using Python and familiar wif all of its commands.
  • Skilled in using collections in Python for manipulating and looping through different user defined objects.
  • Performed Data Ingestion from multiple internal clients using Apache Kafka.
  • Load and transform large sets of structured and semi structured data.
  • Collecting and aggregating large amounts of log data using Apache and staging data in HDFS for further analysis.
  • Analyzed data using Hadoop components Hive and Pig.
  • Developed a data pipeline using kafka, Spark and Hive to ingest, transform and analyzing data.
  • Involved in running Hadoop streaming jobs to process terabytes of data.
  • Gained experience in managing and reviewing Hadoop log files.
  • Involved in writing Hive/Impala queries for data analysis to meet teh business requirements.
  • Worked on streaming teh analyzed data to teh existing relational databases using Sqoop for making it available for visualization and report generation by teh BI team.
  • Involved in creating teh workflow to run multiple Hive and Pig jobs, which run independently wif time and data availability.
  • Analysed teh SQL scripts and designed teh solution to implement using Scala.
  • Developed Spark scripts by using Python as per teh requirement.
  • Developed Pig Latin scripts for teh analysis of semi structured data.
  • Imported data using Sqoop to load data from MySQL to HDFS on regular basis.
  • Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Spark SQL using python and Scala.

Environment: Hadoop, Cloudera Manager, HDFS, Hive, Pig, HBase, Kafka, Sqoop, Yarn, SQL, Java (jdk 1.6), Eclipse, Python.

Confidential, San Francisco, CA

Hadoop Developer

Responsibilities:

  • Hands on using log files and to copy them into HDFS.
  • Hands on writing Map Reduce code to make unstructured data as structured data and for inserting data into HBase from HDFS and Yarn.
  • Good understanding and experience in spring frame work and Restful API’s development.
  • Developed java classes using spring framework.
  • Migrated teh data from cluster into teh AWS environment.
  • Created streaming cubes wif data from Kafka and persists into HBase for building cubes.
  • Launching and Setup of Hadoop Cluster on AWS.
  • Created tables on top of data on AWS S3 obtained from different data sources.
  • Experience in creating integration between Hive and HBase.
  • Experience indefining jobflows and wrote some simple to complex Map Reduce jobs as per teh requirement.
  • Working on Agile scrum methologies.
  • Involved increating Hive tables, loading wif data and writing hive queries.
  • Implemented JSONData loading using Spark, Storm, Kafka and Elastic Search.
  • Implemented business logic by writing Pig and Hive UDFs for some aggregative operations and to get teh results from them.
  • Experience using IDE tools like Eclipse and Spring Tool Suit.
  • Experience in developing multiple MapReduce programs in java for data extraction, transformation andaggregation from JSON.
  • Hands on experience in exporting teh results into relational databases using Sqoop for visualization and to generate reports for teh BI team.
  • Experienced wifNoSQLdatabase and handled using teh queries.
  • Monitored teh health of Map Reduce Programs which are running on teh cluster.
  • Involved in loading data from UNIX file system to HDFS.
  • Installed and configured Hadoop Map Reduce, Yarn, HDFS and Hive, Pig, Sqoop and Oozie on theHadoopcluster are installed and configured.
  • Cloudera Manger was used to monitor and manage teh Hadoop Cluster.

Environment: Hadoop, Map Reducer,Cassandra, Cloudera Manager, HDFS, Hive, Pig, Kafka, HBase, Sqoop, Oozie, AWS, SQL, Java (JDK 1.6), Eclipse.

Confidential, Boston, MA

Hadoop Developer

Responsibilities:

  • Hadoop Installation and configuration on Cloudera platform.
  • Written queries in Hive QL and Pig Scripts for reporting purpose.
  • Developed teh Map-Reduce programs and defined teh job flows.
  • Developed a data pipeline using Kafka and Storm to store data into HDFS.
  • Manage and review Hadooplog files.
  • Supported/Troubleshoot Map-Reduce programs running on teh cluster.
  • Loaded teh data from Linux/UNIX file system into HDFS.
  • Installed and configured Hive and written Hive UDFs.
  • Created teh tables, loaded teh data, and written teh queries in Hive.
  • Developed teh scripts to automate routine DBA tasks using Linux/UNIX Shell Scripts (i.e. database refresh, backups, monitoring etc.).
  • Worked on Agile methologies.
  • Modifying SQL queries for batch and online processes.
  • Managed teh cluster through performance tuning and enhancement.

Environment: CDH - 4.1.2, HDFS, HBase, MapReduce, Kafka, Hive, PIG, Oozie, Eclipse.

Confidential, Houston, TX

OBIEE Developer

Responsibilities:

  • Involved in teh Analysis, functional and technical specifications, development, deployment and testing of teh project.
  • Gatheird business needs by Interacting wif teh business users for analytical data requirements.
  • Requirement validation of teh reports and drilldowns.
  • Assisted in designing repository based on business requirements, followed design best practices for teh RPD and Dashboard designs. Implemented Star schema/snow flake schema methodologies.
  • Customized teh OBIEE Repository (physical, BMM, and presentation layers) and worked on teh design of logical data model.
  • Used scrum agile methodology to improve and reduce cost and time.
  • Worked on repository and session variables.
  • Performed production support for teh project.
  • Created groups in teh repository and added users to teh groups and granted privileges explicitly and through group inheritance.
  • Generated Reports and Dashboards by using Report features like Pivot tables, charts and view selector.
  • Exposure to Medical code sets: ICD, CPT and HCPCS.
  • Proficiency in Informatica Designer Components (source analyzer, warehouse designer, mapping designer, mapplet designer, transformation developer, workflow manager and workflow monitor).
  • Experience in Installing, configuring and customizing DAC(execution plans, subject areas, tables, task) and monitoring ETL process using DAC &Informatica workflow manager.

Environment: OBIEE 11.1.1.6, Oracle11g, SQL, Oracle SQL developer,ODI, zLinux.

Confidential, Warren, NJ

OBIEE Developer

Responsibilities:

  • Developed proof of concept to check teh functional feasibility of teh project.
  • Used Informatica Power Center for Financial Data Extraction, Data Mapping and Data Conversion.
  • Created mappings using various transformations like Source Qualifier, Lookup, Update Strategy, Router, Filter, Sequence Generator, and Joiner on teh extracted source data according to teh business rules and technical specifications.
  • Developed simple and complex mappings for financial data using Informatica to load Dimension and Fact tables as per STAR schema techniques.
  • Developed a number of Informatica Mappings, Mapplets and Transformations to load data from relational and flat file sources into teh data warehouse.
  • Designed and developed teh OBIEE Metadata Repository (.RPD) of financial analytics using OBIEE Admin tool by importing teh required objects (Dimensions and Facts) wif integrity constraints into Physical Layer using connection pool, developing multiple Dimensions (Drill-Down, Hierarchies) and Logical Facts / Measures objects in Business Model Layer, and creating teh Presentation catalogs in Presentation Layer.
  • Identified granularity level of teh Financial data required to be available for analysis.
  • Created Security settings to setup groups, access privileges, query privileges and Implemented Object level as well as Date level security for teh end users using OBIEE Admin tool.
  • Worked extensively on OBIEE Answers to create teh Financial Statement reports and Intelligence Dashboards (Financials) as per teh detailed design requirements.
  • Extensively used page prompts and dashboard prompts for filtering data values in financial reports.
  • Created drill down charts and drill down tables to gather more information on Financial Analytics using navigation.
  • Developed BI-Publisher end user reports which have financial formatting capabilities meeting teh end-user needs and integrated Prompts between teh BI-Publisher and OBIEE using teh Presentation Variables.
  • Used SQL queries and database programming using PL/SQL (writing Packages, Stored Procedures/Functions, and Database Triggers).

Environment: OBIEE 10.1.3.4, DAC, Oracle BI Apps 7.9.5/7.9.6 , Informatica 8.1.1, Oracle EBS, BI Publisher, Windows XP.

Confidential

ETL Developer

Responsibilities:

  • Designing ETL jobs as per business requirements.
  • Developing ETL jobs wif organization and project defined standards and processes
  • Develop Unit Test Cases.
  • Implemented UNIX shell scripts to invoke teh Datastage jobs.
  • Assisted Systems Administrator in DataStage installation and maintenance.
  • Deployment to QA and hence Production.
  • Unit Testing
  • Coordinating wif onshore to ensure an issueless delivery.

Environment: DB2, UNIX Shell Scripting, Oracle 10g, PL/SQL, Data Stage.

Confidential

Software Engineer

Responsibilities:

  • Understanding requirement and developing modules.
  • Preparing detailed test plans, acceptance criteria and test scenarios for each project.
  • Involved in Performance Testing, Scalability / Stress and Load Testing.
  • Testing full product suite's, identifying problems & resolving them wif teh team.
  • Worked wif Complex SQL queries, Functions and Stored Procedures.
  • Involved in coding, maintaining, and administering Servlets and JSP components to be deployed on a WebSphere application server.
  • Developed programming module for loading data into data warehouse.

Environment: Java, SQL Server, JBuilder.

Confidential

Software Engineer(Intern)

Responsibilities:

  • Developing and maintaining a detailed project plan.
  • Code maintenance - bug fixing and enhancements, designing and implementing new codes as per teh business requirements.
  • Technical support for business issues.
  • Responsible for designing and developing GUI and Protocol Utilisation Modules.
  • Implemented a Statistics-Based Packet Filtering Scheme against Distributed Denial-of-Service Attacks.
  • Defining requirements for detailed project plan.
  • Recording and managing project and escalating where necessary.

Environment: Java, Winpcap, Swings, MySQL.

We'd love your feedback!