We provide IT Staff Augmentation Services!

Sr. Hadoop Developer Resume

0/5 (Submit Your Rating)

Cranston, RI

SUMMARY

  • 6 years of experience with full capability of identifying and prioritizing business information needs, analyzing business requirements, research data, design, development, testing and support.
  • Over 2 year of experience in Hadoop using Hadoop ecosystems like PIG, HIVE, HBASE and 4 years on DataStage 7.x/8.x (IBM InfoSphere Information server) environments.
  • Expertise in developing PIG Latin Scripts and using Hive Query Language for data Analytics.
  • Deep knowledge in Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, NameNode, DataNode and MapReduce programming paradigm.
  • Experience in developing applications using Oracle 10g/11g, SQL Server 2005/2008, DB2, HDFS, job scheduling and monitoring using tools like Oozie and ZooKeeper.
  • Working knowledge with core Java and writing MapReduce jobs in Java.
  • Extensive knowledge in developing and analyzing the data modeling models such as E/R model and Dimensional model and Implemented data warehousing methodology using Star schema models as per the business process.
  • Experience in Data profiling, Data Modeling using tools like Erwin, Oracle, TOAD, and Designer for OLTP and Data Warehousing environment.
  • Experience in writingPL/SQL Stored Procedures, Functions and UNIX shell scripts.
  • Proficient in building analytical repository, business models, generating reports, and creating dynamic Dashboards in Siebel Analytics (OBIEE) using Administration Tool, Siebel Analytics Web, Answers, and Interactive Dashboards.
  • Process centric and strong team contributor with good communication skills and ability to work independently and as part of a team.
  • Great learning skills to adapt the rapid changing technologies and implementing the same at work.

TECHNICAL SKILLS

Programming Languages: SQL, PL/SQL, C, C++, UNIX, LINUX, PIG and Java

Data warehousing Tools: IBM InfoSphere 8.x/7.x DATASTAGE - Designer, Director, Manager, Administrator, Parallel Extender, Workbench, Quality Stage and Manager

Development Tools: SQL* Plus, BTEQ, SQL*Loader, TOAD, MS Query, SQL Developer, Discoverer 10g, OBIEE, Aqua Data Studio, Oozie and ZooKeeper

Hadoop Distributions: Cloudera, Hortonworks, MapR & Greenplum

Hadoop Eco System: HDFS, HIVE, PIG and HBASE

Reporting Tools: Oracle Discoverer, OBIEE

Databases: Oracle 11g/10g, Informix, DB2, TERADATA, SQL Server, MS Access

Operating System: UNX Sun Solaris 9.0, Unix, Windows, Linux

PROFESSIONAL EXPERIENCE

Confidential, Cranston, RI

Sr. Hadoop Developer

Environment: CDH4, HADOOP 2.0, OOZIE, HDFS, MAPREDUCE, HIVE, JAVA, CLOUDERA, MySQL, SQL, IBM INFOSPHERE DATASTAGE 8.5, DB2 UDB 8.0, SQL SERVER 2008, UNIX, AQUA DATA STUDIO 9.0

Responsibilities:

  • Involved in multiple pilots to assess the power of BigData for Capital Markets and Business Capital Domains.
  • For business capital, used data processed by ETL and provided analytics using Hadoop eco system tools.
  • For Capital Markets, followed parallel processing approach where Hadoop processing runs along with existing ETL jobs to provide close functionality.
  • Responsible for spooling data from DB2 sources to HDFS using sqoop.
  • Created HIVE tables and provided analytical queries for business user analysis.
  • Followed agile methodologies in implementing pilot projects.
  • Involved in Requirement gathering by participating in business meetings, understanding existing data flows, existing sources, and reviewing technical documentation.
  • Worked on writing PIG's Load and Store functions.
  • Exported analyzed data to relational databases using Sqoop for visualization to generate reports for the BI team.
  • Supported marketing and personalization strategies using Hadoop tools PIG and HIVE.
  • Created and maintained technical documentation for launching HADOOP Clusters and for executing HIVE queries and PIG scripts.

Confidential, Cleveland, OH

Sr. Hadoop Developer

Environment: APACHE HADOOP, JAVA, PIG, MySQL, CLOUDERA, SQOOP, OOZIE, ORACLE 10g, LINUX

Responsibilities:

  • Participated in all phases including Requirement Analysis, Design, Coding, Testing, Support and Documentation.
  • Developed PIG scripts for analyzing large datasets for analysis on customer behavior for personalized offers.
  • Involved in processing datasets that loads everyday from social websites to analyze the reviews written by customers on websites, to generate reports in order to better reach customer satisfaction.
  • Extensive knowledge on PIG scripts using bags and tuples.
  • Created tables in HIVE by partitioning and bucketing for granularity and optimization of HIVEQL.
  • Experienced working on structured, semi structured and unstructured data to load into HDFS and for further processing using Hadoop ecosystem tools.
  • Developed simple to complex MapReduce jobs using Hive and Pig.
  • Assisted in exporting analyzed data to relational databases using Sqoop.
  • Wrote Hive Queries and custom UDF's according to use cases.
  • Involved in identifying job dependencies to design workflow for Oozie and resource management for YARN.

Confidential, Dallas, TX

Data Engineer

Environment: DATASTAGE 8.0, OBIEE 10g, ERWIN, ORACLE, UNIX, CRT-M, TERADATA

Responsibilities:

  • Participated in discussions with Project Manager, Business Analysts and Team members on different Business Requirement issues.
  • Used Erwin Data Modeler to build the logical and physical data models.
  • Worked on views as part of enhancement of code and changed query as per the requirement as views data has been used as source for some jobs.
  • Worked on extracting data from Flat files, Sequential files, SQL Server and Db2 database.
  • Designed ETL jobs by identifying various source files and applying transformations and ETL logic to them.
  • Prepared technical specification document for the project and identified the business requirements.
  • Used Toad (a graphical user interface for Oracle 10g) extensively for creating and editing database objects such as tables, views, indexes, constraints etc.,
  • Developed various jobs using ODBC, Aggregator, Sequential file stages.
  • Used the DataStage Director and its run-time engine to schedule running the solution, testing and debugging its components, and monitoring the resulting executable versions.
  • Worked with job Sequencers and used different job activities such as Command Activity, Wait-For-File activity, Zero row count check.
  • Did research and analysis on data and code in to analyze the defect whether it is source data related or code.
  • Used control-M to schedule the jobs based on prerequisites in and dependencies out of the jobs.
  • Involved in generating reports using OBIEE Reporting tool version 10gR3
  • Created schema and public objects (facts, attributes, filters, reports, hierarchies) using OBIEE.
  • Developed the reports for the Management team using OBIEE and made available on web for the users.

Confidential, Dayton, OH

Data analyst & developer

Environment: DATASTAGE 7.5, Oracle 10g, UNIX, AUTOSYS, SQL SERVER 2005

Responsibilities:

  • Actively involved in building Data Warehouse. Used Ascential DataStage for extracting, transforming, sorting and integrating the data from various source databases, flat files and loading them into base tables of the warehouse.
  • Involved in retrieving the data from source database using shell scripts, SQL scripts, Stored Procedures and the data is mapped into the target database.
  • Used DataStage Manager to import and export jobs from Development to Sit.
  • Analyzed data issues for mapping to existing database structures.
  • Involved in developing and applying business rules to enhance data consistency.
  • Involved in designing Star schema data model for organizing the data in data warehouse using Erwin data modeler.
  • Developed various jobs using stages like Transformer, Lookup, Funnel, Change Capture Data Capture stage, ODBC stage etc.,
  • Developed multiple instance jobs as per the requirement for reporting purpose to validate source and target data.
  • Created Shared containers to use in multiple jobs.
  • Developed various jobs using ODBC, Aggregator, Sequential file stages.
  • Scheduled various ETL batch processes using AUTOSYS.
  • Involved in the Performance tuning of DataStage Jobs at different levels (source and target levels)
  • Designed and implemented slowly changing dimensions methodologies.
  • Performed Unit testing for jobs developed to ensure that it meets the requirements.

Confidential

Programmer Analyst

Environment: DATASTAGE 7.5, ERWIN, ORACLE 10g

Responsibilities:

  • Involved in requirement gathering in co-ordination with business Analysts.
  • Designed ER Diagrams and worked with other team members in Designing database schemas.
  • Worked on designing detailed design documents, unit test cases and job constructions.
  • Worked with DBA’s to develop dimensional model.
  • Involved in coding DataStage jobs.
  • Involved in scheduling DataStage jobs.
  • Involved in designing data conversion systems from legacy system to ERP system.
  • ETL development including creation of data mappings and workflows, jobs scheduling, maintenance and enhancement of custom ETL code..

We'd love your feedback!