Hadoop Consultant Resume
New, YorK
SUMMARY
- 8+ years of overall experience in Oracle PL/SQL, Oracle Warehouse Builder, HPPPM,, 2+ years of experience in Development of Big data Hadoop Framework with industry experience in Telecom, CRM and Consumer Business.
- Proficient in the area of project implementation (SDLC )specifically in integration of business intelligence strategy, requirement gathering, requirement analysis, data modeling, information processing, system design, testing and training.
- Possess experience in conducting current state (as - is) system analysis, defining future state (to-be), eliciting requirements, developing functional or technical requirements, mapping business processes to application capabilities, conducting fit gap analysis, developing/configuring/prototyping solutions, implementing to-be processes / solutions based on application capability and industry best practices.
- In depth knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, and YARN / MapReduce programming paradigm and for working with Big Data to analyze large data sets efficiently.
- Hands on experience in working with Ecosystems like Hive, Pig, Sqoop, Map Reduce, and Flume.
- Hands-on experience with related/complementary open source software platforms and languages (e.g. Java, Linux, Unix, Python).
- Experience in importing and exporting terra bytes of data using Sqoop from HDFS to Relational Database Systems and vice-versa.
- Experience in analyzing data using HiveQL, Pig Latin, and custom MapReduce programs in Java.
- Hands On experience in NoSQL database like HBASE. Knowledge of job workflow scheduling and monitoring tools like Oozie and Zookeeper
- Experience working on processing data using Pig and Hive. Involved in creating Hive tables, data loading and writing hive queries
- Followed Test driven development of Agile and scrum Methodology to produce high quality software
- Experience working in the Cloud environment like Amazon Web Services (AWS)
- Good knowledge on Integrating the BI tools like Talend with the Hadoop stack and extracting the required Data .
- Experience in working with the Different file formats like TEXTFILE, AVRO and JSON.
- Good knowledge on Hadoop administration activities such as installation and configuration of clusters using Apache and Cloudera
- Involved in various projects related to Data Modeling, System/Data Analysis, Design and Development for Data warehousing environments. Strong knowledge on ETL methods, Developed mapping spreadsheets for (ETL) team with source to target data mapping with physical naming standards, data types, volumetric, domain definitions, and corporate meta-data definitions.
- Established and maintained comprehensive data model documentation including detailed descriptions of business entities, attributes, and data relationships.
- Comprehensive knowledge and experience in process improvement, normalization/de-normalization, data extraction, data cleansing, data manipulation, performance tuning. Scheduling PL/SQL jobs in Toad.
- Worked on Database and ETL Testing - Functions, Stored procedures, packages, Constraints, Loading data into tables, Executing scripts. Performed Test case preparation/execution and Defect Management as well.
- Experience in Object Oriented Analysis and Design, Java/J2EE technologies including HTML,XML.
- Possess experience with working on large projects/teams and the ability to mentor junior developers as a good team player, quick learner as have worked in different teams.
- Demonstrated success many times under aggressive project schedules and deadlines, flexible, result oriented and adapts to the environment to meet the goals of the product and the organization
- Excellent work ethics, self-motivated, quick learner and team oriented. Continually provided value added services to the clients through thoughtful experience and excellent communication skills
TECHNICAL SKILLS
Hadoop/Big Data: HDFS, Map Reduce, Hive, Pig, Sqoop, Flume, Oozie, Talend, Spark and Zookeeper.
NoSQL Databases: HBaseRelation
Databases: Oracle 7i, 8i, 9i, 10g and 11g,, Sql Server
Languages: C, C++, Java, PL/SQL,SQL, Pig Latin, HiveQL, Unix shell scripts, Python
Operating Systems: UNIX, Red Hat Linux, Ubuntu Linux and Windows XP/Vista/7/8
Web Technologies: HTML, XML
Tools: and IDE: Toad, SQL Developer, Eclipse, Oracle Warehouse Builder (ETL), Talend, PPM, ADT, Clearcase,. Microsoft Project Server(MSPS), Microsoft Office suite desktop applications (e.g., MS Visio, Word, Excel, PowerPoint)
Networks: TCP/IP, LAN / WAN
PROFESSIONAL EXPERIENCE
Confidential, New York
Hadoop Consultant
Responsibilities:
- Involved in running Hadoop jobs for processing millions of records of text data
- Data validation between existing system and new cluster.
- Written the Apache PIG scripts and Python to process the HDFS data. Developed Map Reduce program for parsing and loading into HDFS information.
- Developed PIG UDF'S for manipulating the data according to Business Requirements and also worked on developing custom PIG Loaders
- Developed Pig Latinscripts for transformations,sort, group, event joins, filter.
- Assisted in exporting analyzed data to relational databases using Sqoop
- Developed multiple MapReduce jobs in java for data cleaning and preprocessing
- Involved in analysis,ETLdesign and development for extracting data from different interfaces like SQL, Flat Files.
- Involved in creating Hive tables, designing patterns, and loading and analyzing data using hive queries
- Involved in Design, develop Hive Data model, loading with data and writing Java UDF for Hive. Used the Used hIVE Windowing and analytical functions for data Analysis.
- Worked with file formats TEXT, AVRO, JSON files.
- Involved in loading data from LINUX file system to HDFS
- Performed Data Loading Techniques through Hive and HBase, ETL through Talend
- Hands on experience in writing Spark SQL scripting.
- Deployment plan across the different applications (products)
- Analyzed business requirements and cross-verified them with functionality and features of NOSQL databases like HBase.
- Created HBase tables to store variable data formats of data coming from different applications
- Load and transform large sets of structured, semi structured and unstructured data
- . Worked on Transporting data from HBASE to HIVE using map reduce and HIVE - Hbase storage handlers.
- Identifying Hadoop Configuration changes. Integrated test plan across the applications
- Experience in moving all files generated from various sources to HDFS for further processing through Flume.
- Peer Code Reviews and assisting the team in full project life cycle implementation (requirement gathering to production)
- Setting up monthly call with Dev, support, Test team to discuss production issues .
- Worked with Test team test cases reviews call and suggesting different kind of test scenarios.
- Research and troubleshoot technical problems and provided business logic consultation to the development team.
- Provided data flow after merging various workflows.
- Delivering complex PL/SQL queries to Testing team to test data flow across different system
- Providing Production Support to the Applications. Create, maintain and develop documentation.
- Working on Datastage to fetch various reports.
- Providing SQL script to support and test team to capture data issues in production database
Environment: Hadoop, Apache Pig, Hive, Sqoop, Mapreduce, Python, Hbase, Talend, Toad Oracle, SQL PLUS, Linux
Confidential, Connecticut
Hadoop Developer/ Pl SQL Developer
Responsibilities:
- Performed current state system analysis, developed functional and technical specifications, enhancing existing objects like procedures, functions, packages, views etc and maintaining documentation.
- Involved in development and design of a Hadoop cluster using Apache Hadoop for POC and sample data analysis.
- Loading the data from source system to HDFS and imposing Hive Schema on the data
- Analyzed large data sets by running Hive queries and Pig scripts
- Enhancing existing database objects like procedures and views. Implemented Performance Tuning.
- Database Testing - Testing of Functions, Stored Procedures, Packages, PL-SQL scripts; ensuring loading of correct data in target tables, preparing test cases.
- Performing interface implementation and execution.
- Performing Defect Management - Logging the defects in the defect tool and retesting the fixed scripts.
- Provided offshore training & team building activities.
- Worked on assigning task to offshore team and making sure that team is working on time.
- Loading of Data in tables using SQL*Loader - Toad.
- Working with test and production support team to get application info
- Working on post production support
- Building complex sql queries to get data from database
Environment: Hadoop, Hive, Pig, Toad Oracle, PL/SQL, SQL PLUS, BI Objects, Sql Developer
Confidential, Burbank
Technical Analyst/ Developer
Responsibilities:
- Gather business requirements from Business Users and develop the Technical and Functional Specs. Requirement and Feasibility Analysis of the activities assigned.
- Self trained on OWB(Oracle Warehouse Builder). The project started from the scratch so involved in Impact Analysis of the system. Delivering the activities of OWB including all the phases of SDLC, performed current state analysis (as - is and to-be),implemented Data Modeling, Performing RCA of the issues raised in OWB and delivering the assignment without any defects. Trained other members on OWB knowledge and activities.
- Database Testing - Testing of Functions, Stored Procedures, Packages, PL-SQL scripts; ensuring loading of correct data in target tables, preparing test cases.
- Handled this client independently for 8 months.
- Performing Defect Management - Logging the defects in the defect tool and retesting the fixed scripts.
- Working on the creation of Views, Materialized Views, Procedures, Functions, Triggers, Hints and Performance Tuning.
- Enhancing existing database objects like procedures and views.
- The project started from the scratch so also involved in Impact Analysis of the system.
- Performed Microsoft Project Server (MSPS) task creation and reviewing, approving of hrs logged by the team for project portfolio management.
- Coded Pl/SQL block to fetch different VOIP services with their sales order
- Making PL/SQL code change as per new Business Requirement.
- Providing training to new Associates in team
Environment: Toad Oracle, PL/SQL, SQL PLUS, OWB, Sql Developer
Confidential
Requirement Analyst/Developer
Responsibilities:
- Handling client interactions for requirement gathering fit gap analysis, preparing deliverables and performing the feasibility analysis for the same. Developed Functional and Technical Docs.
- Have worked as Application Developer for coding, testing and maintaining the programs, including the development and maintenance of custom applications; documenting the changes and the results as well.
- Developed various reports, requests, package creation and other customizations based on PL SQL as per the requirements of the client.
- Have also worked for Risk Assessment in identifying and assessing risks for all the customization done.
- Involved in Problem Team for root cause analysis for resolving issues of different clients.
- Loading of data using SQL*Loader - Toad, worked on UNIX Shell Scripting.
- Exposure in client/server configuration, conceptual knowledge of web server, application server, database server and server services.
- Extensively worked in managing overall queue for change requests and assignment of these to the whole the team.
- Have worked on end to end implementation of the change requested by the client starting from initial requirement gathering, providing estimates till the deployment of the change to client environments
Environment: Toad Oracle, PL/SQL, HP PPM,UNIX
Confidential
Pl-SQL Developer
Responsibilities:
- Participating client Business call to understand new requirements.
- Providing transition to team to understand the flow of data to downstream.
- Performed Database Testing to check data validity and integrity on SQL Server.
- Making PL/SQL code change as per new Business Requirement.
- Worked on creating anonymous PL/SQL code.
- Working with team, providing them the process document to understand applications
- Writing complex SQL queries to check data in database
- Implementing anonymous Pl/Sql block to check data
Environment: TOAD, Sql Developer, MSSQL
