Sr.hadoop /aws Developer Resume
2.00/5 (Submit Your Rating)
Iselin, NJ
SUMMARY:
- Over 15 Years of IT industry experience as a JAVA Developer, SAP HANA with Business Objects Data Services and Hadoop / AWS implementation consultant. .
- Design and implement Big Data analytic solutions on a Hadoop / Cloud based platforms.
- Design and build data processing pipeline focused on unstructured and semi - structured data refinement.
- Designing, building, installing, configuring and supporting Hadoop cluster using Cloudera and Hortonworks Hadoop distributions.
- Translate complex functional and technical requirements into detailed design.
- Participate in deep architectural discussions to build confidence and ensure customer success when building new solutions and migrating existing data applications on the AWS platform.
- Lead the discussions and early analysis of the cloud concepts as it relates to Clients’s Journey to Cloud methodology so that clear use cases are developed and prioritized as well as transitioning these concepts from ideas to proof-of-concepts all the way to production delivery with the full support of the appropriate teams.
- Learning new and emerging technologies/patterns/methodologies and build quick proof of concepts of new data sources to showcase data capabilities and help analytics team identify key metrics and dimensions.
- Adapt to existing methods and procedures to create possible alternative solutions to moderately complex problems.
- Understand the strategic direction set by senior management as it relates to team goals.
- Explore and discover new data sources and quickly familiarize with the available APIs or other data acquisition methods like web-scraping to ingest data.
- Design, implement and manage a near real-time ingestion & integration pipelines.
- Exceptional analytical, conceptual, and problem-solving abilities.
- Craft technical solutions and assemble design artifacts, such as solution design documents, data flow diagrams and data models.
- Own the technical solution design, lead the technical architecture and implementation of data acquisition and integration projects.
- Self-motivated and curious towards learning new emerging technologies.
- Flexible in work environment and ready to accept challenges as per the organization needs.
- Curiosity in learning new technologies and sharing with team members.
- Involved in Confidential enterprise level meetings to share /acquire knowledge from other teams.
- Had an opportunity to evaluate various tools to build several Proof Of Concepts streaming / batch applications (AWS,Cloudera/Hortonworks Hadoop distribution/Nifi/Kafka/Talend/Mule ESB) to bring in data from multiple data sources, transform and load data in to target systems and successfully implemented in Production.
- Very good exposure in building applications using Cloudera/Hortonworks Hadoop distributions.
- Full lifecycle, hands on data integration and data delivery solutions using the Hadoop technology stack.
- Sourcing data from multiple applications, profiling, cleansing and conforming to a common standard.
- Hands on experience in developing AWS lambda functions using Python 3.6.
- Hands on experience in configuring lambda functions to extract data from external sources to S3 storage.
- Hands on experience in debugging lambda functions using AWS CloudWatch service.
- Hands on experience in dealing with various Hadoop storage formats.
- Design solutions for managing highly complex business rules within the Hadoop ecosystem.
- Hands on experience in real time data ingestion / streaming using Apache Flume / Kafka, Nifi/Kafka in to HDFS, Hive.
- Hands on experience in building and automating the Oozie workflows to ingest data in to target systems on regular intervals.
- Good exposure to installing, configuring and tuning Hadoop Cluster using Cloudera and Hortonworks Hadoop distribution vendors.
- Familiar with UNIX / LINUX OS.
- Experience with Agile methodologies.
- Certified in C HANAIMP 1 Application Associate - SAP HANA 1.0
- More than 5 years of experience in designing, developing and implementation of Hadoop based applications.
- More than 1 year of standalone SAP HANA implementation experience.
- More than 4 years of experience in designing, developing and implementation of SAP Business Objects Data Services.
- More than 4 years of experience as a JAVA developer in developing and implementing web based applications.
- Strong working knowledge of relational databases.
- Understanding of Project Management, business stake holders, issue logs, status meetings
- Thorough knowledge of SDLC process.
EXPERIENCE:
Confidential, Iselin, NJ
Sr.Hadoop /AWS Developer
Responsibilities:
- Successfully delivered first AWS Cloud Project in to Production in Confidential .
- Developed Hadoop based solutions using CDH(Cloudera) and Hortonworks.
- Good exposure in building and configuring Hadoop cluster using Cloudera and Hortonwork.
- Responsible for analyzing new data sources (Kafka,RDBMS,log etc) and bring in data in to target system in near real time.
- Full lifecycle, hands on data integration and data delivery solutions using the Hadoop technology stack.
- Build quick proof of concepts of new data sources to showcase data capabilities and help analytics team identify key metrics and dimensions.
- Educate other source system owners to push the data to enterprise data store(Kafka) to bring data in near real time in to target system.
- Coordinating with Data Science team to fulfill the request.
- Designed and Implemented real time streaming from Kafka using Mule-Kafka Connector.
- Designed and Implemented real time streaming from Kafka using Nifi.
- Successfully migrated existing batch mode applications in to near-real time streaming applications using Kafka and Mule.
- Explore and analyze various data sources.
- Coordinating with multiple source system owners to bring the data in to HDFS.
- Designing the workflow for data integration from various source systems.
- Sourcing data from multiple applications, profiling, cleansing and conforming to a common standard.
- Helping business users while executing queries against the Hadoop cluster.
- Coordinating with source system owners for subsequent data loads in to HDFS.
- Tracking the source data from multiple sources and monitoring the size of the data coming in to HDFS regularly.
- Implemented Hive best practices to improve Hive query performance.
- Scheduling data ingestion jobs using Oozie workflow scheduler.
- Involved in performance tuning of Hadoop cluster to optimize the performance.
- Hands on experience in writing Hive QL scripts for processing and querying the data stored in HDFS.
- Supporting and monitoring Hadoop cluster in production.
Confidential, Parsippany, NJ
Sr.Hadoop Consultant
Responsibilities:
- Full lifecycle, hands on data integration and data delivery solutions using the Hadoop technology stack.
- Coordinating with multiple source system owners to bring the data in to HDFS.
- Designing the workflow for data integration from external sources.
- Sourcing data from multiple applications, profiling, cleansing and conforming to a common standard.
- Handling adhoc business user requests in granting privileges, creating schema and providing space.
- Helping business users while executing queries against the hadoop cluster.
- Coordinating with source system owners for subsequent data loads in to HDFS.
- Tracking the source data from multiple sources and monitoring the size of the data coming in to HDFS regularly.
- Scheduling data ingestion jobs using Oozie workflow scheduler.
- Involved in performance tuning of Hadoop cluster to optimize the performance.
- Debugging Map / Reduce programs.
- Reviewing Hadoop log files for debugging.
- Hands on experience in writing Hive QL scripts for processing and querying the data stored in HDFS.
- Supporting and monitoring Hadoop cluster in production.
Confidential, Mount Laurel, NJ
Hadoop Consultant
Responsibilities:
- Good exposure in developing Hadoop solutions using CDH(Cloudera).
- Full lifecycle, hands on data integration and data delivery solutions using the Hadoop technology stack.
- Experience in configuring and installing Sqoop for moving structured data from multiple traditional RDBMS sources to Hive for analytic purpose.
- Sourcing data from multiple applications, profiling, cleansing and conforming to a common standard.
- Performance tuning of initial and incremental loads.
- Hands on experience in replicating data from traditional RDBMS to HDFS and HBASE
- Scheduling data ingestion jobs using Oozie workflow scheduler.
- Involved in performance tuning of Hadoop cluster to optimize the performance.
- Designing the workflow for data integration from external sources and API’s for real time streaming.
- Debugging Map / Reduce programs.
- Reviewing Hadoop log files for debugging.
- Hands on experience in writing Hive QL scripts for processing and querying the date stored in HDFS.
- Supporting and monitoring Hadoop cluster in production.
Confidential, Minneapolis, MN
SAP HANA Consultant
Responsibilities:
- Importing data from multiple sources.
- Data Profiling using Business Objects Data Services.
- Involved in creating Data Store for various sources in Data Services.
- Involved in creating Data Store for target SAP HANA in Data Services.
- Involved in designing the workflows, data flows and scheduling and executing the job for Delta load or Initial Load
- Involved in creating information models (Attribute, Analytic and Calculation Views) in HANA Studio.
- Involved in creating HANA table structure before initial data is loaded in to SAP HANA
- Involved in classifying the data while creating information models in HANA as per the business requirements.
- Experience in creating Attribute Views using Hierarchies.
- Implemented best practices while developing information models in SAP HANA.
- Import/Export Information Models.
- Knowledge of Creating Users and Roles in SAP HANA.
- Knowledge of consuming HANA content using Excel through MDX.
- Knowledge of Universe creation and consuming HANA content using Dashboards and WEBI
- Impact Analysis of information models due to the frequent changes in requirements
- Closely worked with business users to understand the requirements.
- Trained business users
- Responsible for sending daily status report to the project management.
- Attended weekly meetings and daily project status meetings.
Confidential, San Antonio, TX
SAP HANA Consultant
Responsibilities:
- Interacted with Business Users and Managers in gathering business requirements.
- Involved in meetings with functional users, to determine the flat file layouts, data types and naming conventions of the column and table names.
- Created mappings using Designer and designed workflows using Workflows to build DW as per business rules.
- Most of the transforms used were like the Source Lookup, Case, Merge, and Sequence Generator, Query Transform, Map Operations, Table Comparison, SQL, and Validation Transforms
- Used various Transforms like Map Operation, Table Comparisons, Row-Generation, Query History Generation, Key Generation, Pivot, Reverse Pivot and functions like lookup, index search for transforming data.
- Configured the mappings to handle the updates to preserve the existing records using History Preserve Transformation (Slowly Changing Dimensions Type-2)
- Developing and modifying changes in data flows according to the business logic.
- Understand technical issues and identify architecture and code modifications needed to support changing user requirements for multiple Data Services jobs and application
- Recovering Data from Failed sessions and Workflows.
- Experience in debugging execution errors using Data Services logs (trace, statistics and error) and by examining the target data.
- Involved in writing the Unit Test Cases.
- Involved in user sessions for and assisting in UAT (User Acceptance Testing)
- Provided Knowledge Transfer to the end users and created extensive documentation on the design, development, implementation, daily loads and process flow of the mappings.
Confidential, Marshall, MN
SAP Data Services Consultant
Responsibilities:
- Performed source data assessment and identified the quality and consistency of the source data.
- Extensively worked on Data Services XI 3.2 for migrating data from Source to Target datbases.
- Worked with tables involving Hierarchies and resolved them using Hierarchy flattening whenever required.
- Implemented Server groups to use advance performance tuning feature such as parallel data flow, dividing dataflow in to sub dataflow and Degree of Parallelism
- Made modifications to existing universes by adding new tables in the data warehouse, creating new objects and exported the universes to repository.
- Responsible for using various error handling techniques in data services to avoid duplication of rows and/or missing rows
- Responsible for creating/implementing both source and target based CDC using time stamp and history preserving transforms.
- Developed data loading strategies for master data, transactional data and delta/Full uploads
Confidential, Plano, TX
SAP Data Services Consultant
Responsibilities:
- Performed source data assessment and identified the quality and consistency of the source data.
- Extensively worked on Data Services XI 3.2 for migrating data from one database to another database.
- Worked with tables involving Hierarchies and resolved them using Hierarchy flattening whenever required
- Implemented Server groups to use advance performance tuning feature such as parallel data flow, dividing dataflow in to sub dataflow and Degree of Parallelism
- Made modifications to existing universes by adding new tables in the data warehouse, creating new objects and exported the universes to repository.
- Responsible for using various error handling techniques in data services to avoid duplication of rows and/or missing rows
- Responsible for creating/implementing both source and target based CDC using time stamp and history preserving transforms.
Confidential, San Antonio, TX
JAVA Developer
Responsibilities:
- Developed Java based applications.
- Client side validations.
- Involved in Unit Testing.
- Involved in Code reviews.
- Attended project status meetings.
- Coordination with QA and business teams.