We provide IT Staff Augmentation Services!

Senior Hadoop Architect Resume Profile

2.00/5 (Submit Your Rating)

Danbury, CT

Professional Summary

  • A dynamic professional with over 8 years and 8 months of experience in Banking and Financial Services domain and subject matter expert in Vendor Finance and Lease/Loans.
  • Have over 8 years and 8 Months of experience in implementing and supporting 100 TB size Data warehousing applications using tools like Informatica Power Center 9.5.1 , Decision experts Oracle , Teradata, Business Objects and many other reporting tools. Worked in various roles such as Project Lead, Senior ETL Architect, Data Analyst Data architect.
  • Have over 2 years of experience as Hadoop Architect with very good exposure on Hadoop Technologies like HDFS, Map Reduce, Hive, Hbase, Sqoop, HCatalog, Pig, Zookeeper, Flume, and Mahout.
  • Expert in understanding the data and designing/Implementing the enterprise platforms like Hadoop Data lake and Huge Data warehouses.
  • In-depth experience in translating key strategic objectives into actionable and governable roadmaps and designs using best practices and guidelines. Worked on all facets of software development life cycle.
  • Fine-tuned several complex ETL Reporting applications with a goal of providing faster and more efficient BI platform for business users.
  • Successfully led executed several Simplification Process Optimization initiatives to bring efficiency into business processes.
  • An effective communicator with excellent relationship building interpersonal skills. Follow effective time management and goal driven approaches. Strong analytical, problem solving organizational abilities and a good team player
  • Have strong relational Database concepts and worked on several databases such as Oracle and Teradata.

TECHNICAL SKILLS:

Big Data : Hadoop, Map Reduce, Hive, HBase, Mahout, Sqoop ,Pig, Zookeeper, Flume

ETL : Informatica PowerCenter V7, V8, V9

Database : Oracle, Teradata

OS : UNIX, Windows

Programming : SQL, Core Java, UNIX Scripting

Reporting : Business Objects BO XI R3 , Qlikview, Spot fire, Tableaue , Pentaho

Others : Informatica ILM Data Subset Data Masking, StarTeam Version Control Tool, SVN, Appworx Scheduler

PROJECT EXPERIENCE

Confidential

Senior Hadoop Architect

Confidential

Confidential to bring huge volumes of data from Confidential systems into Hadoop environment. The architecture for the Enterprise Data Hub is based on Pivotal HD and is Confidential. Confidential is an integrated analytics platform that accelerates analysis of Big data assets with in a single integrated appliance. The Enterprise data lake will be used to provide support for various uses cases including Analytics, processing, storing and Reporting of voluminous, rapidly changing, structured and unstructured data.

Confidential and Informatica 9.6 tools were used to inject the data into Data Lake. Mapreduce and Pig used for data processing and transformation . Confidential played role as staging and core databases. Exploring the feasibilities of spot fire, business objects and Tableau as the reporting tools.

Responsibilities:

  • Technical Lead for global team spread across 6 locations in US, Mexico and India.
  • Playing key role in designing Big data initiatives in GE Capital BI.
  • Understanding the data nature from different OLTP systems and designing the injection processes for HDFS
  • Using Informatica 9.6 and Sqoop.
  • Working on Hadoop File formats TextInputFormat and KeyValueTextInputFormat
  • Designing data model on Hbase and Hive.
  • Creating Mapreduce jobs for Adhoc data requests.
  • Partitioning and Bucketing techniques in hive to improve the performance.
  • Optimizing Hive and Hbase queries.
  • Designing HBase column schemas.
  • Creating common data interface for Pig and Hive using Hcatalog.
  • Understanding the business requirements and needs and drawing the road map for Big data initiatives.
  • Driving POC initiatives for finding the feasibilities of different traditional and Big data reporting tools with the data lake Spotfire BO, Tableue etc
  • Scheduling big data jobs using the in-house scheduler Appworx.
  • Using Kerberos and LDAP security authentications.
  • Implementing POC for big data tools like Mahout, Impala etc.
  • Driving initiative to automate the recurring manual activities for monitoring and operations using Unix Scripting.

Confidential

Senior ETL Lead and Team Lead

Confidential

The project provides IT systems support to Leasing and Loaning business of Confidential. System is of high business criticality and is in the process of undergoing considerable enhancements. These systems contain the subsystems as: Oracle warehouses for Equipment finance system warehouses, Enterprise Data warehouse, Appworx, Informatica 9.5.1 administration. Project uses the methodologies and tools of Six Sigma Design to make sure all the processes are 100 defect-free. This project includes documentation of existing jobs including Data Flow Diagrams , developing ETL components to enhance the existing process, working on problem and change requests, applying application upgrade patches, code deployments, providing support for online and batch processes for the above applications.

Responsibilities:

  • Designing and Supporting 80 TB Teradata and 20 GB Oracle warehouses.
  • Supporting and Enhancing 100 Data warehousing applications hosted on Teradata and Oracle.
  • Worked on Informatica Transformations like lookups, source qualifier, update strategy, router, sequence generator, aggregator, rank, filter jointer, sorter, mapplets, and web services, Java transformation.
  • Involved in query tuning, performance triage/troubleshooting and data distribution
  • Designed and implemented Primary Partitioning Indexes, Join Indexes, and other techniques for performance tuning.
  • Reviewing Architecture and Designs of all GE Capital Data warehousing designs.
  • Work on Informatica Power Center, Informatica PowerExchange for Metadata Analysis
  • Using FSLDM for all Teradata based warehouses and Start Schema for Oracle Data warehouses.
  • Driving initiatives to reduce the manual and recurring efforts.
  • Playing key role in drawing road map for GE Capital BI initiations.
  • Performance tuning by using PDO, redesigning legacy data marts.
  • Project Management related activities like task tracking, Resource allocations and status calls.

Confidential

Senior ETL Lead and Team Lead

Confidential

This project is to migrate the mainframe dependent BI applications to Unix based BI applications. Confidential portfolio management system which was hosted on Mainframes is a biggest source system for BI application. As the source system has been migrated from Mainframes to unix, all the dependent BI applications have been redesigned to support the source system configurations.

Responsibilities:

  • Gather requirements by interacting with business process owners to translate the business requirements in Technical requirements
  • High Level and low Level designing of the solution utilizing UML diagrams to provides team and users a framework for the effective transformation of data
  • Develop Informatica mappings, design workflows and create batches and implement.
  • Work on Informatica Power Center, Informatica PowerExchange for Metadata Analysis.
  • Work on Teradata physical data modeling and use Teradata SQLs for data analysis and ET designing.
  • Help debug existing ETL problem areas and provide effective solutions for problems including performance tuning of the batches
  • Review and Analyze different data source like mainframe files, DB2, COBOL files, oracle, SQL server and Teradata for data sourcing
  • Work on all Informatica Transformations like lookups, source qualifier, update strategy, router, sequence generator, aggregator, rank, filter jointer, sorter, mapplets, and web services, Java transformation.
  • Design and implement Primary Partitioning Indexes, Join Indexes, and other techniques for effecting database space management and archival.
  • Work on Shell Scripting, Bteq and SQL Scripts.
  • Develop test cases and coordinate system testing beta and regression testing with working group committee Ensure Walkthroughs Inspections are carried out as per standards.
  • Provide support during business user acceptance testing and release of the product into the production environment and post production support

Confidential

Program Analyst, ETL Developer

Confidential

This project involves the Maintenance, Support and Enhancements of Business Intelligence and Business reporting system of Capital Solutions GE Commercial Finance. This system caters to various reporting needs for business users. This helps the users in tend analysis of the data and use it effectively to improve the Decision making process for the Business users.

Responsibilities:

  • Develop information architecture and Manage current and future needs in data design and content.
  • Resolve semantic discrepancies in data definitions that arise among multiple sources and projects.
  • Determine the optimal approach for obtaining data from diverse source system platforms and moving it to the BI.
  • Design and document conceptual and logical data models for the data warehouse architecture to scale out for millions of transactions monthly
  • Involved in informatica mapping design, workflow design, creation and implementation.
  • Woked on Informatica Power Center, Informatica PowerExhange. Experience in Informatica development with Informatica ETL on Oracle/Teradata/SQL Platforms.
  • Used Teradata SQL and involved in Teradata physical data modeling.
  • Design and developed ETL and ELT modules, and involved in ETL performance tuning.
  • Developed and reviewed ETL workflows for multiple data sources like flat files, XML, Teradata, DB2, Oracle, SQL Server.
  • Worked on Informatica Transformations like lookups, source qualifier, update strategy, router, sequence generator, aggregator, rank, filter jointer, sorter, mapplets, and web services, Java transformation.
  • Involved in Teradata query tuning, performance triage/troubleshooting and Teradata data distribution
  • Establish and enforce technical standards for deliverables of projects.
  • Coach project ETL specialists on data acquisition techniques, decision rules and how to apply them.
  • Execute entire components of large BI Analytics projects from start to finish leading the development teams, mitigating issues, and communicating status to stakeholders
  • Ensure best practices across the BI team for development and architecture
  • Ensure optimal database performance and availability

Confidential

Program Analyst, ETL Developer

Confidential

This project involves the designing of various Data marts on Oracle Data ware housing as per requirements from Business users. This project sources the data from mainframes, Oracle and Mysql databases and loads the data into data warehouse using Informatica ETL mappings. Designed BO universe for finance reporting.

Responsibilities:

  • Requirements gatherings, design of Informatica Mappings based on the Business requirements
  • Building of New Mappings/Enhancements in the Mappings
  • Designing and building of data warehousing components and providing support for User Acceptance Testing and deployment of application
  • Risk Monitoring, Mitigation and Management
  • 24x7 support of the entire system
  • Load Monitoring of all batch jobs on daily basis. Identify failure Root Cause provide solutions proper reporting and escalation of all errors/issues
  • Perform effort Estimations for Change Requests do new enhancements in the system
  • Optimize the existing Load for improved performance Robustness
  • Designing and building of data warehousing components and providing support for User Acceptance Testing and deployment
  • Use of Clarify tool for managing user tickets

We'd love your feedback!