We provide IT Staff Augmentation Services!

Etl-hadoop Architect/developer Resume

4.00/5 (Submit Your Rating)

Wilmington, DE

SUMMARY

  • Fourteen years of software life cycle experience in ETL, Data Warehouse, Business Intelligence and Client Server applications
  • Experience in Big Data, Hadoop technologies
  • Worked on CDH, CDM, Hadoop environment (HDFS) setup, Map Reduce Jobs, HIVE, Hbase, PIG and NoSQL and MongoDB.
  • Worked on Multi Clustered environment and setting up Cloudera Hadoop echo - System

TECHNICAL SKILLS

Data Warehousing: Informatica PowerMart 4.7/5.0/6.2/7.0, Informatica Power Center 4.1/5.0/5.1/6.2/7.0/8.1.6/9.01, ETL Informatica PowerConnect.

Hadoop Ecosystem Development Hadoop: HDFS, Sun Grid Engine Administration, HIVE, PIG, Flume, Oozie, Zookeeper, HBASE and Sqoop.

Databases: Oracle 7.x/8.x/9i, MS SQL Server 6.5/7.0/2000, MS Access, DB2 UDB, SybaseOLAP Tools: Business Objects Designer 5.1, BO Supervisor, BO Reports, COGNOS Powerplay, Powerplay Transformer, Impromptu administrator and Impromptu.

Data Modeling: ER/Studio 3.5/4.0, Erwin 3.5/4.0.

Languages Visual Basic: 6.0, Visual Studio .NET, ASP, HTML, XML, Java Script, PL/SQL.

CRM Siebel Tools: eScript/Siebel VB, eBusiness Application Integration (EAI), Actuate Reports, Workflow Manager, Assignment Manager, EIM and Smart Scripts.

Other Software: VB.Net, ASP.Net, Perl, Microsoft Internet Information Server (IIS) 5.0, Microsoft Transaction Server (MTS), TOAD, SQL Station, SQL Navigator, Relational clearest, Crystal Reports 6.x/7.x., MS Windows NT/2000/98/95/XP, MS Office Professional, Solaris, Unix, Linux, MS - DOS.

PROFESSIONAL EXPERIENCE

ETL-Hadoop Architect/Developer

Confidential, Wilmington, DE

Responsibilities:

  • Created Java classes for AVRO file formats.
  • Worked with Maven for code migration.
  • Worked with GitHub to perform the version control.
  • Worked with crucible for code reviews.
  • Worked implementing third party libraries like Dozer, UtilBeans and Reflection.
  • Used Eclipse to write Java and Mapreduce programs.
  • Worked as ETL Architect to make sure all the applications are migrated (along with server) smoothly.
  • Deep understanding and related experience with Hadoop stack - internals, HBase, Hive, Pig and Map/Reduce
  • Deep understanding of schedulers, workload management, availability, scalability and distributed data platforms
  • Expert knowledge developing and debugging in Java/J2EE
  • Wrote Hive Queries and UDF s.
  • Wrote MapReduce jobs.
  • Implemented Fair schedulers on the Job tracker to share the resources of the Cluster for the Map Reduce jobs given by the users.
  • Upgrading the Hadoop Cluster from CDH3 to CDH4 and setup High availability Cluster Integrate the HIVE with existing applications
  • Automated all the jobs starting from pulling the Data from different Data Sources like MySQL to pushing the result set Data to Hadoop Distributed File System.
  • Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
  • Specifying the Cluster size, allocating Resource pool, Distribution of Hadoop by writing the specification texts in JSON File format.
  • Configured Ethernet bonding for all Nodes to double the network bandwidth
  • Automated all the jobs starting from pulling the Data from different Data Sources like MySQL to pushing the result set Data to Hadoop Distributed File System.
  • Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
  • Exported the result set from HIVE to MySQL using Shell scripts.
  • Develop HIVE queries for the analysts.
  • Helped the team to increase Cluster from 25 Nodes to 40 Nodes.
  • Wrote Nagios plugins to monitor Hadoop Name Node Health status, number of Task trackers running, number of Data Nodes running.
  • Maintain System integrity of all sub-components (primarily HDFS, MR, HBase, and Flume).
  • Monitor System health and logs and respond accordingly to any warning or failure conditions.

Environment: Hadoop, HDFS, MapReduce, Hive, Pig, Eclipse, Java, AVRO, GitHub, Maven, Scoop, Oracle 9i/10g, SQL Server. MySQL, ER/Studio 3.5/4.0, UNIX Shell Scripting, SQL, PL/SQL, TOAD, Transact-SQL, Windows NT 4.0, SQL Sever Management Studio, MySQL WorkBench. Informatica PowerCenter 9.0.1, Data Quality 9.0.1, Netezza 4.x

Informatica Architect/Lead

Confidential, Chesterbrook, PA

Major Responsibilities:

  • Prepared the check list to upgrade the existing informatica 8.1.6 to informatica 9.0.1.
  • Upgrade the Informatica server and client tools.
  • Manages and mentors a team of five ETL developers.
  • Involves in making Architecture decisions and technical road maps.
  • Establish coding standards, migration methodology and naming conventions.
  • Communicates with business users to gather requirements.
  • Preparing high and low level ETL design documents.
  • Created NZLoad and NZSQL frameworks for Netezza.
  • Work with the team to set up their goals and do the performance reviews.
  • Talk to the external and internal vendors about the data feeds.
  • Delivered project from beginning to end.
  • Built ETL for calculate the TOT for Veeva and for other call data.
  • Involve in decision making of the technical tools.
  • Prepare project plan and assign tasks to the direct reports. Work with them to accomplish their tasks on time.
  • Promoting informatica processes from Dev to QA and from QA to Production.
  • Worked on identifying and setting up proper distribution keys.
  • Good knowledge on Netezza architecture like Zone maps, distribution keys etc
  • Created design documents for ETL mappings.
  • Involved in gathering requirements.
  • Coordinate with Oracle DBA and Unix Admin to achieve better dB performance and identify performance bottlenecks.

Environment: Informatica PowerCenter 9.0.1, Data Quality 9.0.1, Netezza 4.x, Oracle 9i/10g, SQL Server. MySQL, ER/Studio 3.5/4.0, UNIX Shell Scripting, SQL, PL/SQL, TOAD, Transact-SQL, Windows NT 4.0, SQL Sever Management Studio, MySQL WorkBench.

Informatica-Netezza ETL Architect/Lead

Confidential, Wilmington, DE

Responsibilities:

  • Review the existing informatica mappings and create design documents to migrate them to Netezza.
  • Converted the existing XPONENT process into ELT to achieve good performance. Due to converting to ETL the load time was reduced to 4 hours from 70 hours.
  • Converted Oracle Materialized views into Netezza views.
  • Worked on performance and tuning netezza queries.
  • Worked on identifying and setting up proper distribution keys.
  • Good knowledge on Netezza architecture like Zone maps, distribution keys etc
  • Created design approach to lift and shift the existing mappings to Netezza.
  • Created design documents to convert the existing mappings to use informaitca pushdown optimization.
  • Analyze the impact on the downstream systems and recommend the solutions to keep them intact.
  • Planning the Dev, SIT and QA environments.
  • Involved in designing the D/W using Star Schema. Identifying the Fact, Dimension and slowly changing dimension tables.
  • Taking ETL architecture decisions.
  • Created mappings, WorkFlows/Worklets and scheduled them using workflow manager and UNIX.
  • Create stored procedures in Netezza.
  • Converted the existing Oracle materialized and relational views into Netezza views.
  • Identify issues, debug and resolving issues.
  • Developed reusable frameworks for DB constraints and NZLoad.
  • Coordinate with Oracle DBA and Unix Admin to achieve better dB performance and identify performance bottlenecks.
  • NZ Administrator Tasks:
  • Creating and maintaining databases.
  • Create users, user groups and assigning permissions.
  • Implemented workload management on NZ.
  • Assisting development team with performance and tuning.
  • Use NZMigrate to load data from netezza server to the other server.
  • Extensively used PG.log to see the activity on Netezza.

Environment: Informatica PowerCenter 8.1, ETL, MicroStrategy, Netezza 4.x, Oracle 9i/10g, ER/Studio 3.5/4.0, Unix Shell Scripting, SQL, PL/SQL, TOAD, Transact-SQL, Windows NT 4.0.

ETL Architect/Lead

Confidential, Baltimore, MD

Responsibilities:

  • Involved in re designing and re writing the exiting War Room Reporting project.
  • Managed a team of ETL developers and Database Administrators.
  • Established the QA and DEV environments.
  • Worked with Siebel databases and Netezza.
  • Managed projects to move from BODI to Informatica.
  • Hands - On experience with both BODI and Informatica.
  • Installed and configured informatica 8.1 on Unix Servers.
  • Scheduling work for project team activities and gathering requirements from the business users.
  • Creates and maintains the overall and detailed project plan(s) and supervise the D/W ETL processes.
  • Involved in designing the D/W using Star Schema. Identifying the Fact, Dimension and slowly changing dimension tables.
  • Taking ETL architecture decisions.
  • Involved in designing the ETL processes and writing he design documents.
  • Created WorkFlows/Worklets and scheduled them using workflow manager and Unix.
  • Wrote numerous Pre and Post session SQL and Unix Scripts.
  • Coordinate with Oracle DBA and Unix Admin to achieve better dB performance and identify performance bottlenecks.

Environment: Informatica PowerCenter 8.1, ETL, Business Objects, Business Objects Data Integrator 11.5, Netezza 3.0, Oracle 8i/9i, ER/Studio 3.5/4.0, Unix Shell Scripting, SQL, PL/SQL, TOAD, Transact-SQL, Windows NT 4.0.

Technical/Team Lead

Confidential, Wilmington, DE

Major Responsibilities:

  • Involved in creating design documents, process flow diagrams and technical specs.
  • Managed a team of ETL developers and Database Administrators.
  • Established the ETL infrastructure, ETL/DW standards and ETL naming conventions.
  • Evaluating the latest versions and educating other developers with new changes.
  • Scheduling work for project team activities and gathering requirements from the business users.
  • Creates and maintains the overall and detailed project plan(s) and supervise the D/W ETL processes.
  • Involved in designing the D/W using Star Schema. Identifying the Fact, Dimension and slowly changing dimension tables.
  • Involved in the upgrade of information 5.x to 6.x
  • Taking ETL architecture decisions.
  • Involved in performing informatica Administrative tasks like creating login/passwords, taking backups, Registering repositories and restoring the repository.
  • Involved in designing the ETL processes and writing he design documents.
  • Created WorkFlows/Worklets and scheduled them using workflow manager and Unix.
  • Wrote numerous Pre and Post session SQL and Unix Scripts.
  • Coordinate with Oracle DBA and Unix Admin to achieve better dB performance and identify performance bottlenecks.
  • Worked on Power Mart/PowerCenter client tools like Source Analyzer, Warehouse Designer, Mapping Designer, Repository Manager, Mapplet Designer and Transformations Developer.

Environment: Informatica PowerCenter 5.x/6.2/7.x/8.1, ETL, Cognos Impromtu administrator, Cognos Powerplay Transformer, Cognos Powerplay, Oracle 8i/9i, ER/Studio 3.5/4.0, Unix Shell Scripting, SQL, PL/SQL, TOAD, Transact-SQL, Windows NT 4.0

ETL Tech Lead

Confidential, West Point, PA

Responsibilities:

  • Involved in designing Logical and physical databases for the staging and D/W using ERWIN.
  • Involved in design, analysis, implementation and support of ETL processes.
  • Prepared ETL standards, Naming conventions and wrote ETL flow documentation.
  • Responsible for the code drops and migrating processes into the production.
  • Involved in working with WorkFlow Manager, WorkFlow Monitor, Designer, Repository Manager, Source Analyzer, Warehouse Designer, Mapping Designer, Mapplet Designer and Transformations Developer.
  • Involved in working with Business objects Supervisor, Report designer and Universe.
  • Used Striva/Powerconnect to access DB2 tables and data.
  • Created WorkFlows/Worklets and scheduled them using workflow manager and Unix.
  • Created various Reusable and Non - Reusable tasks like Session, Assignment, Command, Control, Decision, Event and Timer.
  • Wrote numerous Pre and Post session SQL and Unix Commands.
  • Create workflows to partition data using Pipeline, Round-Robin, Hash, Key Range and Pass-through partitions.
  • Integrated third party tools like Postal Certification (FirstLogic) and Vality (Integrity) with Informatica to validate addresses, zip codes and to dedup the addresses.
  • Worked on PowerMart/PowerCenter client tools like Designed and developed various kind of maps using transformations like Expression, Aggregator, External Procedure, Stored Procedure, Look up, Filter, Joiner, Rank, Router, Update Strategy and XML.
  • Wrote store procedures, functions, database triggers and Unix shell scripts to support and Automate the ETL process.
  • Developing new Business Objects Universes and deploying to users worldwide.
  • Identifying the objects and classes need to be created in the universe.
  • Involved in describing, creating, building and maintaining the universe.
  • Involved in resolving loops, fan traps and chasm traps by creating Aliases and contexts.
  • Identifying and creating joins and cardinalities between tables.
  • Developing new Business Objects Universes to users worldwide.
  • Involved in performance and tuning the SQL queries using Explain plan.

Environment: Informatica Powermart 5.1/6.2, Informatica PowerCenter 5.x/6.2, ETL, Business Objects 5.1, Oracle 8i/9i, ER/Studio 3.5/4.0, Unix Shell Scripting, SQL, PL/SQL, TOAD, Transact-SQL, Windows NT 4.0.

Data Warehouse Consultant

Confidential

Major Responsibilities:

  • Involved in Installation and Configuration of Informatica PowerCenter, PowerMart, Informatica Client, Informatica Server.
  • Used Striva/Powerconnect to access DB2 tables and data.
  • Installed informatica server on both windows and Unix platforms.
  • Upgrading Informatica Power Center 1.7 to 5.0/5.1.
  • Trouble shooted connectivity problems. Looked up and read session, event and error logs for troubleshooting.
  • Taking Back ups and Restoring the Informatica Repository.
  • Worked on PowerMart/PowerCenter client tools like Source Analyzer, Warehouse Designer, Mapping Designer, Mapplet Designer and Transformations Developer.
  • Creating and managing Informatica Local and Global Repository.
  • Identifying and loading sources and targets metadata into repository.
  • Created, launched & scheduled sessions. Configured email notification.
  • Setting up Batches and sessions to schedule the loads at required frequency using Power Center Server manager. Generated completion messages and status reports using Server manager.
  • Extensively worked in the Performance Tuning of the programs, ETL Procedures and processes.
  • Creating User Groups, Users, folders and assigning permissions.
  • Extensively used ETL to load data from Oracle 8i, MS SQL Server 7.0 and flat files to Oracle 8i.
  • Used most of the transformations such as the Source qualifier, Aggregators, lookups, Filters, Sequence and Update strategy.
  • Extensively involved in UNIX shell scripting.
  • Created tables, Functions, Packages, Triggers, User defined data types.
  • Wrote PL/SQL Packages, Stored procedures, created triggers to implement business rules and validations.
  • Developing new Business Objects Universes and deploying to users worldwide.
  • Identifying Fact and Dimension tables to build indexes to improve performance.
  • Performance monitoring and tuning of the queries.
  • Created Universes for report generation using Designer module. As a part of Universe development, created Classes, different kind of objects, cardinalities.
  • Resolved the loops using Aliases and contexts.
  • Created Adhoc Reports using Business Objects. As a part of report development created the reports using universes as a main data providers and using the Powerful business objects functionality's like @ functions, user response and formulae's.
  • Export the Universe and documents to the Repository.

Environment: Informatica Powermart 4.7/5.0, Informatica PowerCenter 1.7/5.x, ETL, Business Objects, Oracle 8i, Sybase 12.0, SQL Server 7.0, MS SQL Server OLAP Services, ER/Studio 3.5/4.0, Unix Shell Scripting, SQL, PL/SQL, TOAD, Transact-SQL, Crystal Reports 7/8.5, Windows NT 4.0.

We'd love your feedback!