Sr. Datastage/bigdata Consultant Resume
Rochester, MN
SUMMARY
- 13 years of total IT experience
- 12 years of ETL, Database and Unix/Linux experience
- 3 years of Bigdata and Cloud Experience
TECHNICAL SKILLS
- ETL: Confidential IIS Datastage 11.5/9.x/8.x (Administrator, Quality, Designer, Director), Cloud Dataflow, Cloud Dataproc, Cloud Dataprep (Trifacta), SSIS, Talend, Pentaho, Alteryx, Informatica Power Center 10.x/9.x, Spark, Cloudera Hadoop
- BI, Data Modeling & Scheduling: Business Objects 4.2/4.0, SSRS, Tableu, Control - M, Autosys, Cron, ERwin 3.5.2/3.x, RDA 7.5, Nifi, RPA
- Databases: Google Cloud SQL, Google Cloud Bigtable, Google Cloud Datastore, Google Cloud Spanner, Apache Hbase, MongoDB, Cassandra, DB2 UDB, DB2 11.1/10.x, Netezza, Teradata, Amazon S3, Oracle 11g/10g/9i, MS SQL Server 2016, Sybase 12.0/11.x.
- Analytics: Snowflake, Google CloudSQL, Hive, Pig
- GUI: Rapid SQL 8.7.1, Oracle Developer Suite, Developer 2000, Forms 6i, Reports 6i, Visual Basic 6.0/5.0/4.0/3.0
- Environment: Python 3.6/3.5, Bash, Suse Linux, Confidential AIX 5.3/4.3/4.2, Confidential System-Z 10 Linux, MS-DOS 6.22, Windows
- Integration: Kafka, Alooma, Mulesoft, Dell Boomi
- Data Fomats: CSV,Text, XML,JSON, Avro
- Team Management: Atlassian Jira, Microsoft TFS, ServiceNow ITBM
- Version Control: Git, GitHub, CVS, PVCS, Confidential Rational Clearcase
PROFESSIONAL EXPERIENCE
Confidential, Rochester, MN
Sr. Datastage/Bigdata Consultant
Responsibilities:
- Used the Datastage Designer to develop processes for extracting, cleansing, transforming and loading data into the Database.
- Used different stages of Datastage Designer like Lookup, Join, Merge, Funnel, Filter, Copy, Aggregator, Sort, Column Generator, Remove Duplicates, Modify, Transformer, Sequential files, DS files, Confidential, Execute command, Nested loop, Email Notification, User Variables Activity, Sequencer, Exception Handler, Terminator Activity, Job Activity and Wait for file activity
- Developed several jobs to improve performance by reducing runtime using different partitioning techniques.
- Used Datastage Director for validating, execution, monitoring jobs and check the log files for errors.
- Used stage variables for source validations, to capture rejects and used Job Parameters for Automation of jobs.
- Created KSH scripts to perform validations and run jobs on different instances.
- Used Shell scripts to do basic file operations like moving files, creating directories, purging.
- Created Parameter sets and used them in the jobs and imported them into different environments for later use.
- Responsible for transforming the data from multiple sources like MYSQL, Oracle, Flat files as per the business requirements.
- Hands on experience in Essentials of DataStage components and Oracle PL/SQL programming.
- Used Datastage Designer for importing metadata into repository, for importing and exporting jobs into different projects.
- Used Rapid SQL to write up complex SQL Logic for some operations that cannot be performed in Datastage.
Environment: Confidential IIS DataStage 11.5/9.x Enterprise Edition, Informatica Power Center 9.5.1, Apache Hbase, Hortonworks, Google BigQuery, Dataproc, Dataflow, Dataprep (Trifacta), Python 3.6/3.5/2.x, Microsoft TFS, SharePoint, Oracle 11g/10g, DB2 UDB, SQL server 2016, SSIS, SSRS, Rapid Sql,Quest, IWS/TWS Scheduler, UNIX, Business Objects, Service Now.
Confidential, Chapel Hill, NC
Applications Analyst
Responsibilities:
- Analysis and design of ETL processes.
- Understood the technical specifications and developed Datastage Parallel jobs for Extraction Transformation and Loading process of DW.
- Conducting ETL team meetings and explaining the requirements to team members.
- Walking through Job designs, Code by team members, correcting if necessary.
- Designing job templates to specify high-level framework approach.
- Developing standard ETL process will execute pre-ETL and post-ETL processes to ensure smooth transfer of data from heterogeneous source systems to a homogenous Confidential system.
- Worked in the implementation of DW Incremental (pull and load) that constitutes different subject areas like (Account, Confidential, Bed, Allergy, Account Payer Billing, Diagnosis, Immunization) for different Source Systems.
- Involved in developing jobs for Source to Stage, Stage to ADS (Atomic Data Store) and ADS to Datamarts.
- Designed Job Sequencers for every project to run in a loop based on the success or failure of individual jobs in the Sequencer.
- Developed batches and sequencers in designer to run and control set of jobs
Environment: Confidential Websphere Datastage 8.1, Datastage 8.X (Datastage, Quality Stage, Information Analyser, Business Glossary), Oracle 9i,SQL Server 2005, DB2 8.1 Z-OS, DB2 9.1.5 for Z-OS, Confidential System-Z 10 Linux, AIX 5.3, Rational Data Architect, ETI, Windows NT, Clear Case Clear Quest, SQL, UNIX Shell Scripting, MS Visio.
Confidential, MN
Datastage Consultant
Responsibilities:
- Create Functional and Technical specification design documents.
- Experience working in large DWH environments.
- Designed & Developed Star Schema database and mappings between sources and operational staging targets.
- Queried data from different database tables as per the requirement, and populated data to Data Warehouse tables.
- Created source table definitions in the Repository by studying the data sources.
- Created Log Tables containing data with discrepancies to analyze and re-process the data.
Environment: Confidential Websphere DataStage 8.0.1 (MVS Edition), Oracle 9i, DB2/AIX64 9.5.5, Erwin 4.0, PL/SQL, Mercury Quality Center, Toad for oracle and DB2, MS Visio, AutoSys, SVN Tortoise 1.5.0
Confidential Minneapolis, MN
Senior Datastage Consultant
Responsibilities:
- Analysis and design of ETL processes.
- Understood the technical specifications and developed Datastage Parallel jobs for Extraction Transformation and Loading process.
- Conducting ETL team meetings and explaining the requirements to team members.
- Involved in evaluating the scope of application, defining relationship within and between groups of data
- Interacted with Management to identify key dimensions and measures for business performance.
- Responsible for design of the Star schema and business rules required to populate the fact and dimension tables
- Developed jobs in PX for splitting the data into subsets and flowing of data concurrently across all available processors to achieve job performance.
Environment: Confidential Ascential Datastage 7.3, SQL Server 2005, Oracle 10 g, Confidential DB2 9.0, AIX 5.3, Rational Data Architect, ETI, Windows NT, SharePoint, SQL, UNIX Shell Scripting, MS Visio.
Confidential, Chapel Hill, NC
Senior Datastage Consultant
Responsibilities:
- Analysis and design of ETL processes.
- Understood the technical specifications and developed Datastage Parallel jobs for Extraction Transformation and Loading process of DW.
- Conducting ETL team meetings and explaining the requirements to team members.
- Walking through Job designs, Code by team members, correcting if necessary.
- Designing job templates to specify high-level framework approach.
- Developing standard ETL process will execute pre-ETL and post-ETL processes to ensure smooth transfer of data from heterogeneous source systems to a homogenous Confidential system.
- Worked in the implementation of DW Incremental (pull and load) that constitutes different subject areas like (Account, Confidential, Bed, Allergy, Account Payer Billing, Diagnosis, Immunization) for different Source Systems.
- Involved in developing jobs for Source to Stage, Stage to ADS (Atomic Data Store) and ADS to Datamarts.
Environment: Confidential Websphere Datastage 8.0.1, Datastage 8.X (Datastage, Quality Stage, Information Analyser, Business Glossary), SQL Server 2005, DB2 8.1 Z-OS, DB2 9.1.5 for Z-OS, Confidential System-Z 10 Linux, AIX 5.3, Rational Data Architect, ETI, Windows NT, Clear Case Clear Quest, SQL, UNIX Shell Scripting, MS Visio.
Confidential, Omaha, NE
Datastage Consultant
Responsibilities:
- Analysis and design of ETL processes.
- Understood the technical specifications and developed Datastage server jobs for Extraction Transformation and Loading process of DW.
- Worked in the implementation of DW Incrementals (pull and load) that constitutes different subject areas like (Payroll, Financial Transaction, Compensation Request, Agreement, Party, Contact Point) for different source systems.
- Interacted with business analysts and modelers for better understanding of individual subject areas and modified specifications to reflect accurate user needs
- Worked as a Data Warehousing Analyst administering various databases such as MS SQL Server, DB2 etc
- Mapped the source and Confidential databases by studying the specifications and analyzing the required transforms.
- Used CoSort utility for sorting, joining, and aggregating massive files, speeding data warehouse operations, database reorgs, ranking, searching, and matching.
Environment: Confidential Websphere / Ascential Datastage SE, Datastage 7.5.1 (Ascential Datastage 7.X (Designer, Manager, Administrator, Director, MetaBroker, MetaRecon, Profile Stage, Metastage, Parallel Extender),Webfocus 7.X,CoSort, Clear Case & Clear Quest, ETI, SQL, DB2/UDB 8.1.5, MS Visio, Erwin 4.1, Windows NT, UNIX (AIX 4.3), SQL, Cybermation 2.0.0.0, UNIX Shell Scripting.
Confidential, MN
Datastage Consultant
Responsibilities:
- Thorough analysis of the system to provide inputs for effort estimation and planning of the project.
- Analysis at interface level to understand the different linkage partners and gather their requirements.
- Created the existing and the post implementation flow diagrams for each linkage partner using Microsoft Visio.
- ETL development (mainframe DB2 & IMS) to UDB using Datastage server in a shared container in a Datastage PX job. The shared container is used for the FTP and translation steps, as well as running a stored procedure.
- Identified the impacted jobs and scripts and created a high level design document which would meet all the requirements and would have the best minimum effort required.
- Prepared the low level design documents with the data flow diagrams reflecting the original Parallel extender job flow.
- Developed jobs with High and medium complexity which involved different extraction transformation methodologies which would give the best performance for huge volumes of data.
Environment: Confidential Websphere / Ascential Datastage EE, Datastage 7.5.1 (Ascential Datastage XE/ EE / 7.X (Designer, Manager, Administrator, Director, MetaBroker, MetaRecon, Quality Stage, Profile Stage, Metastage, Parallel Extender), NCR Teradata V2R4, Mainframes, ETL, SQL, PL/SQL, Business Objects 6.5 (Supervisor, Designer, BO reporter, BCA), DB2/UDB 8.1.5, MS Visio, Erwin 4.1, Windows NT, UNIX (AIX 4.3), SQL, PL/SQL, UNIX Shell Scripting