We provide IT Staff Augmentation Services!

Big Data Administrator/architect Resume

3.00/5 (Submit Your Rating)

Mooresville, NC

SUMMARY:

  • Total of 17 years’ working experience on very large database and analytic systems.
  • 5 years as Sr. Hadoop administrator/architect responsible for HortonWorks and Cloudera POCs, architecting teh Production Hadoop setup and delivering it to Production. Supported and maintained multiple Production large Hadoop systems.
  • 12 years as Sr. Teradata DBA administrating multiple large Enterprise Data Warehousing systems.
  • Around 2 years experience as Manager for a team of DBAs and Administrators of Big Data Analytical Platforms dat comprises of Hadoop, Asterdata, Teradata, SAS Grid, Alteryx, Verint Video Analytics Platform.
  • 5 years as Sr. Asterdata DBA responsible for teh Aster POC to teh Production setup.
  • 5 years working in Teradata R&D in Teradata database optimizer area fixing issues and enhancing teh code. Also supported onsite upgrades at customer sites like Wal - Mart, eBay.

PROFESSIONAL EXPERIENCE:

Confidential

Big Data Administrator/Architect

Responsibilities:

  • Technical Lead for teh Project.
  • Review Hive Tables and views to make it consumable for MicroStrategy.
  • Review Hive, Tez, YARN settings to perform concurrency testing.
  • Generate Test Plan for teh Concurrency Testing.
  • Execute teh Test Plan.
  • Gather Runtime metrics using MapR tools like Grafanna & Kibana.
  • Evaluate Performance against Teradata.

Confidential

Big Data Administrator/Architect

Responsibilities:

  • Recommended four tracks.
  • Compression
  • Apply Multi-Value Compression on teh Teradata tables so teh table size is reduced and large scan queries reads less bytes.
  • Partitioning
  • Apply Single level and Multilevel Partitioning to reduce teh number of rows scanned during query processing
  • Workload Management
  • Setup different workloads in Teradata to match teh MicroStrategy Workload Schedule.
  • Apply Penalty box settings so a bad skewed query doesn’t degrade teh system performance.
  • Identify very short running queries and allow them to bypass so they don’t sit in delay queue.
  • Statistics
  • Identify unused statistics and remove them.
  • Identify used statistics and check for length and collect statistics for teh entire length of teh column,
  • Identify missing statistics and collect them to improve performance.

Confidential

Big Data Administrator/Architect

Responsibilities:

  • Technical Lead for teh Project.
  • Reviewed teh HDFS layouts for Stage0, Stage1, Stage2 of teh file ingestion and recommended best practices.
  • Reviewed Hive Schema for teh HDFS file system layout and recommended best practices.
  • Installed Teradata Connector for Hadoop (TDCH) and transfer data from Teradata to MapR Hadoop.
  • Designed Drill Views against HDFS data for teh Lift and Shift of objects from Teradata to MapR.
  • Baseline MicroStrategy reports on Teradata and benchmark teh same MicroStrategy reports against Hadoop.
  • Performed Concurrency tests with MicroStrategy Reports.
  • Performed Mixed workload Testing with Hive Batch process and Drill SQLs running from MSI.
  • Collected System level performance metrics using Grafanna.
  • Provided Recommendation to Management on use of Hadoop for EDW ETL offload and MicroStrategy Reporting.

Confidential

IT BI & Analytics Manager - Platform Administration

Responsibilities:

  • Teradata BAR with DSA - Led teh PoC and Production Implementation
  • Led Teradata Software Upgrades (13.10, 14.10) and Hardware upgrades (Floor Sweeps & Node Additions). Enabled new features.
  • Unity Director Setup, Query Director Maintenance
  • SLES11 Upgrade - Workload Management (TASM) Tuning
  • Enabled Teradata Intelligent Memory (TIM), Teradata Columnar.
  • 2.3 Hadoop Software Upgrade.
  • Teradata workload offload to Hadoop - Historical Sales, Teradata Database logging (PDCR)
  • Node additions to Hadoop cluster - Master and Data Nodes.
  • Data Encryption at Rest - Ranger KMS
  • Solr Installation and setup.
  • Amabri 2.2.2 upgrade enabling Ambari Views, Zeppelin, Grafana.
  • 6.0 and 6.20 Upgrades, Aster Client Upgrades
  • Swing system setup for DR.
  • Query Grid Implementation.
  • SAS Software Upgrades
  • SAS Production Setup within Discovery Cluster.
  • Central Server setup and administration dat captures people count data from teh Video Camera in teh Stores.
  • Alteryx Server Administration and Alteryx Desktop Deployment.

Confidential, Mooresville, NC

Hadoop Developer

Responsibilities:

  • Installation through Ambari for Hortonworks and Cloudera Manager for Cloudera.
  • Responsible for providing scoring on teh infrastructure side and provide teh main recommendation of selecting teh vendor.
  • Production Architecture Design - Hardware configuration, dual Production and development setup.
  • Installation of HDFS, YARN, Hive, MapReduce2, Sqoop, Oozie, Nagios etc.
  • Security setup through Ranger and Knox.
  • PoC with Falcon. Designing Hadoop Ingestion through Falcon workflow.
  • Scheduling through Oozie.
  • Replication through discp.
  • SNMP trap setup in Nagios.
  • Setting up connectivity between Teradata and Hadoop through teh Teradata connectors.
  • HUE installation and userid provisioning.
  • Ranger, Ranger KMS (Data Encryption at Rest) & Knox policy setup.
  • Integration of teh Aster environment with Production Teradata through teh Infiniband infrastructure
  • LDAP integration
  • Teradata-Asterdata connector
  • Databases, schemas, User id & Roles setup
  • Loading data from other data marts into Aster through teh connectors
  • Purge process for teh database logs and audit reports
  • AMC (Aster Management Console) setup
  • Aster client push to user desktops
  • SQL-H installation.
  • Led teh upgrade from 5.10 to 6.0 version
  • Aster app center setup.

Environment: Hortonworks Data Platform (HDP2.0, 2.1, 2.2), HDFS, YARN, MapReduce2, Hive, Sqoop, Falcon, PIG, DisCp Replication, Nagios, HUE, Oozie.

Confidential, Mooresville, NC

Sr. Teradata DBA

Responsibilities:

  • Led Software and Hardware Upgrade Projects.
  • Create and maintain teh physical database structures required to effectively support application software.
  • Provide technical assistance to IT Solution Delivery Project.
  • Monitor RDBMS performance for assigned database platforms and operating environments.
  • Provide Database Object and application tuning.
  • Work with RDBMS utilities to provide services such as table load/unload, reorganizations, backup/recovery, etc.
  • Works with application developers to execute performance benchmarks of production SQL.
  • Assist in teh enforcement of standards for all databases.
  • Assist in teh research, selection, and implementation of RDBMS specific utilities.
  • Assist in teh evaluation, implementation and support of application software packages and their requisite RDBMS platform.
  • Work closely with IT Vendor - Teradata to lead internal project and on support.
  • Develop productivity aids using appropriate batch, scripting, or programming languages.
  • Develops/Implement/Maintain Database utilities to support RDBMS implementations.
  • Provide On-Call Support.
  • Participated in teh review of teh SSD of Encrypting Sensitive Customer Data. Recommended new solutions for maintaining different keys for different customer data. Supported and resolved various issues during development.
  • Primary DBA for DCM application support. Resolved critical issues by restoring databases and resolving archive issues on DCM database.
  • Security administration - Developed scripts to identify weak passwords.
  • Security administration - Developed process to identify and notify multiple logon attempts.
  • Workload Management - Tuned TASM settings to has highest throughput for DART workload
  • V2R6.2 Software and TTU 12.0 Upgrade Project - Successfully upgraded both WS and SA Production to V2R6.2. Provided support for successful upgrade of TTU to 12.0 and supported in resolving compiler issues for COBOL programs.
  • Security administration - Successful implementation of new password controls on Production Teradata.
  • Release Management - Endevor analysis - Identification of dynamic PARMLIB creation - obsolete members - New naming convention - Participation in review meetings
  • Performance and Tuning - Analysis of long running batch jobs - Recommendation to change teh schedule of collect stats jobs - this halped many batch jobs not running into business hours.
  • Developed process for tracking and efficiently manage all Teradata space requests in a spreadsheet. Using VBA macros in excel, developed code to retrieve actual space request from production and provide any overestimation or underestimation
  • Developed process for tracking Object changes which would halp in audit team to track teh changes without DBA intervention
  • Developed new method of doing MVC compression in 14.10 using Statistics.
  • Analysis of long running batch jobs - Recommendation to change teh schedule of collect stats jobs - this halped many batch jobs not running into business hours and saved lot of CPU cycles
  • Major contribution for teh PEN test (PENetration Test). Developed process to identify weak passwords and resolve them, put new job to has security control on test box.
  • Implementation of new password controls on all Teradata Systems. Optimizing teh run times of long running batch jobs.
  • Lead DBA team meetings to resolve issues from DBA abends and other DBA related enhancements.
  • Successfully lead Teradata 6700 Hardware floor sweep Project identifying tasks, risks and coordinating with other teams like Data Center Operation personnel, Network team, solutions team to complete those tasks and mitigate teh risks. Benchmarked teh inclusion of Solid State Drivers (SSD) in teh 6700 Storage with Teradata Virtual Storage (TVS) enabled.
  • Support Viewpoint, Teradata Multi Systems Manager (TMSM) software upgrades.
  • Support Teradata Unity Director Implementation to replace Teradata Query Director (TQD).
  • Responsible for Integration of Informatica Power Center with Teradata using Teradata Parallel Transporter (TPT) Connectors. Setup Standards and guidelines for ETL design.
  • Responsible for generating Teradata Roadmap in Lowe’s. Coordinate with Architects and vendors to come up with teh next generation (BI) Business Intelligence Ecosystem.
  • Generation of Teradata Usage Reports for capacity planning to identify business analytics growth which feeds into teh Roadmap Strategy.
  • Develop Teradata Usage Charts for higher management for both Business and IT.
  • Provide guidance and support for using Protegrity on other platforms like SQLServer.
  • Participate in Disaster Recovery (DR) Exercise for Teradata.
  • Review and evaluate next generation BAR (Backup Archive & Restore) solution dat reduces cost and storage foot print. Also examined current back up process and generated reports supporting it.
  • Participate in internal and external SOX and PCI Audit reviews. Provide Access Control and Change control reports with evidence documents supporting teh change.
  • Coordinate with development team to setup websites dat simplify access request process. Integration with ASP (Active Server Pages) and .Net Participate in design sessions and implement database objects supporting teh website,

Environment: Teradata Administration, Teradata SQL, Teradata Physical Implementation, Teradata V2R4, V2R5, V2R6 releases, Teradata Client Tools and Utilities (Teradata Administrator, Teradata SQL Assistant, Teradata Manager, TSET, BTEQ, FastLoad, Multiload, FastExport, Arcmain), MVS JCL, NCR UNIX (MP-RAS), Zeke, Endevor, Informatica, DataStage, Viewpoint, TMSM, Unity, Teradata Query Director, TPT, Teradata 6700.

Confidential, Minneapolis, MN

Teradata Database Administrator

Responsibilities:

  • Design, creating and tuning physical database objects (tables, views, indexes) to support normalized and dimensional models.
  • Participate in all essential day-to-day database support activities.
  • Supporting and tuning for queries from Business Objects reporting tool and BO XI Release 2 migration.
  • Application Development for Data Marts. Involved in development of load processes on MVS and NT for teh new and existing data marts. Development of applications using OLE-DB Access Module and K-Shell scripting to load and unload data between Teradata and other databases like DB2, SQLServer, Sybase and Oracle to support applications on teh respective databases.
  • Ongoing 24x7 support of teh production environment.
  • Integrating QlikView reporting tool with Teradata for Sales and Reporting Analysis (SARA). Benchmarking and configuring various Teradata client tools and interfaces with QlikView.
  • Writing JCL scripts on MVS for streamlining various process.
  • Administration and monitoring of teh database using Teradata manager and Teradata Administrator (WinDDI).
  • Creating incidents to NCR GSC for issues in Teradata optimizer and monitoring for problem resolution.
  • Coach developers and application teams on best practices from a performance and standards perspective.

Environment: MVS, JCL, FILEAID, ISPF, WSF2, SORT, Queryman, BTEQ, Fastload, Multiload, FastExport, TPump, Teradata SQL.

Confidential, Los Angeles, CA

System Analyst

Responsibilities:

  • This project includes support for teh various features of Teradata V2R6 database. Problems reported by Teradata Customer are resolved and Emergency fix (EFIX) is given to teh customer. Main V2R6 features includes V2R6 Random AMP Sampling, OCES (Optimizer Cost Estimation Subsystem
  • Recursive SQL queries, Improved IN-list handling, PPI Join Enhancement (Dynamic Partition Elimination), Top N (First N) Rows Feature, TDSP Enhancements, Secondary Access Improvements for PPI tables
  • Teradata Dynamic Workload Management (TDWM), V2R6 DBQL(Database Query logging) features, V2R6 Queue Tables, Priority Scheduler features, LOBs in stored procedures, V2R6 Table function, V2R6 External Stored Procedure.
  • Teradata V2R5.1 support includes support of teh entire Teradata DBS feature in dat release. Main database related features dat are supported Inner Join Conversion, allow extra FK-PK joins in join index, Enable hash join by default, Eliminate unnecessary outer joins, TDSP Initiatives, BLOB and CLOB, PPI Dynamic Partition Elimination (DPE), ANSI Database triggers, Extended Grouping, Identity column support for MultiLoad/FastLoad, ROLE ALL feature, DBQL CPU & me/O, UDFs (User Defined Functions).
  • V2R5 is one of teh major releases in Teradata History. Many features went into this release. Most of teh features were supported in this project.
  • Main features dat were supported are Column Limits, Identity columns, Value List Compression, Parser/Dispatcher Combo, Atomic UPSERT, Reduce all-AMP operation, Roles and Profiles, User Level Security Control, I18N formatting for Date, Time, Timestamp, Numeric and Monetary, RSS and ResUsage Enhancements, V2R5 TDSP Features, DBQAT(DataBase Query Analysis Tools), Sparse Index, Non-covering Join Index, Join Index Maintenance Improvement
  • Partial Group by, Block optimization derived tables, SATTC, Batch RI (Referential Integrity), Join Elimination, Soft RI, Semantic Query Optimization (SQO), Avoid Spooling UNION ALL queries, Partitioned Primary Index, Explain estimate display, DiskIO Cost Coefficients, Optimizer coefficients updates, Merge Update, Row Access locking, Multiple DISTINCT aggregate, Enhanced Sampling, SQL -99 Window Functions, DBQL (Database Query Logging), Monitor Session and SQL Enhancements
  • Teradata Dynamic Query Manager (TDQM), V2R5 Migrate and Upgrade, Merge-Into Feature, DDL and statement Text Limits Extensions.
  • This project includes fixing bugs in Teradata Optimizer, providing workarounds for many hot customer issues for all features of Teradata V2R4.1. Main features include Stored Procedure enhancements incl. DDL, CASE, Warning Handlers, etc.
  • Support UTF-8 UNICODE char sets for sessions, Atomic UPSERT statement, More triggers are single AMP operations for increased scalability, Query Capture Database enhancements for VEComp, 128k block size, Optimizer enhancements, Increase max global temp tables from 32 to 1000 and volatile temp tables from 64 to 2000, Remaining SQL-99 complex aggregate statistical functions, PERCENT RANK, Hash Index feature, Partial Covering join indexes, Single table join indexes, Single table join index using compressed join index row format
  • Support for all optimizer related features. Main features dat were supported in this release include Default Date Format, TPC-D: Aggregate Join Index, TPC-D: Improve Delete Performance, TPC-D: Convert Connecting Conditions for Wider JI Usage
  • TPC-D: Recognize OR conditions in JI syntax, TPC-D: Allow EXIST and NOT EXIST sub queries to be combined, TPC-D: Replace any single table with applicable join index, TPC-H: Add Implicit Primary Index to single-column, single-table join index, TPC-H: Combine Having with Sum Step, OPT: LIKE use NUSI, OPT: Improve NUSI Read Performance, Target Level Emulation, RFC: Add SHOW DML statement, Teradata Stored Procedures, Create Table AS, SQL: Aggregate and Stat Functions, TPCD: Remove unique sort on TPC-D Query 21 sub query step, SQL
  • Random Function, Amendment to OLAP function to support ANSI, Fallback for JI, Eliminate Steps from Derived Tables, Query Explain Capture Tool, OLAP GSUM and GCOUNT, RFC: TPC-D: Allow joining from a NUSI to teh base table, Covering NUSI, Consistent COLLECT STATISTICS/CREATE INDEX, New SQL System Variables, Combine Having with Sum stepV2R3 Optimizer Support:
  • Main features dat were supported in this release include Temporary Tables, Place a Timestamp in DD Tables, ANSI Date / Time, Triggers, Rename Column Names
  • Index Sensitive Locking, ANSI string, password expiration, insert default values, Enhance Statistics, OLAP sampling, MLOAD Performance Enhancements, DDL Locking Improvements, Update Performance, Aggregate Performance Enhancements, OLAP Statistical, OLAP Calendar, I18N, increase View / Macro limits.

Environment: UNIX, W2K, C, C++, PASCAL, gdb, Visual C++ 6.0 debugger, Visual Studio .NET Debugger, Teradata SQL, BTEQ, MLoad, FastLoad, FastExport, TPump, TSET, Statistics Wizard, Index Wizard, Visual Explain, other Teradata Utilities and tools, Rational Clearcase (UNIX & W2K).

We'd love your feedback!