Big Data Administrator/architect Resume
Los Angeles, CA
SUMMARY:
- Total of 17 years’ working experience on very large database and analytic systems.
- 5 years as Sr. Hadoop administrator/architect responsible for HortonWorks and Cloudera POCs, architecting the Production Hadoop setup and delivering it to Production. Supported and maintained multiple Production large Hadoop systems.
- 12 years as Sr. Teradata DBA administrating multiple large Enterprise Data Warehousing systems.
- Around 2 years experience as Manager for a team of DBAs and Administrators of Big Data Analytical Platforms that comprises of Hadoop, Asterdata, Teradata, SAS Grid, Alteryx, Verint Video Analytics Platform.
- 5 years as Sr. Asterdata DBA responsible for the Aster POC to the Production setup.
- 5 years working in Teradata R&D in Teradata database optimizer area fixing issues and enhancing the code. Also supported onsite upgrades at customer sites like Wal - Mart, eBay.
PROFESSIONAL EXPERIENCE:
Confidential, Los Angeles, CA
Big Data Administrator/Architect
Responsibilities:
- Review Hive Tables and views to make it consumable for MicroStrategy.
- Review Hive, Tez, YARN settings to perform concurrency testing.
- Generate Test Plan for the Concurrency Testing.
- Execute the Test Plan.
- Gather Runtime metrics using MapR tools like Grafanna & Kibana.
- Evaluate Performance against Teradata.
Confidential
Big Data Administrator/Architect
Responsibilities:
- Recommended four tracks.
- Compression
- Apply Multi-Value Compression on the Teradata tables so the table size is reduced and large scan queries reads less bytes.
- Partitioning
- Apply Single level and Multilevel Partitioning to reduce the number of rows scanned during query processing
- Workload Management
- Setup different workloads in Teradata to match the MicroStrategy Workload Schedule.
- Apply Penalty box settings so a bad skewed query doesn’t degrade the system performance.
- Identify very short running queries and allow them to bypass so they don’t sit in delay queue.
- Statistics
- Identify unused statistics and remove them.
- Identify used statistics and check for length and collect statistics for the entire length of the column,
- Identify missing statistics and collect them to improve performance.
Confidential
Big Data Administrator/Architect
Responsibilities:
- Reviewed the HDFS layouts for Stage0, Stage1, Stage2 of the file ingestion and recommended best practices.
- Reviewed Hive Schema for the HDFS file system layout and recommended best practices.
- Installed Teradata Connector for Hadoop (TDCH) and transfer data from Teradata to MapR Hadoop.
- Designed Drill Views against HDFS data for the Lift and Shift of objects from Teradata to MapR.
- Baseline MicroStrategy reports on Teradata and benchmark the same MicroStrategy reports against Hadoop.
- Performed Concurrency tests with MicroStrategy Reports.
- Performed Mixed workload Testing with Hive Batch process and Drill SQLs running from MSI.
- Collected System level performance metrics using Grafanna.
- Provided Recommendation to Management on use of Hadoop for EDW ETL offload and MicroStrategy Reporting.
Confidential, Mooresville, NC
IT BI & Analytics Manager
Responsibilities:
- Led the PoC and Production Implementation
- Led Teradata Software Upgrades (13.10, 14.10) and Hardware upgrades (Floor Sweeps & Node Additions). Enabled new features.
- Unity Director Setup, Query Director Maintenance
- SLES11 Upgrade - Workload Management (TASM) Tuning
- Enabled Teradata Intelligent Memory (TIM), Teradata Columnar.
- 2.3 Hadoop Software Upgrade.
- Teradata workload offload to Hadoop - Historical Sales, Teradata Database logging (PDCR)
- Node additions to Hadoop cluster - Master and Data Nodes.
- Data Encryption at Rest - Ranger KMS
- Solr Installation and setup.
- Amabri 2.2.2 upgrade enabling Ambari Views, Zeppelin, Grafana.
- 6.0 and 6.20 Upgrades, Aster Client Upgrades
- Swing system setup for DR.
- Query Grid Implementation.
- SAS Software Upgrades
- SAS Production Setup within Discovery Cluster.
- Central Server setup and administration that captures people count data from the Video Camera in the Stores.
- Alteryx Server Administration and Alteryx Desktop Deployment.
Confidential, Mooresville, NC
Teradata, Hadoop &Asterdata Administrator
Responsibilities:
- Installation through Ambari for Hortonworks and Cloudera Manager for Cloudera.
- Responsible for providing scoring on the infrastructure side and provide the main recommendation of selecting the vendor.
- Production Architecture Design - Hardware configuration, dual Production and development setup.
- Installation of HDFS, YARN, Hive, MapReduce2, Sqoop, Oozie, Nagios etc.
- Security setup through Ranger and Knox.
- PoC with Falcon. Designing Hadoop Ingestion through Falcon workflow.
- Scheduling through Oozie.
- Replication through discp.
- SNMP trap setup in Nagios.
- Setting up connectivity between Teradata and Hadoop through the Teradata connectors.
- HUE installation and userid provisioning.
- Ranger, Ranger KMS (Data Encryption at Rest) & Knox policy setup.
- Integration of the Aster environment with Production Teradata through the Infiniband infrastructure
- LDAP integration
- Teradata-Asterdata connector
- Databases, schemas, User id & Roles setup
- Loading data from other data marts into Aster through the connectors
- Purge process for the database logs and audit reports
- AMC (Aster Management Console) setup
- Aster client push to user desktops
- SQL-H installation.
- Led the upgrade from 5.10 to 6.0 version
- Aster app center setup.
Environment: Hortonworks Data Platform (HDP2.0, 2.1, 2.2), HDFS, YARN, MapReduce2, Hive, Sqoop, Falcon, PIG, DisCp Replication, Nagios, HUE, Oozie.
Confidential, Mooresville, NC
Sr. Teradata DBA
Responsibilities:
- Led Software and Hardware Upgrade Projects.
- Create and maintain the physical database structures required to effectively support application software.
- Provide technical assistance to IT Solution Delivery Project.
- Monitor RDBMS performance for assigned database platforms and operating environments.
- Provide Database Object and application tuning.
- Work with RDBMS utilities to provide services such as table load/unload, reorganizations, backup/recovery, etc.
- Works with application developers to execute performance benchmarks of production SQL.
- Assist in the enforcement of standards for all databases.
- Assist in the research, selection, and implementation of RDBMS specific utilities.
- Assist in the evaluation, implementation and support of application software packages and their requisite RDBMS platform.
- Work closely with IT Vendor - Teradata to lead internal project and on support.
- Develop productivity aids using appropriate batch, scripting, or programming languages.
- Develops/Implement/Maintain Database utilities to support RDBMS implementations.
- Provide On-Call Support.
- Participated in the review of the SSD of Encrypting Sensitive Customer Data. Recommended new solutions for maintaining different keys for different customer data. Supported and resolved various issues during development.
- Primary DBA for DCM application support. Resolved critical issues by restoring databases and resolving archive issues on DCM database.
- Security administration - Developed scripts to identify weak passwords.
- Security administration - Developed process to identify and notify multiple logon attempts.
- Workload Management - Tuned TASM settings to have highest throughput for DART workload
- V2R6.2 Software and TTU 12.0 Upgrade Project - Successfully upgraded both WS and SA Production to V2R6.2. Provided support for successful upgrade of TTU to 12.0 and supported in resolving compiler issues for COBOL programs.
- Security administration - Successful implementation of new password controls on Production Teradata.
- Release Management - Endevor analysis - Identification of dynamic PARMLIB creation - obsolete members - New naming convention - Participation in review meetings
- Performance and Tuning - Analysis of long running batch jobs - Recommendation to change the schedule of collect stats jobs - this helped many batch jobs not running into business hours.
- Developed process for tracking and efficiently manage all Teradata space requests in a spreadsheet. Using VBA macros in excel, developed code to retrieve actual space request from production and provide any overestimation or underestimation
- Developed process for tracking Object changes which would help in audit team to track the changes without DBA intervention
- Developed new method of doing MVC compression in 14.10 using Statistics.
- Analysis of long running batch jobs - Recommendation to change the schedule of collect stats jobs - this helped many batch jobs not running into business hours and saved lot of CPU cycles
- Major contribution for the PEN test (PENetration Test). Developed process to identify weak passwords and resolve them, put new job to have security control on test box.
- Implementation of new password controls on all Teradata Systems. Optimizing the run times of long running batch jobs.
- Lead DBA team meetings to resolve issues from DBA abends and other DBA related enhancements.
- Successfully lead Teradata 6700 Hardware floor sweep Project identifying tasks, risks and coordinating with other teams like Data Center Operation personnel, Network team, solutions team to complete those tasks and mitigate the risks. Benchmarked the inclusion of Solid State Drivers (SSD) in the 6700 Storage with Teradata Virtual Storage (TVS) enabled.
- Support Viewpoint, Teradata Multi Systems Manager (TMSM) software upgrades.
- Support Teradata Unity Director Implementation to replace Teradata Query Director (TQD).
- Responsible for Integration of Informatica Power Center with Teradata using Teradata Parallel Transporter (TPT) Connectors. Setup Standards and guidelines for ETL design.
- Responsible for generating Teradata Roadmap in Confidential ’s. Coordinate with Architects and vendors to come up with the next generation (BI) Business Intelligence Ecosystem.
- Generation of Teradata Usage Reports for capacity planning to identify business analytics growth which feeds into the Roadmap Strategy.
- Develop Teradata Usage Charts for higher management for both Business and IT.
- Provide guidance and support for using Protegrity on other platforms like SQLServer.
- Participate in Disaster Recovery (DR) Exercise for Teradata.
- Review and evaluate next generation BAR (Backup Archive & Restore) solution that reduces cost and storage foot print. Also examined current back up process and generated reports supporting it.
- Participate in internal and external SOX and PCI Audit reviews. Provide Access Control and Change control reports with evidence documents supporting the change.
- Coordinate with development team to setup websites that simplify access request process. Integration with ASP (Active Server Pages) and .Net Participate in design sessions and implement database objects supporting the website,
Environment: Teradata Administration, Teradata SQL, Teradata Physical Implementation, Teradata V2R4, V2R5, V2R6 releases, Teradata Client Tools and Utilities (Teradata Administrator, Teradata SQL Assistant, Teradata Manager, TSET, BTEQ, FastLoad, Multiload, FastExport, Arcmain), MVS JCL, NCR UNIX (MP-RAS), Zeke, Endevor, Informatica, DataStage, Viewpoint, TMSM, Unity, Teradata Query Director, TPT, Teradata 6700.
Confidential, Minneapolis, MN
Teradata Database Administrator
Responsibilities:
- Design, creating and tuning physical database objects (tables, views, indexes) to support normalized and dimensional models.
- Participate in all essential day-to-day database support activities.
- Supporting and tuning for queries from Business Objects reporting tool and BO XI Release 2 migration.
- Application Development for Data Marts. Involved in development of load processes on MVS and NT for the new and existing data marts. Development of applications using OLE-DB Access Module and K-Shell scripting to load and unload data between Teradata and other databases like DB2, SQLServer, Sybase and Oracle to support applications on the respective databases.
- Ongoing 24x7 support of the production environment.
- Integrating QlikView reporting tool with Teradata for Sales and Reporting Analysis (SARA). Benchmarking and configuring various Teradata client tools and interfaces with QlikView.
- Writing JCL scripts on MVS for streamlining various process.
- Administration and monitoring of the database using Teradata manager and Teradata Administrator (WinDDI).
- Creating incidents to NCR GSC for issues in Teradata optimizer and monitoring for problem resolution.
- Coach developers and application teams on best practices from a performance and standards perspective.
Environment: MVS, JCL, FILEAID, ISPF, WSF2, SORT, Queryman, BTEQ, Fastload, Multiload, FastExport, TPump, Teradata SQL.
