Big Data Platform Administrator/architect Resume
SUMMARY:
- An accomplished, customer - focused Big Data Platform Engineer with wide ranging 11 years of IT experience in Hadoop, Bigdata and Oracle with multiple project implementations, support and maintenance.
- Strong Hands-on and up to date skills on the state-of-the art Analytics technologies in Big Data including Hadoop Eco-system, Hive, Spark and Visualization tools (Tableau, Qlikview Power BI). Wide ranging experience in Data Management Technologies to provide effective solutions in
- Big Data/Modern Data Architectures. Deep technical expertise and Wide business exposure to understand and solve business problems with technology.
- Proven track record of designing and implementing architectural solutions that solve strategic and needs in big data area.
- Expertise in solving business problems with the best of the breed technical solutions.
- Solid admin/infrastructure expertize on the Apache Hadoop with hortonworks and cloudera distributions.
- Hands on technical architect with wide range of experience in enterprise solutions
- Expertise in designing big data and analytics solutions using diverse big data tools and techniques.
- Possesses proven ability in applying analytical techniques in monitoring the trends and relationships between different data sets.
- Expert in defining security frame work and data governance.
- Compliance with change control standards.
- Develops and implements recovery plans and procedures.
- Very good knowledge and understanding on Data modeling.
- Extensively worked on large scale production environments.
- Experienced with networking infrastructure including VLAN and firewalls
TECHNICAL SKILLS:
BIG Data Platforms: Hortonworks, Cloudera, Map R
Big Data associated: SQl, Hive, Spark, Sqoop, Oozie, Kafka, Flume, Nagios, Nifi
Database (RDBMS): Amazon RDS, Oracle 9i/10g/11g/12C, SQL Server 2008 R2 /2012/2014MySQL, Postgres, DB2
Cloud: Microsoft Azure, AWS, EC2, EMR, Redshift, S3, RDS, Oracle OPC cloud,Google cloud
Modelling: Kimball, Inmon, Data Vault (Hub & Spoke), Hybrid
No SQL: HBase
Languages: HQL, Oracle SQL, T SQL, R, R-Hadoop & Python, Core Java, JDBC
BI/Data Discovery Tools: QLIKVIEW, TABLEAU, SPOTFIRE, Power BI, Oracle OBIEE
ETL/Data Integration Tools: Informatica ETL, BDM
Scripting: UNIX Shell Scripting
Code/Task management: Github, scrum
Tools: & Utilities SQL Developer, Putty, Toad, CVS, SVN, Eclipse, HUE
Security Tools: Knox, Ranger, Kerberos, Sentry
Storage: Exadata, SAN, NAS, HDFS
PROFESSIONAL EXPERIENCE:
Big Data Platform Administrator/Architect
Confidential
Responsibilities:
- Implement Hadoop echo system and tools in Hortonworks Data Platform (HDP 2.6) in Microsoft Azure cloud.
- Lead the datalake project, involved in all the phases of the project Analyze, design, ETL and testing.
- Provide big data solutions by leveraging various Hadoop echo system tools.
- Provide support for Hortonworks Data Platform (HDP) cluster with Ambari and Apps.
- Subject Matter Expert (SME) in HDP Ambari with auto-configuring Apps like Hive, Zookeeper, Oozie, Sqoop, Flume, Spark, HBase.
- Create centralized datalake to store the data coming from various sources.
- Automate data ingestion process using oozie and shell scripting.
- Work closely with infrastructure, network, database and application teams to ensure business applications are highly available and performing within agreed on service levels.
- Accountable for storage, end-to-end performance tuning and volume management of Hadoop clusters and MapReduce routines.
- Monitor Hadoop cluster connectivity and performance.
- Manage and analyze Hadoop log files
- HDFS support and maintenance
- Develop and document best practices.
- Maintain backups of Hadoop environments.
- Responsible for the new and existing administration of Hadoop infrastructure
- Development experience with Sqoop, Hive and Spark.
- Troubleshooting experience to identify and fix bottlenecks for optimal performance of the Hadoop clusters.
- Install and maintain Hadoop clusters (including monitoring, security, access management, backup and performance tuning
- Create data pipelines and develop DDL for transformations.
- Involved in deployment phase meetings for change management.
- Involved in project Life Cycle - from analysis to production implementation, with emphasis on identifying the source and source data validation, developing particular logic and transformation as per the requirement and creating mappings and loading the data into different targets.
- Involved in identifying opportunities for process/cost optimization, process redesign and development of new process.
- Wrote Release notes, Deployment documents and scheduled the jobs.
- Support team on Bigdata testing.
- Define a high level data governance framework addressing data discovery, data lineage, data quality, data catalogue, metadata management, data privacy and security, and data lifecycle management
- Document the environment configuration of all hardware and software.
- Mentor the team to bring up to speed on big data technology areas. Give Knowledge transition sessions to the team.
- Extracted the data from Oracle, SQL server usingSqoop into HDFS.
- Scale up/down the environment by adding/removing the nodes, server resources.
- Job tracking using job tracker.
- Monitor cluster services and ensure they are available.
- Very good understanding ofPartitions, Bucketingconcepts in Hive and designed both managed and externaltables in Hive to optimize performance
- Install/configure security tools for fine grained data security.
- Enable data security (Authentication, Authorization, Data protection) and access management.
- Install/config security tools (Kerberos, Knox, and Ranger). Integrate the cluster with enterprise LDAP.
- Actively interact with all stake holders meetings.
- Have deep understanding on Hive partitioning, bucketing and LLAP.
- Implemented fine grained access controls using Apache Ranger and Knox.
- Solved performance issueswith understanding of Joins, Group and Aggregations.
- Have a great experience on working UNIX like systems.
- Interact with multiple stake holders Business Analysts and Data Modelers, Developers and defined Source and Target Mapping documents, and documenting best practices.
- Used Tableau, Qlikview and Power BI for the reporting purposes.
ENVIRONMENT: HDFS, Yarn, Hive, Sqoop, Hbase, Pig, Flume, Spark, Kafka, Python, Pig, Oozie, HBase(No SQL), Shell Scripting, HCatalog, RDBMS(SQL server & Oracle Warehouse)
Confidential
Hadoop Administrator/Infrastructure Lead
Responsibilities:
- Responsible for implementation and support of the Cloudera Hadoop environment and tools used in Hadoop ecosystem.
- Collaborate with team members and clients to deliver a technical solution that meets the unique needs of our clients.
- Have vast experience in clinical trial procedures and terminology. Have very good understanding on various clinical trial phases.
- Involved in Big data requirement analysis, design and design solutions for ETL and Business Intelligence platforms.
- Proactive monitoring of Hadoop cluster
- Hadoop environment setup using Hortonworks Data Platform (2.4).
- Conduct detailed analysis of system and application architecture components as per functional requirements
- Experience in interacting with Business Analysts and Data Modelers and defined Source and Target Mapping documents, and documenting best practices.
- Expertise of Hadoop1.x (MapReduce) & Hadoop 2.x (Yarn) programming models.
- Used Sqoop extensively to import & export data to and from RDBMS systems like (Oracle, MySQL, Sqlserver) in to HDFS, HIVE data warehouse, HBASE.
- Mapping Hbase Key-values pairs to Impala tables to achieve optimal performance.
- Partitions creation of impala to suit the business requirements
- Experience in writing Hive/HQL scripts to extract and load data in to Hive Data warehouse.
- Experience in writing PIG Scripts to read, transform large sets of structured, semi structured and unstructured data, and load into HDFS, HIVE.
- Create quality documentation to communicate incident reports or to appropriate audiences.
- Experience with Oozie Workflow Engine in running workflow jobs with actions that run Hadoop Map/Reduce and Pig jobs.
- Performed Hadoop day-to-day operations (HDFS, Map-Reduce, Hbase, and Hive) including operation, deployment and debugging of job issues.
- Developed semantic layer on top of Hive to facilitate the analytics team to generate ad-hoc reports.
- Used Hive to analyze the partitioned and bucketed data and compute various metrics for reporting.
- Experienced in managing and reviewing the Hadoop log files.
- Used Pig as ETL tool to do Transformations, even joins and some pre-aggregations before storing the data into HDFS.
- Involved in creating Hive Tables, loading data and writing hive queries.
- Worked on implementing the Master Data Management strategies for the new analytics platform.
- Defining and creating the Unified Data Platform for all the Enterprise needs.
- Exported data from HDFS environment into RDBMS using Sqoop for report generation and
- Visualization purpose.
- Worked on Oozie workflow engine for job scheduling.
- Involved in Unit testing and delivered Unit test plans and results documents.
ENVIRONMENT: HDFS, Yarn, Sqoop, Flume, Hive, Spark, Spark Streaming, Kafka, Python, Scala Hadoop, Java Map Reduce, Pig, Oozie, HBase, Shell Scripting, HCatalog, RedShift, RDBMS, Postgresql
Confidential
DBA Lead
Responsibilities:
- Managed Dev, Test, Prod Oracle 10g, 11g assigned Databases.
- Supporting Developers, Testers on transferring newly created/updated codes from Dev to Test to Production databases release.
- Performance Tuning (using AWR, ASH, ADDM)
- Supported backup and recovery using RMAN, and TSM. Implemented on Dev and Test.
- Cloning of databases Daily database administration doing tables partitioning, export/import using Data Pump, creating Indexes, synonyms, sizing tables, refreshing Materialized view, approving codes for testing and
- Implementation to Production.
- Supporting, troubleshooting RAC and Data guard. RAC Installations, adding/deleting Nodes
- Patching RAC databases. Troubleshooting Node/Instance evictions. Implemented RAC with
- OCFS/OCFS2 file systems. Managing, Restoring and Recovering Clusterware Components (OCR, VD)
- Maintaining databases using Oracle Grid Control and TOAD.
- Managed on-call Production DBA activities such as checking on tablespace problems, nightly/weekly
- Backup problems, responding to Oracle errors, and blocking locks conditions.
- Applying Oracle patches. (CPU)
- Excellent troubleshooting, diagnostic & analytical skills.
- Expert knowledge of database scripting
- Diagnosed and corrected problems in Production and non-Production Databases. Doing root-cause analysis and implementation of permanent solutions
- Rehosting of databases to new servers and storage. Database refresh using RMAN.
- Monitor database capacity and estimating growth based on the current/previous size/load.
- Doing a continuous database health check of the databases.
- Review physical data models, data access patterns, and database design.
- Purging of Production database and Migration of databases from old server/storage to a new one.
- Done patch analysis, applied patches (one-off, mini packs, family packs) and checked the patch impact. Managing patch history till go-live. Review patches
- Observe patch impact and applying Applications patches including pre requisites, co-requisites and post requisites
- Reduced the timelines of patching using features like defaults file, various ad patch options and merge patch.
- Aid development teams for analyzing issues.
- Troubleshooting several worker issues while patching.
- Applied RDBMS patches using OPatch.
- Manage and troubleshoot apps components like apache/forms etc.
- Creation of application user accounts and assigning the responsibilities.
- Moving customized concurrent program executable from one environment to other i.e. development to test, test to production.
- Lead the team in Database design and development.
- Created RPDs, defined Physical Layer, and developed Business Model & Mapping and Presentation Layer using Admin Tool.
- Created and managed OBIEE dashboards.
- Develop/Migrate/Customize ETLs for Data Extraction, Transformation and Loading using Informatica Power Center 9.0.1/9.1.0 .
- Involved in performance tuning and code review.
- Involved in DAC monitor.
- Involved in creation of QC defects, PPM packages, Requests and tuned code uploaded into Clear case.
- Migrated mappings from one environment to other environment.
- Involved in testing the OBIEE existing reports.
- Involved in preparing Unit test plans and test results.
- Involved in preparing high level migration documentation.
- Taking care of SQL*Net related problems like listeners and connect strings
- Manually created single instance and later added it with the RAC
- Building Physical standby database
- Verifying backups in the Recovery Catalog database
- Management of Database Creation and Maintenance of Table spaces, Data files, Control files, Redo Log files.
- Monitoring the Storage Usage and Disk I/O by managing Table spaces, Data files and OS File Systems.
- Implementation of security and Integrity of Users connected to the Database by Enrolling, Monitoring and Dropping Database Users, Roles, Privileges and Profiles.
ENVIRONMENT: Oracle 10g, 11g, Solaris, AIX, RMAN, TOAD, RAC, ASM, Active Dataguard. RAC, Grid OBIEE 10g/11g, BI Publisher, Informatica Power Center 9.0.1/9.1.0, Oracle Database 11g, Oracle SOA
Confidential
Senior DBA/Unix admin
Responsibilities:
- Had a great experience and exposure working with ORACLE provide end to end database/application administration services for top 25 customers (American Air Lines, Clopay, Aramark, Essilor, School Speciality, Delmonte etc...) these deals with high criticality and customer sensitivity.
- Worked in ultimate environments like Multi-Nodes (up to 25nodes) of App Tiers with shared application tops, shared technology stacks, PCP, Discoverer, RAC,ASM, RMAN and Standby for databases.
- Worked in Oracle Applications releases starting from 11.5.9 to 12.1.3
- EBS application patching, cloning, configuration changes, database and application upgrades.
- Worked in databases sizes vary from 500GB to 12TB and the database versions starting from 9i to 11g.
- Building Physical standby database
- Verifying backups in the Recovery Catalog database
- Management of Database Creation and Maintenance of Table spaces, Data files, Control files, Redo Log files, Archive Log files.
- Monitoring the Storage Usage and Disk I/O by managing Table spaces, Data files and OS File Systems.
- Implementation of security and Integrity of Users connected to the Database by Enrolling, Monitoring and Dropping Database Users, Roles, Privileges and Profiles.
- Monitoring the Hit Ratios and tuning the System Global Area (SGA) accordingly
- Applied RDBMS Patches.
- Performing Database Cloning from production to test servers using RMAN.
- Reorganization of tables, indexes and Lobs.
- ASM to Non ASM cloning
- Database Patching and Cloning.
- Automated several processes like daily logical backup and physical backup.
- Creation and maintenance of other database objects like views, synonyms, sequences, and database link.
- Creating database roles & assigning privileges to the roles.
- Creating database users and by setting the default & temporary table spaces, assigning quotas, roles, privileges and profiles to users.
- Making table spaces available to users by adding data files to the table spaces and resizing of data files whenever the table spaces crossed the threshold.
ENVIRONMENT: Oracle Database 10,11g Exadata, Oracle applications (11.5.10, 11.5.10.2, R12), RAC, ASM, Data Guard, RMAN
Confidential
DBA/System Administrator
Responsibilities:
- Monitoring Concurrent Manager & its operations like starting, stopping the concurrent managers & Troubleshooting problems, resolved the view log/out files issues of concurrent requests.
- Cloning, maintaining concurrent manager processes, Patching, space management issues and User management issues related to Oracle Applications.
- Very well versed in Oracle Applications Utilities like AD Administration, AD Controller, Autoconfig, AD Merge Patch, AD Relink, AD Splice, Auto Patch, FNDCPASS, f60gen, WFLOAD, FNDLOAD etc.
- Assigned specific concurrent requests to a concurrent manager.
- Generated the context.xml files, and environment files related to apps.
- Monitoring, identifying and resolving user issues.
- Managing passwords of apps, Product passwords.
- Altered several components configurations like port numbers and made them Functional.
- Monitoring the Application System with Oracle Applications Manager (OAM)
- Have expertise in activities like maintaining instances, maintenance of file system and taking appropriate backups for security and consistency.
- Validating & Creating grants & synonyms on Apps schema, Performed jar file/forms generation as a post patch step, relinking the Oracle apps programs.
- Performed research for patches for specific problems and downloaded patches
- Done patch analysis, applied patches (one-off, mini packs, family packs) and checked the patch impact. Managing patch history till go-live. Review patches
- Observe patch impact and applying Applications patches including pre requisites, co-requisites and post requisites
- Reduced the timelines of patching using features like defaults file, various ad patch options and merge patch.
- Troubleshooting several worker issues while patching.
- Applied RDBMS patches using OPatch.
- Managed and troubleshooted apps components like apache/forms etc.
- Creation of application user accounts and assigning the responsibilities.
- Moving customized concurrent program executable from one environment to other i.e. development to test, test to production.
- Worked on various issues like User deletions, User additions, lock accounts, enable auditing for a user, grant privileges from one schema object to another schema objects, collection of statistics for schema level, database level, Temporary table issues, data file addition, moving data files, snap shot too old issues.
- Taking care of tablespace related issues like increasing the storage parameter values, resizing the datafiles, adding new datafiles etc
- DB Link creation
- Taking care of SQL*Net related problems like listeners and connect strings
- Manually created single instance and later added it with the RAC
- Oracle network Administration by Configuring tnsnames.ora, listener.ora.
- Establish and Maintain Sound Backup Policies and procedures including Hot and Cold O.S. Backup and also logically through Oracle Export Utility.
- Backup using Recovery Manager (RMAN).
- Checking whether archive logs are applying from primary to standby correctly or not If not fixing the issue.
- Moving data files from one disk group to another disk group in ASM.
- Collecting AWR reports.
- Performance tuning.
- Monitoring Logical Standby database.
- Documenting the database setup & updating the document whenever there is a change in the database structure, backup strategy etc.
ENVIRONMENT: Oracle Database 9i/10g, Oracle EBS 11i/R12, RAC, ASM, Data guard, RMAN, Oracle enterprise manager, Unix Sun, Solaris OS.