Hadoop Engineer Resume
Tampa, FloridA
SUMMARY:
- Around 10+ years of Information Technology experience. Extensive experience in administration, Production support, design, development, and implementations of robust technology systems, with specialized expertise in BIGDATA technologies and Data Warehousing solutions.
- Able to understand business and technical requirements quickly; Excellent communications skills and work ethics; Able to work independently; Experience working with clients of all sizes in the healthcare, insurance, financial, Retail and manufacturing industries
- Around 3 +years of experience in Hadoop Administration/Production Support /Architecting & Big Data Technologies and 7 years of experience into Data ware housing tools - Cognos administration/development and ETL Informatica.
- Extensive experience on providing solution In all areas like capacity planning, Applications onboarding, designing, User Management, Resources allocation, Security, High availability and deploying in Production.
- Good Exposure on Analytical tools, third party tools (Pepper data, Unravel, Blue data, Data Meer, Arcadia) that supports/enhance Bigdata system capabilities, DB systems and Amazon Web Services.
- Experience with complete Software Design Lifecycle including design, development, testing and implementation of moderate to advanced complex systems.
- Hands on experience in installation, configuration, supporting and managing Hadoop Clusters using Apache, Hortonworks, Map R and Cloudera.
- Good hands on experience on LINUX Administration and troubleshooting issues related to Network and OS level.
- Working Knowledge on Linux concepts and building servers ready for hadoop Cluster setup.
- Extensive experience on monitoring servers with Monitoring tools like Nagios, Ganglia about Hadoop services and OS level Disk/memory/CPU utilizations.
- Extensive working experience on Big-data integration with tools like INFORMATICA BDE, GREENPLUM, TABLEAU, SSRS, SAS, SQL server.
- Experienced on Setting up clusters in cloud AMAZON AWS and troubleshooting user issues over cloud clusters.
- Worked on setting up tools like R studio and Pyspark for Data Scientist team.
- Hadoop Cluster Capacity planning, Performance tuning.
- Design Big Data solutions for traditional enterprise businesses.
- Installation of various Hadoop Ecosystems and Hadoop Daemons.
- Installation and configuration of Sqoop and flume.
- Excellent command in creating Backups & Recovery and Disaster recovery procedures and Implementing BACKUP and RECOVERY strategies for off-line and on-line Backups.
- Experienced on supporting Production clusters on-call support and troubleshooting issues within window to avoid any delay’s.
- Hands on experience on maintaining and administering on Cluster metadata in MYSQL/ORACLE data base with proper backups and Master Slave replications.
- Involved in bench marking Hadoop cluster file systems various batch jobs and workloads
- Making Hadoop cluster ready for development team working on POCs.
- Experience in minor and major upgrades, Patches of hadoop and hadoop eco system.
- Experience monitoring and troubleshooting issues with Linux memory, CPU, OS, storage and network
- As admin involved in Cluster maintenance, trouble shooting, Monitoring and followed proper backup& Recovery strategies.
- Installing and configuring of all hadoop eco systems like Sqoop, pig, and hive.
- Experience in importing and exporting the data using Sqoop from HDFS to Relational Database systems/mainframe and vice-versa.
- Handsome experience in Linux admin activities on RHEL &Cent OS.
- Extensive knowledge on Data warehousing concepts, reporting, relational data bases.
- Good knowledge and experience on Informatica and its workflow monitoring as part of production support.
- Good Leadership skills and Team player, solely handled Projects and delivered in time.
TECHNICAL SKILLS:
Hadoop Framework: Hortonworks, Cloudera, Mapr, HDFS, Map Reduce,YARN, Pig, HiveHbase, Sqoop,Kafka, Zoo keeper, Oozie, Flume, Spark, Storm.
Hadoop Security Tools: Ranger, Sentrify,kerbores, LDAP,ACLS
Monitoring Tools: Nagios, Ganglia, Ambari Alerts.
Configuration Mgmt tools: Ambari, Cloudera manager, Stack IQ, Puppet.
Reporting Tools: Cognos 10.2/10.1.1/8.4 , 7.x/8.x Suite, Report Net, Tableau, qlickview
ETL Tools: Informatica BDE, Workflow Monitor
Microsoft: MS Office, MS Project, MS Visio, MS Visual Studio 2003/ 2005/ 2008
Databases: Oracle 8i/9i/10g,12c, SQL, MYSQL.
Operating Systems: RedHat, Cent OS,RHEL,Windows 2000/2003/2008/ XP/Vista
Scripting: Shell Scripting, HTML Scripting.
Programming: C, C++, Core Java, R programming.
PROFESSIONAL EXPERIENCE:
Confidential, Tampa, Florida
Hadoop Engineer
Responsibilities:
- As Part of Hadoop Engineer responsible for all Hadoop New application on boarding, architecting providing solutions.
- Handling new Hadoop Components implementation and making them available for applications.
- Helping teams to do POC with New Hadoop components.
- Responsible for Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoring, Troubleshooting, Manage and review data backups, Manage & review Hadoop log files.
- Troubleshooting, Manage and review data backups, Manage & review Hadoop log files.
- Setting up different FLUME agents based on the team requirements to bring the log files from different source systems.
- Extensively worked/experience on Hive and its configurations tuning.
- Monitoring systems and services, architecture design and implementation of Hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
- Changing the configurations based on the requirements of the users for the better performance of the jobs.
- Experienced and working on end to end terminology of Hadoop Big data Eco system Components like Pig, Sqoop, Zookeeper, Hive, Map reduce.
- Hands on experience in analyzing Log files for Hadoop and eco system services and finding root cause.
- Experience on YARN resource allocation with Capacity scheduler/Pools on various environments and tuned configurations according to the application wise job loads.
- Prepare project concepts, Gap Analysis, solution design and Study and analyse user requirements, Business process, workflow to translate them to business and functional requirements and design solutions.
- Supporting many third-party tools integration with Hadoop ECO system and making sue app teams will have smooth run.
- Work on developing the functional requirements and documenting the technical requirements and provide a roadmap for the development and implementation.
- Documenting all the efforts and publishing best practices for application development and performance planning and improvement.
- Ensure proper testing is done to the product.
- Work with the client to get the performance tested as per the requirements specified by the client.
- Working with ETL development team for best practices and performance recommendations.
Confidential, Baltimore, MD
Hadoop Administrator
Responsibilities:
- As a hadoop Administrator working on capacity planning and designing clusters with SSA Standard Enterprise level along with Security model.
- Supporting the existing applications in production clusters with daily jobs and data ingestion jobs.
- Extensive hands on experience on cluster upgrades, patches and configuration changes based on requirements with proper backup strategies.
- Experienced in production support which involves solving the user incidents varies from sev1 to sev5.
- Working with vendor on product-based issues and applying hotfixes if required.
- Day to day responsibilities includes solving hadoop developer issues and providing instant solution to reduce the impact and documenting the same and preventing future issues.
- Experienced in LDAP integration with hadoop and Access provisioning for secured cluster.
- Installation and configuration of Linux for new build environment.
- Installation, configuration and maintenance FTP servers, NFS, RPM and Samba.
- Performed various configurations which include networking and IPTables, resolving hostnames, SSH key less login.
- Managing Disk File Systems, Server Performance, Users Creation and Granting file access Permissions and RAID configurations.
- Automate administration tasks through use of scripting and Job Scheduling using CRON.
- Implementing the SFTP for the projects to transfer data from External servers to hadoop servers.
- Helping the developers in code deployments from DEV to UAT and then PRODUCTION throughout the process.
- Experienced in implementing and configuring CLOUDERA and HDP tools like cloudera sentry Ranger, Ambari Alerts, SMART SENSE.
- Implemented NAMENODE, YARN high availability and Hive Metastore/hiveserver2 for cluster load balancing.
- Installation of SPARK, STORM, KAFKA and configuring it as per the requirements.
- Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes. Communicate and escalate issues appropriately.
- As a admin followed standard Back up policies to make sure the high availability of cluster.
- Involved in Analyzing system failures, identifying root causes, and recommended course of actions. Documented the systems processes and procedures for future references.
- Worked with systems engineering team to plan and deploy new hadoop environments and expand existing hadoop clusters.
Confidential, Phoenix
Hadoop Administrator/Architect
Responsibilities:
- As a hadoop Administrator worked on multiple Cloudera/Hortonworks clusters for different clients as a part of regular day to day admin/support activities.
- Installed multiple clusters for different clusters on physical and Amazon AWS servers and supported multiple applications.
- Designed and implemented multiple hadoop Eco-System/related tools and helped users throughout the process.
- Worked with Data scientist teams to setup and troubleshoot tools like R studio,Pyspark,Jupiter and many other tools.
- Implemented kerbores authentication and AD integration with the clusters.
Confidential, Tampa
Hadoop Administrator
Responsibilities:
- Worked as hadoop Administrator/Architect on 100 node clusters ranges from POC clusters to PROD with Hortonworks Distribution 2.2.4.2.
- Provided Architectural solutions in implementing security within the clusters and while accessing it from hadoop-integrated tools.
- Helped business in Connecting hadoop with the existing tools like tableau, Cognos, SSRS reporting and SAS tools, specially Informatica Big Data Edition to process the data in easiest way.
- Solo responsible for everything related to clusters starting from Maintaining, Monitoring and keeping up the cluster all the time by supporting 24/7 to support business without any outages.
- Extensive experience on Linux Admin skills in maintaining the hadoop services over the Linux servers and balancing the services and its utilizations.
- Good experience on On-Boarding process of new hadoop application teams into hadoop cluster with proper POC’s to give better picture to the teams about how hadoop helps them for their requirements,
- Working with Tools like INFORMATICA BDE to cleanse the data in hadoop hdfs and loading the same to GreenPlum Db for business to prepare reports and business analysis purpose.
- Integrated different tools like SAS, Tableau, SSRS with hadoop, this way users can pull data from HDFS hive.
- Good experience on Ranger tool and its working behavior to provide security to the cluster.
- Responsible for commissioning and decommissioning of Data nodes.
Environment: HDP 2.2.4.2, AMBARI, Hive, pig, Sqoop, Flume,RANGER,LDAP, Zookeeper and Hbase, MYSQL Shell Scripting, Redhat Linix RedHATBodhtree solutions INC
Confidential, Phoenix
Hadoop Administrator
Responsibilities:
- Currently working as hadoop administrator in MapR hadoop distribution for 5 clusters ranges from POC clusters to PROD clusters contains more than 1000 nodes.
- Responsible for Cluster maintenance, commissioning and decommissioning Data nodes, Cluster Monitoring, Troubleshooting, Manage and review data backups, Manage & review Hadoop log files.
- Day to day responsibilities includes solving hadoop developer issues and providing instant solution to reduce the impact and documenting the same and preventing future issues.
- Experience on Mapr patching and upgrading the cluster with proper strategies.
- Monitoring systems and services, architecture design and implementation of hadoop deployment, configuration management, backup, and disaster recovery systems and procedures.
- Hand on experience on cluster up gradation and patch upgrade without any data loss and with proper backup plans.
- Changing the configurations based on the requirements of the users for the better performance of the jobs.
- Worked on configuration management tools like stack Iq to maintain central and pushing the configurations to the overall cluster for all config relates hadoop files like mapred-site.xml,pools.xml.hdfs-site.xml.
- Experienced in Setting up the project and volume setups for the new hadoop projects.
- Involved in snapshots and mirroring to maintain the backup of cluster data and even remotely.
- Implementing the SFTP for the projects to transfer data from External servers to hadoop servers.
- Installation of various Hadoop Ecosystems and Hadoop Daemons.
- Experienced in managing and reviewing Hadoop log files.
- Working experience on maintaining mysql databases creation and setting up the users and maintain the backup of databases.
- Setting up Mysql master and slave replications and helping business applications to maintain their data in Mysql Servers.
- Helping the users in production deployments throughout the process.
- Experienced in production support which involves solving the user incidents varies from sev1 to sev5.
- Managed and reviewed Hadoop Log files as a part of administration for troubleshooting purposes. Communicate and escalate issues appropriately.
- As a admin followed standard Back up policies to make sure the high availability of cluster.
- Involved in Analyzing system failures, identifying root causes, and recommended course of actions. Documented the systems processes and procedures for future references.
- Worked with systems engineering team to plan and deploy new hadoop environments and expand existing hadoop clusters.
- Monitored multiple hadoop clusters environments using Nagios. Monitored workload, job performance and capacity planning using MapR control systems.
Environment: Mapr hadoop M5, Hive,pig, Sqoop, Flume, Zookeeper and HBase,MYSQL Shell Scripting, Redhat LinixBodhtree solutions INC
Confidential, San Jose
Congos consultant/Hadoop Admin
Responsibilities:
- Internally got a chance to work with hadoop team on mapr hadoop distribution.
- As part of initial setup worked on hadoop POC project and setup a new environments with all hadoop eco system components.
- Copied data from Relational databases to hadoop HDFS using sqoop and fulfilled business requirement.
- Integrated LDAP with the cluster and granted access to users.
- Got trained internally in Confidential in hadoop administration as part of new team building for one of the project.
- Worked as cognos consultant where in I need to create project models and develop packages for report creations.
- Responsible for the end to end process of gathering the business requirements from the user and implementing them in project models and applying business rules.
- Creating dashboard based on requirements which contain complex logics to show the data in use friendly view.
- Solely responsible for the almost 30 to 40 packages enhancements new reports creations, production bug fixing and promoting them to Live.
- Used Cognos Connection for organizing, security, scheduling and distributing reports. Created multiple layer report providing a comprehensive and detail report with Drill through facility.
- Production deployments and helping with the production issues.
- Install, configure and maintain base software and Patch & Release Management.
- Defined the users, groups, and roles for security purposes, configure capabilities for the interfaces and studios.
- Setting the properties for user interface profiles to perform task according to the privileges they have Code migrations and documenting the same for future work-instructions.
- Developed adhoc reports using Report Studio that include Crosstab, master-detail reports, drill-through, prompts, bursting reports, charts.
- Performance tuning of the reports.
- Day to day interactivity with the users to solve their issues.
- Involved in Unit testing of the reports
Confidential
Cognos Consultant
Responsibilities:
- Experienced inCognos10 version implementation and administration.
- Configured multi-tier environment level of security in Cognos 8 BI using third-party authenticators like MS Active Directory. Designed and configured user roles and access to packages and reports.
- Provided troubleshooting and diagnostic support at L2 / L3 level for the Cognos suite.
- Gathering requirements from project teams covering areas such as capacity, users, data
- Creation and modification of connections/sources, tools/capabilities required for the Environment.
- Initiate, submit and implement change requests
- Backup, data recovery and service restoration management
- Collaborate with L3 on patch and bug fix deployment
- Identify and raise system capacity related issues (storage, licenses, performance threshold)
- Performance, capacity & utilization monitoring
- Optimization and Performance tuning
- Developed the complex FM models with the different databases like combination of Sybase and oracle data sources in a single project as per the business requirement.
- Enhancing the existing FM model based on the IRIS front end application as a proof and testing the same.
- Strong knowledge on troubleshooting and performance tuning of Framework model.
- Extensively participated in gathering the requirements for the Client sphere dashboards starting from creating the schema, functional and technical documents and FM model and suggesting the best way to business reducing the cost and time.
- Worked with various users all over the globe for other project WIMQMS which is related Trades and Utilities.
- Implemented the user level security at Framework level and updated the backend table.
- Developed the reports with tabular SQL reduce the complexity and improve the query performance
- Participated in Migration from Cognos 8.2 to 8.4 and cognos10.2.
- Demonstrated/Trained the staff in Understanding the Project Front end application to understand the business.
- Co-ordinate with development team by assigning tasks and deliverables and resolving high priority issues.
- Trained end users in understanding the functionality of Cognos Report Studio, Query Studio and Business Insight.
Environment: Cognos 10.2/10.1 and 8.4 Suite (Cognos Connection, Framework Manger, Report Studio, Query studio, Business Insight, Active Reports, cognos administration), Oracle 10g, MS Visio .HP Quality center.
Confidential
Sr.Cognos Consultant /Admin
Responsibilities:
- Involved in requirement gathering and analysis developing specification documents for reports as per client requirements.
- Co-ordinate with development team by assigning tasks and deliverables and resolving high priority issues.
- Used Framework Manager to build models, Segments, Security (Object, Data, and Package), set Governors, resolved complex cardinalities and integrated VSS.
- Developed new reports using Report Studio that include Crosstab, master-detail reports, drill-through, prompts, bursting reports, charts.
- Used Cognos Connection for organizing, security, scheduling and distributing reports. Created multiple layer report providing a comprehensive and detail report with Drill through facility.
- Converted financial reports from MS Excel to Transformer Power cubes and created reports in Cognos Analysis Studio.
- Responsible for definition, deployment and monitoring of important organizational metrics using Cognos Metrics Manager.
- Built models using Framework Manager and power cubes using Transformer.
- Developed Standard Reports, Charts, Drill Through Reports, Master Detail Reports, Bursting Reports and Multilingual Reports Using Report Studio and Query Studio.
- Worked with ETL team to load data into Data Warehouse/Data Marts using Informatica.
- Extensively worked on Analysis Studio to develop multi dimensional reporting.
- Upgraded reports from Cognos 8.3 to 8.4 and modified packages to enhance report performance.
Environment: Cognos 8.3/8.4 Suite (Cognos Connection, Framework Manger, Report Studio, Query Studio,),Oracle 10g, Cognos LDAP, Windows 2003 Server.