OBJECTIVE

• Seeking a challenging position within a fast paced environment that enables me to add value and strengthen my development skills.
• I have experience in HadoopAdministration which includes Map reduce, Hive, Oozie, Scoop, Hbase, Pig, HDFS, Yarn, Cassandra and Flume, SAS interface configuration projects in direct client facing roles working towards ever changing business needs and deadlines.

SUMMARY

• Over 7+ years of experience in IT field including 4 years of experience in Hadoop Administration in diverse industries which includes hands on experience in Bigdata engineering and its related integrated technologies.
• Very good understanding and experience in Java and its exceptions.
• Extremely good and excellent at Bigdata Engineering and Hadoop support
• Experience with both versions of Hadoop 1.x and 2.x
• Hands on experience on major components in Hadoop Ecosystem like Hadoop Map Reduce, HDFS, HIVE, PIG, Hbase, Zookeeper, Sqoop, Oozie, Cassandra and Flume.Well versed in installation, configuration, supporting and managing of Big Data and underlying infrastructure of Hadoop Cluster.
• Understanding of Graceful Degradation, Progressive Enhancement, High Availability and Speculative Execution in Hadoop.
• Experience in architecting, designing, installation, configuration and management of Apache Hadoop Clusters in HDP Hortonworks, Map R and Cloudera Hadoop Distribution.
• Experience on Designing, Planning, Administration, Installation, Configuring, Troubleshooting, Performance monitoring and Fine - tuning of Cassandra cluster.
• Very good understanding/knowledge of Hadoop Administration and various components such as HDFS, JobTracker, TaskTracker, ResourceManager, Nodemanager, AppMaster, NameNode, DataNodes, Standby HA Namenode, and YARN MapReduce concepts.
• Good knowledge on Java and Python to trace the errors and java exceptions while trouble shooting for user application issues and providing good L1/L2 support through JIRA ticketing system
• Good knowledge about Cassandra read and writes paths and its internal architecture.
• Experience with query languages such as SQL, Pig and Hive.Spark SQL used for interactive queries and Hive for batch queries.
• Hands on experience in creating and upgrading Cassandra clusters
• Experience in developing and scheduling ETL workflows in Hadoop using Oozie. Also have substantial experience writing MapReduce jobs in Java, Pig, Flume and Hive and Storm.
• Capacity planning and performance tuning of Hadoop clusters. Extensive experience in SQL and NoSQL development
• Expert in Unix administration and have hands on with various OS environments RHEL, Cent OS and Ubuntu etc.
• Experience in web-based languages such as HTML, CSS, PHP, XML and other web methodologies including Web Services and SOAP.
• Expert working knowledge on databases like RDBMS (Oracle, MySQL, OLE DB and MS-SQL) & NoSQL.
• Planning and deployment of source code by building through MAVEN and push and commit in GIT hub.
• Proficient using OLAP cubes to design backend data models and entity relationship diagrams (ERDs) for star schemas, snowflake dimensions and fact tables and fact datasets.
• Experienced in using the distributed revision control and source code management system like GIT, TFS & SVN Tortoise.
• Involved in all phases of Software Development Life Cycle (SDLC) and Worked on all activities related to the development, implementation, administration and support for Hadoop.
• Worked in Agile, Waterfall & Scrum methodology with strong sense of ownership of software development.

TECHNICAL SKILLS

• HDP 2.6
• Pig 1
• Hive
• Mahout
• HDFS
• MapReduce
• Kerberos
• CDH 3.x 4.x
• Pig 10
• Hive
• Mahout
• HDFS
• MapReduce
• AWS
• MapReduce
• Hive
• EC2
• S3
• Java EE 5 6 7
• C/C++
• Python
• Perl
• Shell Scripting
• SAS
• Rscript
• Oracle
• MySQL
• PostgreSQL
• Red Hat 5 6
• Ubuntu 9 10 12
• Spring MVC
• HTML & CSS
• JavaScript
• Web Services
• SOAP
• REST

PROFESSIONAL EXPERIENCE

Confidential, Hartford, CT

Sr. Web Engineer

Responsibilities:

• Install and Manage HDP Hortonworks DL and DW components.
• Worked on Hadoop Hortonworks (HDP 2.6.0.2.2) distribution which managed services viz. HDFS, MapReduce2, Tez, Hive, Pig, Hbase, Sqoop, Flume, Spark, Ambari Metrics, Zookeeper, Falcon and oozie etc.) for 4 cluster ranges from LAB, DEV, QA to PROD contains nearly 350+ nodes with 7PB data.
• Monitor Hadoop cluster connectivity and security on Ambari monitoring system.
• Led the installation, configuration and deployment of product soft wares on new edge nodes that connect and contact Hadoop cluster for data acquisition.
• Rendered L1/L2 support services for BI users, Developers and Tableau team through Jira ticketing system.
• One of the key engineers in Aetna’s HDP web engineering team, Integrated Systems engineering ISE.
• Managed and reviewed Log files as a part of administration for troubleshooting purposes. Communicate and escalate issues appropriately for the tickets raised by users in JIRA ticketing system
• Worked closely with System Administrators, BI analysts, developers, and key business leaders to establish SLAs and acceptable performance metrics for the Hadoop as a service offering.
• Performance Tuning and ETL, Agile Software Deployment, Team Building & Leadership, Engineering Management.
• Hortonworks Ambari, Apache Hadoop on Redhat, and Centos as data storage, retrieval, and processing systems.
• Setting up Kerberos principals in KDC server and testing HDFS, Hive, Pig and MapReduce access for the new users and creating key tabs for service ID’s using keytab scripts.
• Performed a Major upgrade in production environment from HDP 2.3 to HDP 2.6. As an admin followed standard Back up policies to make sure the high availability of cluster.
• Monitored multiple Hadoopclusters environments using Ganglia and Nagios. Monitored workload, job performance and capacity planning using Ambari.
• Installed OLAP software Atscale on its designated edge node server.
• Implemented dual data center set up for all Cassandra cluster.Performed much complex system analysis in order to improve ETL performance, identified high critical batch jobs to prioritize.
• Conducted cluster sizing, tuning, and performance benchmarking on a multi-tenant OpenStack platform to achieve desired performance metrics.
• Good knowledge on providing solution to the users who encountered java exception and error problems while running the data models in SAS script and Rscript. Good understanding on forest data models.
• Worked on data ingestion on systems to pull data scooping from traditional RDBMS platforms such as Oracle, MySQL and Teradata to Hadoop cluster using automated ingestion scripts and also store data in NoSQL databases such as HBase, Cassandra.
• Provided security and authentication with ranger where ranger admin provides administration and user sync adds the new users to the cluster.
• Created and maintained Technical documentation for launching Hive server trouble shooting, Installation of product software on new edge nodes and also Hive queries and Pig Scriptsand kept in confluence for users’ reference.
• Responsible for Cluster maintenance, Monitoring, commissioning and decommissioning Data nodes, Troubleshooting, Cluster Planning, Manage and review data backups, Manage & review log files.
• Experienced on adding/installation of new components and removal of them through ambari on HDP and manually on EC2 clusters.
• Provided security and authentication with Kerberos which works on issuing Kerberos tickets to users.
• Good troubleshooting skills on over all Hadoop stack components, ETL services and Hue, Rstudio which provides GUI for developers/business users for day-to-day activities.
• Create queues and allocated the clusters resources to provide the priority for jobs in hive.
• Implementing the SFTP for the projects to transfer data SCP from External servers to servers. Experienced in managing and reviewing log files. Involved in scheduling Oozie workflow engine to run multiple Hive, sqoop and pig jobs.

Confidential, Farmington, Michigan

Consultant (Java with Hadoop)

Responsibilities:

• Researched on various Java exceptions and troubleshooting methods
• Worked on multiple projects spanning from Architecting Hadoop Clusters, Installation, Configuration and Management of Hadoop Cluster.
• Monitored Hadoop Infrastructure using Ambari on Hortonworks HDP Distribution.
• There are 2 different clusters another managed by Cloudera CDH3 distribution.
• L1/L2 Troubleshooting based on occurred java exceptions and errors while running apps.
• Managed Big Data and creating datasets using Pig and Hive query languages.
• Complete Big Data management and Application trouble shooting support.
• Maintenance of Hadoop Clusters using HDP Horton Works Distribution and also CDH3.
• Integrated Oozie with the rest of the Hadoop stack supporting several types of Hadoop jobs out of the box (like MapReduce, Pig, Hive, Sqoop) as well as system specific jobs.
• Developed entire data transfer model using Sqoop framework.
• Integrated Kafka with Flume in sand box Environment using Kafka source and Kafka sink.
• Configured flume agent with flume syslog source to receive the data from syslog servers.
• Implemented the Hadoop Name-node HA services to make the Hadoop services highly available.
• Exporting data from RDBMS to HIVE, HDFS and HIVE, HDFS to RDBMS by using SQOOP.
• Installed and managed multiple Hadoop clusters - Production, stage, development.
• Installed and managed production cluster of 150 Node cluster with 4+ PB.
• Performance tuning for infrastructure and Hadoop settings for optimal performance of jobs and their throughput.
• Worked with application teams to install operating system, Hadoopupdates, patches, version upgrades as required. Integrated Kafka with Flume in sand box Environment using Kafka source and Kafka sink.
• Involved in analysing system failures, identifying root causes, and recommended course of actions and lab clusters.
• Designed the Cluster tests before and after upgrades to validate the cluster status.
• Regular Maintenance of Commissioned/decommission nodes as disk failures occur using Cloudera Manager.
• Documented and prepared run books of systems processes and procedures for future references.
• Performed Benchmarking and performance tuning on the Hadoop infrastructure.
• Automated data loading between production and disaster recovery cluster.
• Helping users and teams with incidents related to administration and development.
• On boarding and training on best practices for new users who are migrated to our clusters.
• Guide users in development and work with developers closely for preparing a data lake.
• Log data Stored in HBase DB is processed and analyzed and then imported into Hive warehouse, which enabled end business analysts to write HQL queries.
• Created Hive external tables for loading the parse data using partitions.
• Developed various workflows using custom MapReduce, Pig, Hive and scheduled them using Oozie.
• Responsible for Installing, setup and Configuring Apache Kafka and Apache Zookeeper.
• Setup Cassandra with Apache Solrand Setup Cassandra With Apache Spark
• Experienced on adding/installation of new components and removal of them through ambari on HDP and manually on EC2 clusters.
• Worked as a lead on Big Data Integration and Analytics based on Hadoop, SOLR and web methods technologies.

Confidential

Hadoop/Java Consultant

Responsibilities:

• Install, configure, and upgrade Hadoop components in Cloudera platform CDH2.
• Troubleshoot cluster issues, and support developers running Map Reduce or Tez jobs in Pig and Hive.
• Proactively optimize and tune cluster configuration for performance.
• Organize, document, and improve process around data inventory management for 100+ TB of data.
• Monitor Hadoop cluster with Cloudera Manager, Nagios, and Ganglia.
• Upgraded the new version of Cloudera distribution CDH3 from existing CDH2 version.
• Manage cluster resources and multiple concurrent workloads to maintain availability and SLAs.
• Write Linux Shell scripts for automating tasks.
• Create scripted workflows and work with Pig/Hive developers to develop automated production ETL processes.
• Implement cluster backups and manage production data deployments between Data Centers.
• Implement and maintain security policies in the Hadoop & Linux environment.
• Research latest developments in the Hadoop open source platform and recommend solutions and improvements.
• Evaluate tools and technologies to improve cluster performance and ETL processing.
• Install, configure, and administrate Linux cloud based servers running in Amazon Web Services.
• Create documentation on cluster architecture, configuration, and best practices.
• Work with Hortonworks support team to investigate and resolve tickets for cluster or developer issues.
• Opportunity to contribute in Hadoop architecture, Hive ETL development, and data QA.
• Have worked on various distributions like Cloudera (PROD) and Map R for another cluster.
• Worked in implementing Hadoop with the AWS EC2 system using a few instances in gathering and analyzing data log files. Developed technical solutions to business problems.
• Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.
• Hands on experience installing, configuring, administering, debugging and troubleshooting Apache and Datastax Cassandra clusters.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs.
• Architecture design and implementation of deployment, configuration management, backup, and disaster recovery systems and procedures.
• Extensive knowledge in troubleshooting code related issues.
• Designed and coded application components in an agile environment utilizing test driven development approach.

Confidential

Data Analyst

Responsibilities:

• Manipulating, cleansing & processing data using Excel, Access and SQL.
• Processing and Analysing Geo Data using Java based GIS application “Atlas”
• Responsible for loading, extracting and validation of client data.
• Liaising with end users and 3rd party suppliers.
• Analyzing raw data, drawing conclusions & developing recommendations.
• Complete Unix/Linux environment involved with Unix commanding and also SQL querying.
• Quality controlling (reviewing) Geo Data labeller’s initial level works at QA Level.
• Data mining using applications and databases, Oracle, SAP and My SQL etc.
• Delegating with clients regarding work status every week through presentations and reports.
• Composing power point slides for trainers who train labellers and QC for Quality assurance at testing stage.
• Interacting with various departments involved in SDLC ( Trainers, Developers, BI analysts, Team Leaders, Testers )
• Using Informatica & SAS to extract transform & load source data from transaction systems.

Confidential

Unix / Linux Administrator

Responsibilities:

• Administration of RHEL which includes installation, testing, tuning, upgrading and troubleshooting both physical and virtual server issues.
• Managing Disk File Systems, Server Performance, Users Creation and granting file access Permissions.
• RPM and YUM package installations, patch and other server management.
• Responsible for handling the tickets raised by the end users which includes installation of packages, login issues, access issues
• User management like adding, modifying, deleting, grouping.
• Responsible for preventive maintenance of the servers on monthly basis.
• Configuration of the RAID for the servers and running Cron-tab to back up Data.
• Resource management using the Disk quotas.
• Documenting the issues on daily basis to the resolution portal.
• Responsible for change management release scheduled by service providers.
• Generating the weekly and monthly reports for the tickets that worked on and sending report to the management.
• Managing Systems operations with final accountability for smooth installation, networking, and operation, troubleshooting of hardware and software in LINUX environment.
• Identifying operational needs of various departments and developing customized software to enhance System's productivity.
• Creation of groups, adding User ID to a group as a primary or secondary group, removing User ID from a group.
• Established/implemented firewall rules, Validated rules with vulnerability scanning tools.
• Proactively detecting Computer Security violations, collecting evidence and presenting results to the management.
• Accomplished System/e-mail authentication using LDAP enterprise Database.
• Implemented a Database enabled Intranet web site using LINUX, Apache, MySQL Database backend.
• Trouble shooting User's login & home directory related issues.
• Monitoring System Metrics and logs for any problems.
• Applied Operating System updates, patches and configuration changes.
• Maintaining the MySQL server and Authentication to required users for Databases.
• Appropriately documented various Administrative & technical issues

Confidential

Free Lancer

Responsibilities:

• Freelance works using Confidential ’s AdWords and AdSense.
• Optimized client’s web sites using On-page and Off-page search engine optimization techniques.
• Created visual mock-ups and prototypes using HTML CSS and JavaScript for client’s web sites.
• Carried out research for websites on various SEO, SEM and Article spinning techniques.
• Optimized websites by doing keyword research and using them to write highly targeted content.
• Ensured that the Meta and Geo tags were well optimized to increase the chances of visibility on web.
• Proofread, edited and updated website content