Hadoop Consultant Resume
San Jose, CA
OBJECTIVE:
- To obtain dynamic and challenging opportunity that contributes to the outstanding success of the business and continuous learning opportunities that would provide ample bandwidth to hone my technical and project management skills so that I could contribute positively to my organization while moving up the corporate ladder.
PROFESSIONAL SUMMARY:
- A proactive, result oriented and PMP® certified project lead with over 12 years of IT experience including Hadoop, System Programming, Client - Server architecture, Parallel processing, High availability and Network programming using C, C++ and ‘open source’ development platforms on Linux software distributions. Working knowledge of Grid/Cluster/Cloud computing/Big data technologies andexperience in project management activities.
- Hands on experience using Hadoop components - Map Reduce, Hive, Pig, Hbase, ZooKeeper
- Hands on experience on operating system internals in multithreaded environment using Inter Process Communications and deep knowledge of UNIX operating system internals and working knowledge of Linux software distributions and High Performance Computing (HPC).
- Knowledge of scripting languages and Relational Databases.
- Experienced in leading teams that worked on Confidential /Cloud Computing.
- Worked on all aspects of software development process with thorough knowledge of the functional system decomposition and object oriented design and designed complex technical solutions from proposal through implementation and go-live.
- Responsive to change in a fast moving, high technology product development environment.
- Worked on Hardware Platform Interface (HPI) and Application Interface Service (AIS) for Service Availability Forum (SAF). Working experience on high availability middleware using ATCA platforms based on Chassis architecture.
- Experience in Software Configuration Tools (SCM) and complete SDLC.
- Knowledge in load balancing and performance monitoring and fine tuning of parallel applications using Message Passing Interface (MPI).
- Proficiency in using and contributing to open source technologies and related developer communities.
- Represented affiliated organizations in developer conferences, events and engaged with various developer communities.
- Expertise in debugging and analyzing Memory dumps and Core file analysis.
- Tuned application performance aspects including OS, Network and database and provided recommendations to improve coding efficiency.
- Good project management abilities with excellent written and spoken communication skills.
TECHNICAL SKILLS:
Operating Systems: LINUX, UNIX (Aix, Solaris) and Windows Vista/XP
Languages: C, C++, Java, XML, c#, Fortran
Protocols: TCP/IP, UDP, SNMP and ftp
Version Control: Rational Clear Case, CVS
Project Management Tool: MS Project
Build Tools: Make, Gmake
Memory leak detection tools: Purify, Valgrind
Scripting Languages: Perl, shell programming
Database: RDBMS- MySQL, Oracle 9i
Debugging tool: GDB, DBX
Web service/Grid Service tools: Web sphere, SOAP,WSDL, Globus 2.4 and 4.0
Performance monitoring tool: Gprof
GUI Programming: Xwindows
PROFESSIONAL EXPERIENCE:
Confidential, San Jose, CA
Hadoop Consultant
Responsibilities:
- Responsible for coordinating end to end project management related activities.
- Involved in Design and Development of technical specification documents using Hadoop
- Developed MapReduce programs to parse the raw data, populate tables and store the refined data in partitioned tables in the MMIS. Managed and reviewed Hadoop log files.
- Developed and written Apache PIG scripts and HIVE scripts to process the HDFS data.
- Monitored Hadoop scripts which take the input from HDFS and load the data into Hive.
- Migrated the needed data from Oracle, MySQL in to HDFS using Sqoop and imported various formats of flat files in to HDFS.
- Defined job work flows as per their dependencies in Oozie.
- Maintain System integrity of all sub-components related to Hadoop.
- Worked on Apache and Cloudera's Hadoop clusters.
Environment: Apache Hadoop, HDFS, Hive, Map Reduce, Java, Eclipse, Hive, Pig, Sqoop, Flume, Oozie, MySQL, Hadoop Distribution of Cloudera.
Confidential, San Jose
Lead Consultant
Responsibilities:
- Lead Hadoop developer for core tools group. Setting up the Hadoop Cluster and developing the map reduce applications to handle the data from the library definition files. Architected Big data solutions to handle millions of records.
- Developed, maintained and tested core technology software modules for cadence tools group in c and c++. Supported platforms using Linux, Solaris and IBM-AIX.
- Improved the software quality by proactively identifying and fixing purify and valgrind bugs.
- Plug-in software development: Specified and implemented key plug-in for Cadence library definition files.
- Wrote test cases and implemented a testing software suite. Provided migration path for Cadence applications to adopt this new plug-in.
- Design library management software: Lead developer for a critical high-visibility project involving migration of Integrated Circuit design library management software to the Cadence next generation design database (Open Access). Consistently completed milestones as per schedule; delivered successful code on schedule.
- Software development using c, c++, Mysql on Linux, Solaris and AIX in the domain of high availability.
Confidential, Arlington Heights, IL
Lead Consultant
Responsibilities:
- Designed this to be multi-threaded server, to accept client request using a predefined TCP/IP socket and port. The call- processing payload cards triggers events on the server side by sending IP data to the Proxy server’s listening socket. Also on the server side, events like data value change, alarms raised, critical error reported etc would trigger callbacks, which the Proxy Component receives and sends out the info to the play load cards through IP data packet.
- Designed and implemented with the capacity to handle high volume information transfer by taking full advantage of HAP’s high availability and to handle multiple play load cards requests.
Confidential
Team lead
Responsibilities:
- Technical Team lead of software engineers who work in grid environment for national Confidential initiative project (GARUDA). Work includes - Building grid infrastructure and collaboration in Grid GARUDA.
- Developing and porting application using c /c++ on Param YUVA mainly in providing high availability.
Confidential
Senior Faculty
Responsibilities:
- Developed the modules for high availability middleware as per Service Availability Specifications (SAF) release
- Optimized the existing code which resulted in 30+% improvements in terms of memory usage and recovery time and architected the new design
- Participated in overall review activity - design, code and test plan
Confidential
Sr. Technical Lead
Responsibilities:
- In charge of implementation of project management activities and mentoring the team at lead level
- Developed the modules for high availability middleware as per Service Availability Specifications (SAF) release
- Optimized the existing code which resulted in 30+% improvements in terms of memory usage and recovery time and architected the new design
- Participated in overall review activity - design, code and test plan
- Involved in the Low Level Design and coding for Information Management Module(IMM) of High availability middleware- This involved in developing the APIs for fetching the object information and class details and corresponding server side implementation to fetch the data from object management system.
- Module Lead for the IMM from inception to successful delivery and implementation at the client site.
- Provided functional and technical specs for adaptation of Open HPI to the team/organization
- Project estimation, requirement analysis and design and Project execution for Integrating Open Hardware Platform Interface (HPI) with ATCA platform using chassis based architecture using IPMI(Intelligent Platform Interface
- Enhancements to Open HPI to integrate it with Service Availability middleware
Confidential
Project Lead
Responsibilities:
- Single point of contact from Confidential in collaboration with SAC Ahmadabad for setting up the computational grid between Confidential Bangalore and SAC
- Led a team of members working on WFMS and comprehensive study of various workflow systems that could be suitable for grid Garuda setup.
- Deployment and stabilization of Grid Garuda setup at Bangalore Centre. The task was to setting up a computational grid at national level by connecting 17 R&D centers across the country and providing seamless computational power to the HPCC Community
- Initiated the project JSAS (Job Submission and Accounting Software) with first level elicitation report and SRS. In charge of Project initiation, planning, execution and closure.
- Lead the Design, coding and code Reviews for this Product’s front end and back end. Front end of this product is a portal for submitting the jobs to Param Padma remotely and backend module is focused on accounting and billing software
- Embedded the JSAS Sys Admin tool to help PARAM Padma System Administrators to perform day-to-day system administrative responsibilities
- In charge of successful delivery of this product end to end
- Enhancements of the features based on customer inputs
Confidential
Developer
Responsibilities:
- Developed and maintained cdb (component debugger part) a task level debugger utility to the end user to debug the parallel application) part of DIViA (Debugger Integrated Visualizer and Analyzer), an essential part of Confidential ’s HPCC suite for parallel applications.design reviews, code reviews and test plan reviews
- Ported ParViD (Parallel Visualizer Debugger) from Linux to Aix5.1 in PARAM PADMA super computer and stabilization of the software
- Development of wrapper part for ParViD and non-blocking MPI support
- Involved in debugging the software to solve the potential MPI related issues.
- Involved in design and coding of Confidential ’s parallel visualizer and debugger tool ParVid and developed parallel applications using Message passing Interface (MPI) on platforms with emphasis on high bandwidth and low latency.
Confidential
Responsibilities:
- Involved in coding and enhancements of MARK-5, The Confidential ’s 5th generation Supercomputer, using GCM-T-80 code.
- Development of wrapper part for ParViD and non-blocking MPI support
- Involved in debugging the software to solve the potential MPI related issues.
- Involved in design and coding of Confidential ’s parallel visualizer and debugger tool