Hadoop Administrator Resume
Irving, TX
SUMMARY
- Total 12 years of experience in Software analysis, design, development and maintenance in diversified areas of Client - Server, Distributed and embedded applications
- 2+ years of experience in all phases of Hadoop administration and HDFS development.
- Good in planning big data strategy and designing big data solution architecture, including data acquisition, storage, transformation, analysis, business intelligence and integration with other frameworks for tailor made solutions to meet specific business needs.
- Experience in developing Map Reduce Programs using Apache Hadoop for analyzing the big data as per the requirement.
- Experienced on major Hadoop ecosystem’s projects such as PIG, HIVE and HBASE.
- Good working experience using SQOOP to import data into HDFS from RDBMS and vice-versa
- Good knowledge in using job scheduling and monitoring tools like Oozie and ZooKeeper
- Experience in Hadoop administration activities such as installation and configuration of clusters using Apache and Cloudera
- Knowledge of administrative tasks such as installing Hadoop and configure its ecosystem components such as Hive and Pig
- Knowledge of NoSQL databases such as HBase
- In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, JobTracker, TaskTracker, NameNode, DataNode and MapReduce concepts
- Good understanding of Data Mining and Machine Learning techniques
- Experience in analyzing data using HiveQL, Pig Latin, and custom MapReduce programs in Java.
- Expert in planning big data strategy and designing big data solution architecture, including data acquisition, storage, transformation, analysis, business intelligence and integration with other frameworks for tailor made solutions to meet specific business needs.
- Experienced in porting cross platform applications from Windows to Linux and vice versa, meaning application translation from x86 to x64/AMD64 architecture.
- Experienced in automation, black box, white box, Functional non- functional, unit, modular and integration testing.
- Strong experience in Object Oriented Programming concepts.
- Strong experience in complete Project Life Cycle, which includes Requirements analysis, Feasibility Analysis, High Level Design, Low Level Design, Design Reviews, Coding, Testing, Peer Reviews, Implementation and Maintenance.
- Self-motivated and a good team player with excellent communication and interpersonal skills and also capable of working independently even under high-pressure challenges.
- Proven troubleshooter with strong debugging skills.
TECHNICAL SKILLS
Big Data Technologies: HDFS, MapReduce, Hive, Pig, Sqoop, Flume, HBase, Oozie, Zookeeper, Kerberos
Operating Systems: Windows XP,7,Vista and Unix/Linux
Languages: C, C++, VC++ MFC, Unix shell, RUBY Scripting, Java JDK1.4/1.5/1.6 (JDK 5/JDK 6), HTML,, PL/SQL
Source control: GIT, CVS, Clear Case, TFS
Technology: OOAD, OOPS, UML, XAML
Development Tools: Visual studio 6.0, VS 2008, VS 2010, TFS, Eclipse, GCC 4.2
Testing Tools: HP QC ALM, Rally, JIRA, Perforce, BUGZILLA,STAR team
Methods: WATERFALL, AGILE and MIXED
PROFESSIONAL EXPERIENCE
Confidential
Hadoop Administrator
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop
- Installed and configured Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster
- Setup and benchmarked Hadoop/HBase clusters for internal use
- Developed Simple to complex Map/reduce Jobs using Hive and Pig
- Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop
- Analyzed the data by performing Hive queries and running Pig scripts to study customer behavior
- Used UDF’s to implement business logic in Hadoop
- Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and other sources.
- Continuous monitoring and managing the Hadoop cluster using Cloudera Manager
- Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required
- Installed Oozie workflow engine to run multiple Hive and Pig jobs
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team
Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Hue, Oozie, Core Java, Eclipse, HBase, Flume Linux, C++, gcc4.2, git.
Confidential, Irving TX
Hadoop developer/Administrator
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop
- Installed and configured Hive, Pig, Sqoop, Flume and Oozie on the Hadoop cluster
- Setup and benchmarked Hadoop/HBase clusters for internal use
- Developed Simple to complex Map/reduce Jobs using Hive and Pig
- Optimized Map/Reduce Jobs to use HDFS efficiently by using various compression mechanisms
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop
- Analyzed the data by performing Hive queries and running Pig scripts to study customer behavior
- Used UDF’s to implement business logic in Hadoop
- Implemented business logic by writing UDFs in Java and used various UDFs from Piggybanks and other sources.
- Continuous monitoring and managing the Hadoop cluster using Cloudera Manager
- Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required
- Installed Oozie workflow engine to run multiple Hive and Pig jobs
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team
Environment: Linux, Windows, Visual studio 2010, C++ 11, ACE, C-Sharp
Confidential, MN
C-Sharp/Java and Managed C++ developer
Responsibilities:
- Assistance to the project team members in Coding, Unit test planning and execution.
- Tracking the status of the project using TFS project management.
- Carry out Quality assurance activities (reviews, walkthroughs etc.) as defined in the Project Plan.
- Carry out Product design and involve in changing algorithms.
- Researching and resolving production defects.
- Collect metrics and document maintenance activity identified tasks and solutions.
- Be responsible for the overall quality and timeliness of the deliverables.
- Recommend changes to increase efficiency, minimize operating expense or improve accuracy and timeliness of application systems.
Environment: Windows 7, C, C++, C# wpf, XAML, Core Java, Structs
Confidential
Automation expert and QA engineer
Responsibilities:
- Involved in writing automation scripts for integrated build.
- Assistance to the project team members in Coding, Unit test planning and execution.
- Involve in manual test case design and execution for acceptance.
- Set the functional and non functional test cases to improve the performance and power management of the device.
- IQMS aids in QA activities
- Carry out Production implementation.
- Researching and Resolving Production defects.
- Collect metrics and document maintenance activity identified tasks and solutions.
- Report progress to Scrum team on daily basis to address blockers issues over kanban to product owners, scrum owners, scrum masters and developers.
- Be responsible for the overall quality and timeliness of the deliverables.
- Recommend changes to increase efficiency, minimize operating expense or improve accuracy and timeliness of application systems.
Environment: Linux, C++, QT, RUBY, QC, Bugzilla
Confidential
Application porting expert
Responsibilities:
- Involved in Migrating the project from VC6 to VS 2008 and VS 2010.
- Designing Components in COM based objects.
- Recompile and rewrite the components which were developed using obsolete APIs.
- IQMS aids in QA activities
- Researching and Resolving Production defects.
- Be responsible for the overall quality and timeliness of the deliverables.
Environment: Windows XP, VS 2006, VS 2008, VS 2010, VC++ MFC, C++, Bugzilla
Confidential
VC++ Developer
Responsibilities:
- Involved in Design and development of the software.
- Coding and testing the components
- CVS has been used for source code control and BUGZILLA is used for QA activities.
- Researching and Resolving Production defects.
- Be responsible for the overall quality and timeliness of the deliverables.
Environment: Windows XP, VS 2006, VS 2008, VC++ MFC, Win 32 API, Gdi Sdk.
Confidential
Application Developer
Responsibilities:
- Have been part of POC, HLD and LLD.
- Involved in Design and development of the software.
- Coding and testing the components.
- CVS has been used for source code control and BUGZILLA is used for QA activities.
- Researching and Resolving production defects.
- Be responsible for the overall quality and timeliness of the deliverables.
Environment: Windows XP, VS 2006, VS 2008, VC++ MFC, Win 32 API, Gdi Sdk.
Confidential
Firmware Engineer
Responsibilities:
- Have been part of development team.
- Involved in assembly level programming on 8051 set of microcontrollers.
- Coding and testing the components.
- Interface memory modules using I2C protocols.
- Improve firmware performance by migrating to new age microcontrollers such as PIC.
- Be responsible for the overall quality and customer support.
Environment: Windows XP, Assembly language 8051. i2c, eeprom, C, Watchdog
