Hadoop Developer Resume
Northridge, CA
SUMMARY
- Over 8 years of experience in application development and strong background in computational sciences in modelling / simulation of molecules
- Extensive experience in complex data analyses in computational software packages
- Developed customized tutorials and protocols for computational software packages
- Broad experience in writing, reviewing and editing scientific manuscripts
- Over 3 years of experience in big data domain (Cloudera, HortonWorks and MapR distribution)
- Extensive experience in building and managing multi tenant hadoop clusters that spans across data centers, designing and implementing security (Kerberos, Hive Authentication), Data Access control (design group policy for file system access) managing space quotas, performance optimization, backups and recovery
- Extensive experience in writing shell scripts for performing various monitoring and administrative operations
- Extensive experience in preparing hardware requirements for building clusters and large - scale big data platforms using various hadoop distributions
- Experienced in building hadoop/eco system release using source, apply patches, evaluate new builds
- Experienced in dealing with Vendor to resolve product related issues, evaluate new releases and recommend technologies to end client
- Created Hadoop working group with documentation and demos to mentor J2EE Java programmers
- Experienced in designing, developing and executing test plan for small and large clusters
- Experienced in HDFS, Flume, MapReduce, Yarn, Hive, HiveServer2, Oozie, Hue, Sqoop, ZooKeeper, HBase, Pig, Cloudera Navigator and Mahout
- Experienced in setting up standards and processes for Hadoop based applications design and implementations
- Exposure to developing UDFs, Custom MapReduce application development using Java
- Experienced in NOSQL databases (HBase)
TECHNICAL SKILLS
Methodologies: Agile Scrum, Waterfall, Design patterns
Big Data Technologies: PIG, HIVE, HBASE, SQOOP, Hadoop, Map Reduce, HDFS, HortonWorks
Languages: PIG Latin, NOSQL, JAVA, XML, XSL, SQL, PL/SQL, HTML, JavaScript, VB.NET, C, C++, COBOL, CICS, IMS, JCL
J2EE Technologies: JSP, Servlet, Spring, Hibernate, Web services
Web Technologies: HTML, CSS, JavaScript, AJAX, ASP.NET
Databases: HDFS, DB2, ORACLE and SQL server
Application Servers: WebSphere, WebSphere Portal Server, Weblogic, Tomcat, Apache, Web Aware
Operating Systems: WINDOWS, OS 7, 8, UNIX and DOS
PROFESSIONAL EXPERIENCE
Confidential, Northridge, CA
Hadoop Developer
Responsibilities:
- Interacting with other data scientists and developing custom solutions to process data using Map-reduce in Hadoop.
- Scripting in R for Linguistic analysis or Text mining.
- Working with experts in fields of Human factors, Market strategists and program managers to develop the POC for usage of Hadoop in Confidential ’s Diabetes section.
- Also extract device related JSON data for statistical market performance.
- Maintain Hadoop, Hadoop ecosystems, third party software, and database(s) with updates/upgrades, performance tuning and monitoring
- Responsible in modification of API packages
- Well exposed of accessing massive volumes of client data
- Implemented pattern detection in customer complaint text data.
- Prepare Multi-cluster test harness to exercise the system for performance and failover
Environment: Spotfire, Mahout, RHadoop & Hadoop, MapReduce, Java (JDK 1.6), SAP BO
Confidential, Danbury, CT
Hadoop Administrator/ Developer
Responsibilities:
- Plan data ingestion and integration process from the EDW env into a Datalake in HDFS
- Install, Configure, and Administer Pivotal Hadoop distribution 1.0.1 for both Dev and Prod Clusters
- Upgrade Pivotal Hadoop distribution from 1.0.1 to 1.1.1 and then to 2.0.1 stable version
- Optimize Hadoop performance by reconfiguring Yarn RM for increased heap memories
- Configure Namenode High Availability for Namenode failovers
- Configure Pivotal HAWQ database for Namenode failovers
- Support/Troubleshoot jobs or Mapreduce programs running in the Production cluster
- Resolve issues, answer questions, and provide support for users or clients on a day to day basis related to Hadoop and its ecosystem including the HAWQ database
- Upload data from Linux/UNIX file system into HDFS for data manipulation in HAWQ and Hadoop
- Install, configure, and operate Apache stack i.e. Hive, HBase, Pig, Sqoop, Zookeeper, & Mahout
- Install, configure, and operate data integration and analytic tools i.e. Informatica, Chorus, SQLFire, & GemFireXD for business needs
- Create tables, load data, and write queries/UDFs for jobs running in Hive
- Create Pig UDFs for jobs running in the Production environment
- Develop scripts to automate routine DBA tasks (i.e. refresh, backups, vacuuming, etc.)
- Develop CRON jobs in Linux (RHEL) to monitor the system and system resources
- Maintain Hadoop, Hadoop ecosystems, third party software, and database(s) with updates/upgrades, performance tuning and monitoring
- Manage/implement LFS and HDFS security using tools i.e. PBIS and Kerberos authentication protocol
- Configure IDM Access and Disaster Recovery for HDFS using tools i.e. Distcp, and WANDisco
Environment: Pivotal Hadoop (HDFS), Mapreduce, AWS, Hive, Java (JDK 1.6), Flat/XML/JSON Files, PSQL, Linux/UNIX Shell Scripting
Confidential - Dallas, TX
Big data Administrator/Developer
Responsibilities:
- Hadoop installation, configuration of multiple nodes in AWS-EC2 & Hortonworks platform
- Setup and optimize Standalone-System/Pseudo-Distributed/Distributed Clusters
- Build/Tune/Maintain Hive QL and Pig Scripts for reporting purpose
- Help develop mapreduce programs and define job flows
- Manage and review Hadoop log files
- Support/Troubleshoot mapreduce programs running on the cluster
- Load data from Linux/UNIX file system into HDFS
- Install and configure Hive and write Hive UDFs
- Create tables, load data, and write queries in Hive
- Develop scripts to automate routine DBA tasks using Linux/UNIX Shell Scripts/Python (i.e. database refresh, backups, monitoring etc.)
- Tune/Modify SQL for batch and online processes
- Monitor cluster using Ambari and optimize system based on job performance and criteria
- Manage cluster through performance tuning and enhancements
Environment: Hortonworks Hadoop (HDFS), MapReduce, AWS, Hive, Java (JDK 1.6), Flat/XML/JSON Files, PSQL, Linux/UNIX Shell Scripting
Confidential, Boston, MA
Java Developer
Responsibilities:
- Perform architectural, design, and development work on a variety of projects involving Data Visualization, Business Intelligence, Real-Time Simulations, and / or Marketing
- R & D: Perform experiments testing the capabilities of various RIA technologies
- Mentor team members in Flex development and the Cairngorm framework
- Architect and Developed a complete near-real-time message management system with a Flex 2.0 (Cairngorm) UI, JSP (REST) middle tier running in Tomcat, and Hibernate / Oracle providing data
- Developed a flex UI for a network messaging application.
- Designed and developed interactive Flex 2.0 world map event-monitoring UI and integrated it with an existing Jason application to display near-real-time data
- Developed Flex 2.0 UI for a network monitoring system that interacts with a Java / Servlets middle tier via REST
Environment: J2EE, JSP, JSTL, Custom Tags, CSS, Java Script, Oracle 8.X, PL/SQL, Windows 2000
Confidential, IA
Research Associate
Responsibilities:
- Developed algorithms and codes for advanced scientific software to perform numerical simulations (C++, Fortran)
- Designed and implemented numerical simulations to solve challenging scientific problems in molecular modeling and computational chemistry
- Performed complex data analyses using Matlab, VMD, GROMACS
- Performed large-scale simulations on clusters, supercomputers and GPU accelerated clusters
- Experience in scripting, file handling (Shell, Perl, grep, sed/awk) and Unix/Linux command line
- Tested/benchmarked scientific application software packages on different hardware and software platforms
- Configured, compiled and installed software packages on different clusters for large-scale simulations
