Sr. Hadoop Admin\developer Resume
Piscataway, NJ
SUMMARY
- 12 plus years of experience in Information Technology in all phases of SDLC.
- Around 2 years of experience in Big Data processing using Apache Hadoop.
- Experienced in distributed systems to leverage Hadoop/Big Data, HDFS, MapReduce, analyzed performance bottlenecks and recommended tuning optimizations.
- Proficient in Apache Hadoop ecosystems PIG Scripts, Python UDF’s, Talend, FLUME, HBase, Zookeeper, Hive, SQOOP, strong understanding of HDFS architecture.
- Deployed, Managed and administered Hadoop clusters to maintain distributions.
- Having working knowledge on Hadoop Administration.
- Worked in Windows, UNIX/Linux platform with different Technologies such as BigData, SQL, PL/SQL, XML, HTML, Core Java, Python etc.
- Knowledge of creating Map Reduce codes in Java as per the business requirements.
- Good experience in Oozie Framework and Automating daily import jobs.
- Hands on work experience in writing application on No SQL database like Cassandra.
- Worked in ETL tools like Talend to simplify mapreduce jobs from the front end.
- Worked with Tableau for report creation and further analysis from the front end.
- Knowledge of Pentaho report creation and analysis.
- Experience installing JBoss and Tomcat Apache server in Unix environment
- Experience with application/web servers includes BEA Weblogic, IBM WebSphere, and Jboss. Tomcat.
- Experience in SQL, PL/SQL and Java/JDBC using Oracle 8i & 9i.
- Extensive experience in Full Software Development Life Cycle including Analysis, Design, Development, Deployment and Maintenance of web applications.
- Experience writing unit and integration test cases.
TECHNICAL SKILLS
Languages: Java 1.4/1.5/1.6/1.7, C, C++, SQL, XML, PL/SQL,PERL, VB
BigData Ecosystem: HDFS, PIG, MAPREDUCE, HIVE, SQOOP, HBase
Java Technologies: Core Java, Servlets, JSP, PHP, Java Beans, Hibernate, Spring
Scripting: JavaScript, Perl Scripting, Unix Shell Scripting, ANT.
Technologies: Hive, Pig, MapReduce, Hbase, Sqoop, Java, SQL, C, C++, Python AJAX, HTML, HTML5, CSS, DHTML, Java Script, JSON.
Web Servers: Web Logic 8.1/9.2/10, Tomcat 5.5 /6.0.
Enterprise Servers: BEA's WEB Logic 10.0/9.1/8.0, WebSphere7 (WAS)
Databases: Oracle 10g/ 9i, MS SQL Server2000/03/05,MongoDB,CouchDB and DB2
Version Control: CVS, SVN, Microsoft VSS, Clear Case
Operating Systems: UNIX, Solaris Windows 03/08, NT/XP, Vista, Mac.
Testing Tools: Mercury Test Director 7.x, Mercury Load Runner 8.1
Reporting Tools: Talend, Tableau, Knowledge of Pentaho, OBIEE
Tools: Maven, ANT, SQL*Loader, SQL Developer, TOAD, PL/SQL Developer, Matlab, Visual Studio, Apache Tomcat, Putty, WinSCP, Autosys
IDE: My Eclipse, Eclipse, NetBeans and Sun Studio IDE
PROFESSIONAL EXPERIENCE
Confidential, Piscataway, NJ
Sr. Hadoop Admin\Developer
Responsibilities:
- Installed and configured Hadoop Map Reduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
- Experience in installing, configuring and using Hadoop Ecosystem components.
- Experience in Importing and exporting data into HDFS and Hive using Sqoop.
- Experienced in defining job flows.
- Experienced in managing and reviewing Hadoop log files.
- Participated in development/implementation of Cloudera Hadoop environment.
- Load and transform large sets of structured, semi structured and unstructured data.
- Experience in working with various kinds of data sources such as Oracle.
- Successfully loaded files to Hive and HDFS from Oracle.
- Responsible for managing data coming from different sources.
- Gained good experience with NOSQL database.
- Supported Map Reduce Programs those are running on the cluster.
- Involved in loading data from UNIX file system to HDFS.
- Installed and configured Hive and also written Hive UDFs.
- Involved in creating Hive tables, loading with data and writing hive queries, which will run internally in map, reduce way.
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
Environment: Cloudera HDFS, Hadoop, MapReduce, Hive, Hive UDF Pig Latin, Java, SQL, Sqoop, Centos, NOSQL database.
Confidential, Bothell, WA
Hadoop Admin\Developer
Responsibilities:
- Installed and configured Hadoop MapReduce, HDFS and developed multiple MapReduce jobs in Java for data cleansing and preprocessing.
- Involved in loading data to HDFS and Hive using Sqoop.
- Installed Oozie workflow engine to run multiple Hive and Pig.
- Responsible for building scalable distributed data solutions using Hadoop.
- Handled importing of data from various data sources, performed transformations using Hive, Map - Reduce, and loaded data into HDFS.
- Loaded and transformed large sets of structured, semi structured and unstructured data.
- Installed and configured Pig and Hive.
- Created UDFs in Pig and Hive as well as scripts.
- Evaluated business requirements and prepared detailed specifications that follow project guidelines required to develop written programs.
- Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
Environment: Hadoop, MapReduce, HDFS, Pig, Hive, HBase, Oozie, Java (jdk1.6), Eclipse, Cloudera, Sqoop, Pig, Oracle 10g
Confidential, Santa Clara, CA
Senior Software Engineer
Responsibilities:
- Extensively involved in gathering business requirements and system requirements and specifications.
- Used UML to create use cases, use case diagrams, classes & sequence diagrams.
- Develop web application using Spring Framework.
- Responsible for design and development of Web Application using Struts Framework and written Action Classes, Form Bean Classes and configure the Application using Struts Configuration file and implementation of log4j.
- Involve in Initial designing and creating Use case diagrams, Sequence Diagrams and class diagrams using the MS Visio Tool.
- Develop mapping using orm relationship with JPA Entity Manger.
- Developed Message Driven Beans as Queue listeners for JMS queues
- Developed DAO design pattern for hiding the access to data source objects.
- Designed the Web interface using JSP (including JSP Tag Libraries), HTML, DHTML, CSS and JavaScript, jQuery based on Struts and Tiles Tag Libraries Framework.
- Used AJAX to improve the application performance by getting the data from the server without refreshing the page on client side.
- Built XML based data exchanges and used DOM for parsing and XML creation out of Java code.
Environment: Java, J2EE, EJB 1.1, JSF, XML, JDBC, Oracle 9i, Log4J 1.2.,PL/SQL Developer, REST framework, Spring framework.
Confidential, San Francisco, CA
Senior Software Engineer
Responsibilities:
- Responsible for design and development of Web Application using Struts Framework and written Action Classes, Form Bean Classes and configure the Application using Struts Configuration file and implementation of log4j.
- Involve in Initial designing and creating Use case diagrams, Sequence Diagrams and class diagrams using the MS Visio Tool.
- Develop web application using Spring Framework.
- Used JavaScript and struts validation framework for performing front end validations.
- Develop DAO design pattern for hiding the access to data source objects.
- Use Hibernate framework for the backend persistence
- Developed DAO design pattern for hiding the access to data source objects.
- Reviewing the PL/SQL scripts of the team before it is deployed on live.
- Worked on Servlets, JSP, JDBC and Java script under MVC Architecture and implemented OOAD concept in the applications.
- Responsible for updating the supplier database if new updates are available. Configured and Implemented Web Services Security - Message-Level Security
- Developed persistence layer for the web services and integrated modules
Environment: Java, J2EE, EJB 1.1, JSF, XML, JDBC, Oracle 9i, Log4J 1.2.,PL/SQL Developer, REST framework, Spring framework.
Confidential, Santa Clara, CA
Senior Software Engineer
Responsibilities:
- Participate in project planning sessions with business analysts and team members to analyze business IT Requirements and translate business requirements into working model.
- Involve in Initial designing and creating Use case diagrams, Sequence Diagrams and class diagrams using the MS Visio Tool.
- Develop Java application using Java Spring framework.
- Used JavaScript and struts validation framework for performing front end validations.
- Develop DAO design pattern for hiding the access to data source objects.
- Use Hibernate framework for the backend persistence
- Use Eclipse as IDE tool to develop the application and JIRA for bug and issue tracking
- Use CVS for software configuration management and version control
- Optimized and modified the triggers, complex store functions, procedures, user base data type etc.
- Responsible for the dealing with the problems, bug fixing and troubleshooting.
- Developing Oracle PL/SQL stored procedures, Functions, Packages, SQL scripts to facilitate the functionality for various modules.
Environment: Java, J2EE, EJB 1.1, JSF, XML, JDBC, Oracle 9i, Log4J 1.2.,PL/SQL Developer, REST framework, Spring framework.
Confidential, Milwaukee, WI
Senior Software Engineer
Responsibilities:
- Prepare Functional Design Document & Technical Design Document to meet the Business requirements
- Code Reviews, customization and maintenance of code across applications.
- Developed and deployed Servlets and JSP are on Tomcat server.
- Responsible for release of code into production and its integration with other Applications.
- Creating environment for user-acceptance testing and facilitating Integration and User Acceptance Testing with JUNIT, fitness Tests.
- Developed customized client-side modules (Java Servlets, HTML, Apache)
- Installation, configuration of Oracle server software and related products.
- Evaluate Oracle new features and oracle related products.
- Responsible for developing and maintaining all the entity and session beans.
- Take care of Database design and Implementation.(Database creation / configuration, maintenance of database and application level integrity)
- Implement and maintain database security. (create / maintain users /profiles /roles, assign/revoke privileges)
- Tuned Report performance by exploiting the Oracle’s new built-in functions and rewriting SQL statements
- Participated in on call support 24/7 with offshore team.
Environment: Java, J2EE, EJB 1.1, JSF, XML, JDBC, Oracle 9i, Log4J 1.2.,PL/SQL Developer, Rest framework, Spring framework.
Confidential, CT
Senior Software Engineer
Responsibilities:
- Responsible for the analysis, designing and coding of the database
- Designed technical specifications and implemented business components for the site (Java, JDBC, Applets, JavaScript, HTML, ASP, IIS 3.0 - 4.0)
- Created database triggers and managed the subprograms and triggers.
- Provide ad hoc reports on request from users and Business.
- Keep track on backend objects as per the coding standards.
- Responsible for the analysis, designing and coding of the database.
- Reviewed security around the database by monitoring the Critical Roles of the Oracle database.
- Work as part of team and provide DBA support as and when required.
- Trained managers in using the system, using reports and writing and executing complex queries.
- Developed procedures, functions and packages to implements the system’s business logic using.
- Tuning of problematic queries for performance boost.
- Perform general technical trouble shooting and give consultation to development teams.
Environment: Java, J2EE, EJB 1.1, JSF, XML, JDBC, Oracle 9i, Log4J 1.2.,PL/SQL Developer, REST framework, Spring framework.
Confidential, FL
Senior Software Engineer
Responsibilities:
- Prepare Functional Design Document & Technical Design Document to meet the Business requirements.
- Implemented business objects and business logic of the system (Java).
- Responsibilities implementing database objects (Package, procedures, functions, triggers)
- Creating data mappings and data dictionaries.
- Understanding business data relationships.
- Wrote Complex SQL queries and PL/SQL
- Prepare Batch scripts using PL/SQL procedures for transferring data from Siebel System to external legacy systems
- Performance tuning of the Siebel database.
- Created pl/sql procedures and functions for pulling data to staging table.
- Check log files, creating and schedule component requests whenever the need aroused.
Environment: Java, J2EE, EJB 1.1, JSF, XML, JDBC, Oracle 9i, Log4J 1.2.,PL/SQL Developer, REST Framework, Spring Framework.
Confidential, LA
Senior Software Engineer
Responsibilities:
- Responsible for database administration and management by using knowledge ofPL SQL programming.
- Designed technical specifications and implemented business components for the site (Java, JDBC, Applets, JavaScript, HTML, ASP, IIS 3.0 - 4.0)
- Mainly responsible for communicating with clients and understanding their business requirements to develop applications.
- Handling other database servers like Oracle, My SQL
- Convert Functional Designs into Technical Design documents.
- Involved in SQL scripts development, DBA tasks, schema management.
- Perform Unit testing in Mercury test director to test the Siebel application.
Environment: Java, J2EE, EJB 1.1, JSF, XML, JDBC, Oracle 9i, Log4J 1.2.,PL/SQL Developer
Confidential
Programmer Analyst
Responsibilities:
- Design and Implementation of Data Warehouse/Data Mart, Gathering Business User Requirements, System Analysis, Design and Data Modeling.
- Involved in the analysis, design, coding of Database system specific to ETL/Business Intelligence practice.
- User requirement gathering and business user discussions to identify scope and data to provide solutions in Health Care practice for claims management.
- Review the technical architecture, present the client with the technical presentation for project sign off.
- Developed mappings and workflow for loading processed claims and provider data into staging schemas and to the data warehouse.
- Attend operations meeting with senior IT and business users for proposal and implementation of upgrade and patches on servers and follow up on system down times and availability.
- Develop and unit test UNIX scripts for pre and post loading processes.
- Extracted data from multiple databases and presented to the management in the form of Excel Sheets.
- Developing SQL Scripts, Stored Procedures.
- Spooling & Uploading the generated reports to Excel files.
Environment: Java, PL/SQL Developer, Informatica power center 8, Oracle.