Hadoop Developer Resume
Boston, MA
SUMMARY
- MapR Certified with 7+ years of experience in full life cycle development of complex commercial applications using Hadoop, Object - oriented Java and J2EE technologies.
- Excellent knowledge on Hadoop architecture and eco system components such as HDFS, MapReduce, Yarn, Hive, Pig, Sqoop, Flume and Oozie.
- Excellent knowledge on Cloudera, MapR and Hortonworks distributions
- Provisioning, installing, configu ring, monitoring, and maintaining HDFS, Pig,Hive, Spark, HBase, Yarn, Flume, Kafka, Sqoop, Oozie, Hue.
- Processing large sets of structured, semi-structured and unstructured data and supporting systems application architecture. Moving data from HDFS to RDBMS and vice-versa using Sqoop.
- Collecting large amounts of log data using Apache Flume and aggregating using Pig and Hive in HDFS for further analysis.
- Very good understanding of Partitioning, Bucketing concepts in Hive and designed both Managed and External Tables in Hive to optimize performance.
- Developed UDF’s in Java as and when necessary to use in Pig and Hive queries. Developed MapReduce jobs in Java for data cleaning and preprocessing.
- Basic knowledge on Hadoop Yarn, Apache Zookeeper and Spark and Storm.
- Experience in development methodologies such as Agile, Scrum, BDD, Continuous Integration and Waterfall.
- Hands on experience with SQL on Oracle10g and MySQL Data Bases.
- Proficient in Java/J2EE design patterns including Singleton, Model View Controller (MVC), Data Access Object (DAO), Factory andAOP in the development of web based & distributed enterprise applications.
- Expertise in developing applications using Java and J2EE technologies like Servlets, JSP, JDBC using tools like Eclipse.
- Expertise with Junit framework for unit testing to evaluate the application functionality.
- Expertise in Object Oriented Analysis and Design (OOAD) and Object Oriented Paradigm (OOP) from designing UML diagram using IBM Rational Rose, Microsoft Visio tools to development phase.
- Integration and deployment of applications done using ANT and Maven.
- Exposure to high availability configurations, Hadoop cluster connectivity and tuning, and Hadoop security configurations.
- Familiar with ServiceNow, Jira, MS Share Point and Quality Center.
- Exposure to all stages of SDLC, 24X7 support environment, customer services.
TECHNICAL SKILLS
Languages: C,C++, Java, J2EE, Scala, Python
Web Technologies: JavaScript, CSS, AJAX,HTML5, JDBC, JSP
Frameworks: Spring, Hibernate, MVC Spring
Application Servers: Web Logic, Web Sphere, TOMCAT
IDE/GUI Tools: Eclipse, IntelliJ
Operating Systems: Windows, UNIX
Databases: Oracle, MySQL,HBase, Cassandra
Tools: Git, SVN, Clear Case, Rational Clear Quest, ServiceNow, Jira, Quality Centre, UML, Tableau
Unit Testing Framework: Junit, MRUnit
Hadoop: HDFS, Map Reduce,Hive,Spark, Pig, Sqoop, Flume,Kafka, Hue, Zoo Keeper, Impala
PROFESSIONAL EXPERIENCE
Confidential, Boston, MAHadoop Developer
Responsibilities:
- Worked with engineering Leads to strategize and develop data flow solutions using Hadoop, Hive and Javain order to address long-term technical and business needs.
- Involved in all phases of SDLC, from gathering requirements from business users, designing the structure of the application, UML design, testing and deployment and maintenance of the application.
- Involved in building the Hadoop cluster environment and its ecosystem components.
- Configured the properties of the cluster according to the application requirements.
- Developed map reduce programs to perform multiple adhoc requests.
- Developed map reduce programs to pre-process the data, structuring the unstructured data and filtering the raw data.
- Developed map reduce programs in Java to get the metrics of the application.
- Performed the unit testing on the MapReduce jobs by writing MRUnit test cases.
- Created partitioned, external and managed Hive tables to store the structured data produced from map reduce programs using pre-defined and custom defined SerDe.
- Imported the data from SQL to Hadoop/HDFS using SQOOP.
- Developed Hive UDF in Java to meet application requirements.
- Developed Hive queries to perform analytics on the data stored in Hive tables.
- Improved the hive queries performance by partitioning and bucketing the index of the hive tables.
- Experienced in writing the SQL queries and procedures in MySQL.
- Developed Pig Latin scripts for adhoc requests.
- Experienced on loading and transforming of large sets of structured, semi structured and unstructured data.
- Designed the HBase schema and creating the HBase tables to store the data.
- Worked on bulk loading the data into HBase tables.
- Redesigned the HBase tables to improve the performance according to the query requirements.
- Scheduled Hadoop ecosystem jobs using Oozie.
- Developed UNIX scripts to trigger MapReduce jobs.
- Visualized the analytical results using Tableau visualization tool.
- Performed the regression testing on the application.
- Deployed the application on Amazon Web Services (AWS).
- Debugging the production issues and worked on the root cause analysis of the issue and fix accordingly.
- Worked on optimizing the cost reduction of production jobs by applying techniques like compression.
Environment: Hadoop 2.6.0,YARN, MapR, Redhat Linux, Cent OS, Java 1.6, Hive 0.13, Pig, MySQL, HBase, Spark,Sqoop, Oozie.
Confidential, San Diego, CA
Hadoop/Java Developer
Responsibilities:
- Involved in analysis, design and development of Expense Processing system.
- Designed Use Case Diagrams, Class Diagrams and Sequence Diagrams and Object Diagrams to model the detail design of the application using UML.
- Installed, configuring, and administrating Hadoop cluster of major Hadoop distributions.
- Written MapReduce jobs in Java, Pig and Python.
- Extensively worked with workflow/schedulers like Oozie and Scripting using Unix Shell Script, Python, and Perl.
- Worked with SQL and NoSQL (MongoDB, Cassandra, Hadoop) data structures
- Managing and reviewing Hadoop log files
- Running Hadoop streaming jobs to process terabytes of xml format data
- Worked on Hadoop Cluster migrations or Upgrades
- Extensively worked with Cloudera Hadoop distribution components and custom packages
- Build Reporting using Tableau
- Applied ETL principles and best practices
- Developed the application using Spring MVC Framework.
- Performed Client side validations using Angular JavaScript& Node JavaScript.
- Developed user interface using JSP, HTML,CSSand Java Script to simplify the complexities of the application.
- Used AJAX Framework for Dynamic Searching of Bill Expense Information.
- Created dynamic end to end REST API with Loopback-Node JS Framework.
- Configured theSpringframework for the entire business logic layer.
- Developed code using various patterns like Singleton, Front Controller, Adapter, DAO, MVC, Template, Builder and Factory Patterns
- Used Table per hierarchy inheritance of hibernates and mapped polymorphic associations.
- Developed one-to-many, many-to-one, one-to-one annotation based mappings in Hibernate.
- Developed DAO service methods to populate the domain model objects using Hibernate.
- Used SpringFramework‘sBean Factory for initializing services.
- Used Java collections API extensively such as List, Sets and Maps.
- Wrote DAO classes usingSpringand Hibernate to interact with database for persistence.
- Used Apache Log4J for logging and debugging.
- Used Hibernate in data access layer to access and update information in the database.
- Followed TDD and developed test cases usingJUnitfor all the modules developed.
- Used Log4J to capture the log that includes runtime exceptions, monitored error logs and fixed the problems.
- Created Maven build file to build the application and deployed onWebSphere Application Server.
Environment: Struts, Hibernate ORM,LoopBack Framework, Spring Application Framework, EJB, JSP,Servlets, JMS, XML, SOAP, WSDL, JDBC, JavaScript, UML, HTML, Angular JS, Node JS,JNDI, Subversion(SVN),Maven, Log4J,SpringSourceTool Suite(STS), Windows XP, Web Sphere App server, Oracle.
Confidential, Chanhassen, MN
Sr.JAVA Developer
Responsibilities:
- Used MySQL server at the backend because of its efficient, easy to use and very flexible commands
- Developed the application usingSpring MVCarchitecture.
- Importing and exporting data into HDFS and Hive using Sqoop
- Loading and transforming large sets of structured, semi structured and unstructured data
- Managed data coming from different sources
- Worked on Jobs management using Fair Scheduler and Cluster coordination
- Worked upon Automating data jobs and defining job flows
- Assign schemas and create HIVE tables
- Applied other HDFS formats and structure (Avro, Parquet, etc. ) to support fast retrieval of data, user analytics and analysis
- Developed variousUIcomponents usingSpring,JSP, and HTML, JavaScript, AJAX.
- Extensive involvement in design, development and coding.
- Coded the logic for performing the integrity checks for numerous source files in the Controller layer.
- UsedHibernate3.0as the ORM tool to communicate with the database.
- Extensively usedEJB 3.0, developed Entity Beans and Session Beans for business Logic.
- Wrote build script using maven for deployment.
- Developed differentSOAPservices andWeb ServicesusingWSDL, SOAP.
- Designedworkflowsand developed applications implementing the business processes.
- DevelopedAJAXscripting to process server sideJSPscripting.
- Incorporated model relationships and access controls for complex APIs using LoopBackframework.
- Involved in the GUI development for implementing new JSP pages.
- Developed the Action Classes, Action Form Classes, created JSPs using Struts tag libraries and configured inStruts-config.xml, Web.xmlfiles.
- Involved in integration ofGUIcomponents of the code.
- Assisted team lead with task management andScrums.
- Involved inunittesting, validating through User Acceptance Testing.
Environment: JDK 1.5,Java EE 5.0, Jboss Application Server 4.2.2, Eclipse Indigo 3.7,AJAX,IBM JRules 6.0,EJB 3.0,Oracle 10g, JSP 2.1,Spring 3.0, Hibernate 3.0,Struts2.0,LoopBack,JavaBeans,HTML, CSS, Java Script,XML, Maven2.0.7,Log4j,Db Visualizer6.5.7,WindowXP.
Confidential
JAVA/Mainframe Developer
Responsibilities:
- Developed the application using Agile methodology and planned thescrummeetings.
- Built multi-tieredhigh volume transaction, scalable, distributed application.
- Persistence layer was implemented usingHibernateFramework which interacts with the database.
- UsedDependency Injectionmechanism ofSpring FrameworkinBusiness logic implementation.
- Modified the web pages to include new fields on the browsing page to display new information about financial instruments.
- Developed the front-end of a compilation system usingC.
- Developed Spring AOP logging methods to log the price fluctuations.
- DesignedJSP and HTMLpages usingCSS, JavaScript, andAJAXfor validations.
- ParsedXMLdata usingSAXandDOMparsers.
- Worked on root cause analyses for all the issues that occur in batch and provide the permanent fixes for the issues.
- Developed a middleware application in Java using hibernate framework to retrieve the data from mainframe VSAM files to MySQL database.
- ConsumedWeb Servicesfrom a remoteFinancial Institutionsto pay the bill usingSOAP Protocol.
- CreatedStored Procedures, Functions.
- UsedChangeManfor version control.
- Developed the application using theSession Façade, Data Access Object, Data Transfer Object,and BusinessDelegate.
Environment: JAVA EE 5, IBM WebSphere Application Server 6.1, RAD 7.0, UML, IBMRationalRose,Hibernate 3.0, Spring 2.0, JSP 2.1, COBOL, VSAM, JCL, CICS, FileAid, TOM, XML Web Services, SOAP, WSDL, JMS, JNDI,Log4j, Java Script, XSL, Servlets 2.5,MySQL,JUnit, CVS,Windows Server 2003.