Hadoop/java developer Resume
Richmond, VA
PROFESSIONAL SUMMARY:
- Over 8 years of professional IT experience in requirement gathering, design, development, testing, implementation and maintenance. Progressive experience in all phases of the iterative Software Development Life Cycle (SDLC)
- Good Knowledge on Hadoop Cluster architecture and monitoring the cluster.
- Focus on Hive querying and Hive optimization.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice versa.
- Good Knowledge on Spark, Storm and HBase to do real time streaming.
- Experienced with writing Hive UDF and GenericUDF.
- Experienced with data cleansing in writing Map Reduce jobs and Spark jobs.
- In - depth understanding of Data Structure and Algorithms.
- Work with colleagues to transfer from Hadoop System to Spark System.
- Experience in managing and reviewing Hadoop log files.
- Excellent understanding and knowledge of NOSQL databases like HBase, Cassandra.
- Experience in implementing in setting up standards and processes for Hadoop based application design and implementation.
- Knowledge of Databases and data warehousing platforms
- Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
- Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, NameNode, Data Node and Map Reduce programming paradigm.
- Hands on experience in installing, configuring, and using Hadoop ecosystem components like
- Hadoop Map Reduce, HDFS, HBase, Hive, Sqoop, Pig, Zookeeper and Flume.
- Good Exposure on Apache Hadoop Map Reduce programming, PIG Scripting and Distribute Application and HDFS.
- Experience in managing Hadoop clusters using Cloudera Manager Tool.
- Very good experience in complete project life cycle (design, development, testing and implementation) of Client Server and Web applications.
- Experience in Administering, Installation, configuration, troubleshooting, Security, Backup,
- Performance Monitoring and Finetuning of Linux Redhat.
- Extensive experience working in Oracle, DB2, SQL Server and My SQL database.
- Hands on experience in VPN, Putty, winSCP, VNCviewer, etc.
- Scripting to deploy monitors, checks and critical system admin functions automation.
- Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
- Ability to adapt to evolving technology, strong sense of responsibility and accomplishment.
TECHNICAL SKILLS:
Languages/ Technologies: Data HDFS, HBase, Impala, Hadoop Map Reduce, Zookeeper, Hive, Pig, Sqoop, Spark.Java (JDK), JavaBeans, Struts, Servlets, EJB, RMI, JSP, Applets, ASP, Java Mail API 1.1.1, XML/XSLT, C++, C, SQL, PL/SQL, HTML, SOAP, JAXB, JAXP, JAX-RPC, JavaScript, ODBC, JDBC, Webservices, MAINFRAMES, CVS, CICS, COBOL, JCL, REXX, LDAP, MQSERIES, AOP, Dynamic proxy,VB 6.0, PrimeFaces, Icefaces,Spring,Hive,OOzie,HDFS,Pig
IDE/Tools/Utilities: WSAD 5.1,RAD,RSA 7.0/8.0,Visual Café 2.5, Eclipse 3.x,IBM Visual Age for Java 3.5, RationalRose 98/2000, PVCS, Jbuilder4.0/6.0, Jdeveloper 3.2, Visio 2000, SQL*Plus, SQL*Loader, Toad, Rational Rose, PVCS, XML Spy,, CTG,ECI,EPI,Wily, ESB, SI BUS,Apache XercesApplication/Web Servers: IBM WAS 5.x,6.x,7.x,WebLogic 6.x/7.x/8.x,IBM Web sphere 5.0,RAD 6.0,RSA 7.0,,Apache Web Server, Tomcat4.x, JBoss 3.x.Websphere Portal Server,oc4j,Jrules,Jquery
Databases: Oracle 7.x/8.x/9.x/10.x/11.x, SQL Server7.0/2000, DB2,MS Access 7.0, Microsoft Active Directory, MYSQL
Operating Systems: Windows NT/98/2000/XP/2003, UNIX, Sun Solaris 2.x, and DOS.
Software Products: Putty, Eclipse, Toad 9.1, DB Visualizer, Comptel's AMD 6.0.3 & 4.0.3, InterConnecT v7.1 & 6.0.7, MS Project 2003, HP Quality Center, MS Management studio, MS SharePoint
Web Technologies/Frameworks: HTML, XML, AJAX, SOAP, ODBC, JDBC, Java Beans, EJB, MVC, JSP, Servlets, Java Mail, Struts, Junit Frameworks MVC, Spring, Struts, Hibernate, .NET, JSF
PROFESSIONAL EXPERIENCE:
Confidential, Richmond, VA
Hadoop/Java Developer
Responsibilities:
- Developed workflows using custom Map Reduce, Pig, Hive, Sqoop, Oozie, NI-FI, Spark, Storm, and Kafka messaging.
- Successfully loaded files to Hive and HDFS from Teradata Database
- Installed and configured Hadoop Mapreduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
- Transfer formal Hive queries to Spark SQL.
- Importing and exporting data into HDFS and Hive from Oracle using Sqoop.
- Wrote Mapreduce jobs and PIG to do ETL in Hadoop System
- Tuned the cluster for optimal performance to process these large data sets
- Built reusable Hive UDF libraries for business requirements which enabled users to use these UDF's in Hive querying the logs and semi structured content that are stored on HDFS
- Configured big data workflows to run on the top of Hadoop using Control M and these workflows comprises of heterogeneous jobs like Pig, Hive, Sqoop and MapReduce
- Developed suit of Unit Test Cases for Mapper, Reducer and Driver classes using MR Testing library
- Cooperate with Data Scientists to implement Network Analysis algorithms.
- Transfer formal Hive queries to Spark SQL.
- Transfer machine learning jobs from Mahout to Spark MLlib
- Importing and exporting data into HDFS and Hive from Teradata using Sqoop.
- Wrote Mapreduce jobs and PIG to do ETL in Hadoop System
- Tuned the cluster for optimal performance to process these large data sets
- Used native Hadoop libraries as well as Webhdfs to load the data to Hadoop
- Used Oozie for managing the workflows with in the application
- Extensively used pig/hive scripts in creation, validation, transformation of the raw files in processed tables.
- Used Maven extensively for building jar files of Map Reduce programs and deployed to Cluster Bug fixing and 247production support
- Used websphere application server 8.0 and RAD 8.5 for development.
- Used primefaces 3.5/4.0/5.0 for UI development along with JSF composite and custom components.
- Used webservices and P6 API to retrieve the data for the Business process created for capturing the PTC project information.
- Used Spring to work with Command Chaining,Aspects,Interceptors,IOC injecting beans and properties to Business objects and Spring Batch to load the data
- Designed the application using DAO Layer, Service Controller, MVC, Factory, Data Access Object, Service Locator design patterns
- Used Clear case for the source control and clearquest for defect management along with HP quality center.
- Used primefaces and Jquery and Map API in configuring the real time map with the application.
- Used SOA Web services to expose business methods for the application processing
- Used Oracle 11G as back end and developed stored procedures, Batch jobs, and triggers.
- Work on implement methodologies like Agile and Scrum
Environment: CDH3, PIG(0.8.1), HIVE(0.7.1), Sqoop (V1), Oozie (V2.3.2), Core Java, Oracle 11g, SQL Server 2008, Hbase, Cloudera Hadoop Distribution, Map Reduce, Teradata, DataStax, IBM DataStage 8.1, Oracle 11g / 10g, PL/SQL, SQL*PLUS, Toad 9.6, Windows NT, LINUX, UNIX Shell Scripting. Java, Mqseries, JSF, Primefaces, java, J2EE1.4, RSA, Websphere, Oracle,JPA,UML/Rational Rose 2000, Log4j, JMS, JSP, XML, XSL, EJB, Servlets, JNDI,Tomcat 4.0, Sun/Solaris, Oracle 11.x, AQT, RMI,Mbeans,.net,Clearcase,Jquery,Messaging,Quality Center,, SQL Server, .net,Spring,hibernate,JPA,JSF,Primefaces. Json, JAXB, JIRA.
Confidential, Greensboro, NC
Hadoop /Java Developer
Responsibilities:
- Involved in review of functional and nonfunctional requirements.
- Facilitated knowledge transfer sessions.
- Installed and configured Hadoop Map reduce, HDFS, Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
- Working with Storm and Hbase to do real time steaming.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Experience in managing and reviewing Hadoop log files.
- Extracted files from RDBMS through Sqoop and placed in HDFS and processed.
- Experience in running Hadoop streaming jobs to process terabytes of xml format data.
- Got good experience with NOSQL database.
- Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
- Replaced default Derby metadata storage system for Hive with MySQL system.
- Executed queries using Hive and developed Map Reduce jobs to analyze data.
- Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
- Developed the Pig UDF's to preprocess the data for analysis.
- Developed Hive queries for the analysts.
- Involved in loading data from LINUX and UNIX file system to HDFS.
- Supported in setting up QA environment and updating configurations for implementing scripts with Pig.
- Developed a custom File System plug in for Hadoop so it can access files on Data Platform. This plugin allows Hadoop MapReduce programs, HBase, Pig and Hive to work unmodified and access files directly.
- Designed and implemented Map reduce based large scale parallel relation learning system
- Setup and benchmarked Hadoop/HBase clusters for internal use
- Wrote recommendation engine using mahout.
- Involved in System Requirement Analysis and Design
- Business Requirement Analysis and Use Case Analysis using Rational Rose 2000 - UML
- Designed the application using Front Controller, Service Controller, MVC, Spring, Factory, Data Access Object, Service Locator, and Session Façade Design Patterns
- Designed and Implemented administration screens with MVC architecture using Struts Coding involves writing Action Classes/Forms/Custom Tag Libraries, JSP and EJB
- Used SOA Web services to call the rating engine for claim processing.
- Used AOP aspects and JDK dynamic proxy for creating the interceptors and aspects.
- Used Rational ClearCase for version Management.
- Used Visual Source safe for the requirement and document versioning.
- Designed and developed several EJB’s (Session,MDB and Entity beans)
- Developed Classes in RAD 6.0 for Java, using the Class Specification provided in Rational Rose.
- Created several JSPs, Java Beans and custom tag lib for front end designing of the website.
- Used Java Script for Data validations in the front end part.
- Designed, developed and deployed necessary stored procedures and triggers in Oracle 9.x and oracle 10g
- Also Used RAD 6.0 for developing Java, RMI, Java Mail, JDBC, EJBs code and RAD 6.0 for deploying EJB’s, JSPs, Servlets and all the Java classes
- Used Apache Xerces, JAXP and JAXB XML parsers for data parsing and validation of xml schemas.
- Implemented log4j by enabling logging at runtime without modifying the application binary
- Coding Different Java helper and validation classes for the Application logic and utilities
Environment: Java, Eclipse, Oracle 10g, Sub Version, Hadoop, Hive, HBase, Map Reduce, HDFS, Pig Hive, Cassandra, Java (JDK 1.6), Hadoop Distribution of Cloudera, Map Reduce, IBM Data Stage 8.1, Oracle 11g / 10g, PL/SQL, SQL*PLUS, Toad 9.6, Windows NT, LINUX, UNIX Shell Scripting.,Java, Mqseries, Spring,AOP, Jdk Dynamic proxy java,J2EE1.3, UML/Rational Rose 2000, Log4j, JMS, JSP, XML, XSL, EJB, Servlets, JNDI RAD 6.0, Tomcat 4.0, Rational ClearCase, Sun/Solaris, Oracle 8.x/9i, SQL Navigator, Siebel
Confidential, Dover, NH
J2EE Developer
Responsibilities:
- Designed the project using MVC, Session Fa ade, and Service Locator and Business Delegate patterns.
- Followed agile methodology, interacted with the client, provide/take feedback on the features, suggest/ implement optimal solutions and tailored application to customer needs.
- Integrated the application with Spring Framework for implementing Dependency Injection and provide abstraction between presentation layer and persistence layer.
- Developed Web pages using Struts view component JSP, JavaScript, HTML, jQuery, AJAX, to create the user interface views Migration 3rd party applications.
- Implemented validation on the JSP fields using Struts Validation framework which included writing validation.xml and validation - rules.xml files
- Implemented all DAO's using Spring Framework. Implemented SOA using Web Services (SOA over WESB and SOAP over JMS) on ESB utilizing WebSphere Integration Developer (WID).
- Used Hibernate ORM framework as persistence engine, configured O/R mapping and wrote hibernate queries.
- Involved in designing front-end screens using Java script, JSP, Angular JS, AJAX, HTML5 and DHTML.
- Designed Presentation layer using JSP, Struts framework and did front-end validations using JavaScript and JQuery.
- Involved in design and development of UI component, using frameworks Angular JS, JavaScript, HTML5, CSS and Bootstrap.
- Followed Single Page Application (SPA) Pattern and used Angular JS MVC to build a cross-browser complaint application.
- Implemented core features of Angular JS framework such as dependency Injection, Data-binding, Filters, Directives, Templates, Services and Deep Linking.
- Designed and integrated the full scale Hibernate / spring.
- Developed hibernate DAO framework with spring framework.
- Configured and deployed the application on WebLogic Application Server.
- Implemented Complex business Delegate layer using DAO's created and business logic.
- Exposed various capabilities such as Web Services using SOAP and WSDL.
- Worked on J2EE components like Servlets, JSPs and MDBs.
- Used MVC architecture, Business Delegate, Session facade, Data Access Object, Value Object and Singleton patterns
- Used Xfire to read the XML response coming from the web service.
- Created JUnit test cases, and Development of JUnit classes.
- Configured log4j to enable/disable logging in application.
- Developed ANT script to build .war and .ear files for the application.
Environment: Java, Eclipse, J2EE1.3, Struts 1.0.1/1.0.2, UML/Rational Rose 2000, Log4j, JMS, JSP, XML, XSL, EJB, Servlets, JBoss 3.x, Oracle 8I, Rational ClearCase, Mainframes, Sun/Solaris, Oracle 8.x/9i, Toad, CTG, CICS, Cobol.
Confidential, NC
Jr. JAVA Developer
Responsibilities:
- Involved in designing modules and documenting them as a Module Design Document (MDD).
- Worked on both front end using JSP, spring framework and back end part usingEJB and Hibernate.
- Involved in Object Oriented Analysis (OOA) and Object Oriented Design (OOD) Techniques using Unified Modeling Language (UML).
- Persisted common application level data like domain libraries using Cache.
- Developed user interface using JSP, Java Script, HTML5 and CSS.
- Maintained CSS and HTML, XSL, XSLT, JAVA SCRIPT, JSF, Angular JS, for designing Webpages.
- Hibernate was used as admin tool persistence framework.
- Implemented interceptors using Hibernate and Spring.
- Used Ajax for intensive user operations and client - side validations, Using JavaScript for client side validation.
- Expertise in web development and related software Angular JS, HYML5, CSS.
- Developed JSP's, strictly confining to the policy of no java use in them by using JSTL.
- Presentation tier of the application was built completely on Spring/Struts (For the Admin App).
- Page Navigation, Java bean management and input validation is taken care by JSF.
- Used Spring Tags, Struts Tag Libraries, and Validators in the development.
- Used Spring-Core and Spring-AOP.
- Used Hibernate as an ORM Framework for productive application development by reducing development time, complexities and increase flexibility.
- Developed end-to-end composite RESTFUL Web Services by using SOA with CXF framework to use with different clients including Web & Mobile.
- Developed Stored Procedures, Triggers and Functions and actively involved in tuning SQL queries, Stored Procedures, Triggers.
- Used JSON strings to transfer java object data to Angular JS and vice versa.
- Used Web Services for creating rate summary and used WSDL and SOAP messages and used XML parsers for data retrieval. Also Developed and consumed REST base web services.
- Test driven development approach was used and hence worked intensively on developing test cases.
- Quality Control Activities such as defect tracking, fixing using JIRA.
- Involved in code review process, validating designs done by other developers and development of the application.
Environment: UNIX, J2EE (Servlets, JSP, JSF, EJB, JAX-WS), Struts, Spring, Hibernate, Restful, Multithreading, Oracle, HTML5, CSS, Angular JS, Java Script, CSS, XML, ANT, Rational Clear Case, WebLogic, JTA, JUnit,
