- Overall 7+ years of experience in Architect, Analysis, Design, Development, Testing, Implementation, Maintenance and Enhancements on various IT Projects. Around 3 Years of experience in Big Data in implementing complete Hadoop solutions.
- Hands on experience in using Apache Hadoop ecosystem components like Hadoop Distributed File System (HDFS), Map Reduce, PIG, HIVE, Zookeeper, Sqoop, Hue, Maven and JSON.
- Expertise in writing Hadoop Jobs for analyzing data using Hive and Pig.
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems(RDBMS) and vice - versa
- Experience in working with Map Reduce programs using Apache Hadoop for working with Big Data
- In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce concepts
- Worked on Cascading API for Hadoop application development and work flows
- Experience in analyzing data using Hive QL, Pig Latin, and custom Map Reduce programs in Java.
- Knowledge of job workflow scheduling and monitoring tools like Oozie and Zookeeper
- Experience in optimization of Map reduce algorithm using combiners and partitions to deliver the best results.
- Good understanding of NoSQL databases like HBase, Cassandra, and Mongo DB.
- Good understanding of Data Mining and Machine Learning techniques
- Expertise in core Java, J2EE, Multithreading, JDBC, Web Services Shell Scripting and proficient in using Java API’s for application development
- Solid background in Core Java concepts like Threads, Collections Framework, Java Reflection and also have handsome experience in building Class diagrams, activity diagrams, sequence diagrams, activity diagrams and flow charts using Rational Rose and Visio.
- Strong application knowledge of various design patterns including MVC, VO, Session Facade, Abstract Factory, Business Delegate, Singleton, Command, Front Controller and DAO for the development of Multi-Tier distributed Enterprise Applications.
- Strong database connectivity skills which includes Oracle, MYSQL, and DB2 and in programming with SQL, PL/SQL, and Stored Procedures, Triggers, Functions and Packages besides writing DDL, DML and Transaction queries with development tools like sql developer.
- Proficient in Working with Various IDE tools including Eclipse Galileo, IBM Rational Application Developer (RAD) and IntelliJ IDEA
- Worked on different operating systems like UNIX/Linux, Windows XP and Windows 2K
- Excellent working experience in Scrum / Agile framework and Waterfall project execution methodologies.
- Very good experience in customer specification study, requirements gathering, system architectural design and turning the requirements into final product.
- Strong technical background, excellent analytical ability, good debugging skills, good communication skills, team player, goal oriented and ability to quickly learn new technologies as required.
Technologies: Hadoop (Cloud era, Horton works), Hive, Pig, Sqoop, Storm, Mahout, Oozie, HBASE, Solr, Spark, Tableau, Java Beans, Servlets, JSP, JDBC, EJB, JNDI, JMS, RMI.
Architecture & Framework: Client-Server, MVC, J2EE, Struts, Spring, Hibernate.
Database: Cassandra, HBase, Oracle 11g, SQL server 2008,MySQL
IDE: Eclipse, WSAD, Net Beans, IBM RAD, JBuilder.
Design Methodology: UML, Water Fall, Perl, Agile
Operating Systems: Windows 9x/NT/XP/2003/VISTA/7/8, Linux, Unix
Conceptual knowledge: Web Services, Socket Programming.
Query Languages: SQL, PL/SQL.
Design patterns: Business Delegate, Business Object, Value Object, Front Controller, Database Access Object, Factory, Singleton, Session Facade.
Tools: BEA WebLogic 8.1, JBOSS, IBM WebSphere Application Server 6.1, Tomcat 6.0, JUnit 4.0, ANT, Log4j, Mercury Quality Centre, Rational Clear Quest. ANT, Maven, SVN, Toad
Design & Control: UML, Rational Rose, CVS, Clear Case
Senior Hadoop Developer
Confidential, Union, NJ
- Used Sqoop to extract data from Oracle SQL server and MySQL databases to HDFS
- Developed workflows in Oozie for business requirements to extract the data using Sqoop
- Developed Map Reduce(YARN) jobs for cleaning, accessing and validating the data
- Wrote Map Reduce jobs using Pig Latin
- Used Hive and Impala to query the data in HBase
- Hive scripts were written in Hive QL to de-normalize and aggregate the data
- Automated the work flows using shell scripts (Bash) to export data from databases into Hadoop
- Used JUnit framework to test the Unit testing of the application
- Hive queries for data were written to meet the business requirements
- Developed product profiles using Pig and commodity UDFs.
- Designed workflows by scheduling Hive processes for Log file data, which is streamed into HDFS using Flume
- Developed schemas to handle reporting requirements using Tableau
- Actively participated in weekly meetings with the technical teams to review the code
- Involved in loading data from UNIX file system to HDFS
- Implemented test scripts to support test driven development and continuous integration
- Responsible to manage data coming from different sources
- Have deep and thorough understanding of ETL tools and how they can be applied in a Big Data environment
- Participate in requirement gathering and analysis phase of the project in documenting the business requirements by conducting workshops/meetings with various business users
- Involved in moving all log files generated from various sources to HDFS for further processing through Flume
Environment:: Hadoop, Map Reduce, Hive QL, Hive, HBase, Sqoop, Solr, Cassandra, Flume, Tableau, Impala, Oozie, MYSQL, Oracle SQL, Java, Unix Shell, YARN, Pig Latin.
Confidential, Columbus, Ohio
- Worked on analyzing Hadoop cluster using different big data analytic tools including Pig, Hive, and Map Reduce
- Responsible for writing Hive Queries for analyzing terabytes of customer data from Hbase and put the results in CSV file.
- Supported Map Reduce Programs those are running on the cluster
- Worked on debugging, performance tuning of Hive & Pig Jobs
- Implemented test scripts to support test driven development and continuous integration
- Worked on tuning the performance Pig queries
- Involved in loading data from LINUX file system to HDFS using Kettle
- Importing and exporting data into HDFS and Hive using Sqoop
- Experience working on processing unstructured data using Pig and Hive
- Gained experience in managing and reviewing Hadoop log files.
- Wrote Hive UDFS to format the data.
- Involved in scheduling Oozie workflow engine to run multiple Hive and Pig jobs
- Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts
Environment:: Hadoop, HDFS, Pig, Hive, Map Reduce, Sqoop, Oozie, LINUX, and Big Data.
Senior Java/J2EE Developer
Confidential, Scranton, PA
- Actively participated in requirements gathering, analysis, design, and testing phases.
- Designed use case diagrams, class diagrams, and sequence diagrams as a part of Design Phase using Rational Rose.
- Developed the entire application implementing MVC Architecture integrating JSF with Hibernate and spring frameworks.
- Designed User Interface(UI) using Java Server Faces (JSF),Cascading Style Sheets (CSS), and XML.
- Developed the Enterprise Java Beans (Stateless Session beans) to handle different Sessions.
- Developed deployment descriptors for the EJB have to deploy on Web Sphere Application Server.
- Implemented Service Oriented Architecture (SOA) using JMS for sending and receiving messages while creating web services.
- Developed Web Services for data transfer from client to server and vice versa using Apache Axis, SOAP, WSDL, and UDDI.
- Developed the applications on Linux Environment by connecting to Database and implementing Programs.
- Extensively worked on MQ Series using point-point, publisher/subscriber messaging Domains for implementing Exchange of information through Messages.
- Developed XML documents and generated XSL files for Payment Transaction and Reserve Transaction systems.
- Implemented various J2EE Design patterns like Singleton, Service Locator, Business Delegate, DAO, Transfer Object, and SOA.
- Used Subversion to implement version control System.
- Build ANT Script for the application and used Log4J for debugging.
- Used JUnit Framework for the unit testing of all the java classes.
Confidential, Hartford, CT
- Contributed to the Design, Created Class diagrams, Sequence diagrams and Activity Diagrams.
- Created the scripting code to validate the data.
- Worked on different J2EE Design Patterns such as Front controller, Session Facade, Service Locator, Singleton and DAO.
- Developed/modified the bean components to in corporate new business level validations.
- Implemented SAX parsing of XML production data.
- Developed the Core Services which consume the partners Web Services and exposed them as Web services using Axis 1.2.
- Deployed Web Services in Web Sphere.
- Wrote queries, stored procedures and functions using SQL, PL/SQL
- Developed the Business logic Java components.
- Developed the email component using Java Mail to send confirmation emails to users.
- Was responsible for deploying and testing entire Web Services layer developed as a part of Core, using Soap UI.
- Involved in coding Data Layer, which is wrapper class over database.
- Generated build file using Ant 1.6.
- Responsible for performing end-to-end system testing of application writing JUnit test cases
- Developed test environment for testing all the Web Service exposed as part of the core module and their integration with partner services in Integration test.
- As part of the development team Contributed for Application Support in Soft launch and UAT phase.
Environment:: Java, Servlets, JSP, JDBC, Struts, Tiles, AJAX, JAXP, XML, XSL, XSLT, SOAP,JSON, Web Services, Java Mail, Eclipse, Web Sphere, Axis, JUnit, Oracle 8, PL/SQL, Ant, Rational Rose 2000, TOAD, Edit plus, HTML, Windows NT, Unix, Java / J2EE Patterns.
- Involved in design and development of web front end using HTML, Java Script, CSS and JSP’s for Administration, Efficiency Management and Self-Assessment modules
- Developed and tested the Efficiency Management module using EJB, Servlets, and JSP & Core Java components in WebLogic Application Server
- Developed Struts framework, providing access to system functions of a server’s business layer
- Implemented business components as persistent object model as EJBCMP and BMP Entity Beans for storing and retrieving data objects from Resources
- Implemented the application MVC Architecture using Struts framework
- Involved in stored procedures using PL/SQL to interact with the Oracle database required by the Efficiency Module
- Deployed web components, presentation components and business components in WebLogic Application Server
Environment:: Java, J2EE (Servlets, JDBC, EJB, JSP, JMS), HTML, CSS, Java Script, eclipse, Struts Frame Work 1.1, ANT, XML, CVS, Oracle 8i,PL/SQL, Log4j, Windows XP.