- Around 7 years of experience in Designing, Developing Web applications and Big data applications using Java, J2ee, Oracle and Cloudera Hadoop based Big data technologies
- Experience in writing Map Reduce programs for both Hive and pig in java.
- Experience in data load management, importing and exporting data using Sqoop and FLUME.
- Experience in creating Hive Internal/External tables and views using shared metastore, writing scripts in HiveQL, data transformation & file processing using Pig Latin Scripts.
- Knowledge on installation, configuration, support, maintenance of Cloudera’s Hadoop platform along with CDH4&5 clusters.
- Excellent understanding of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name node, Data node and MapReduce.
- Trained in Cloudera based Hadoop Distribution.
- Knowledge on writing python scripts for internal testing which pushes the data reading form a file into Kafka queue which in turn is consumed by the Storm application.
- Experience in application programming using Servlets, EJB’s.
- Designed and develop web based UI application using HTML, CSS, and JSP.
- Well versed in MVC (Model View Controller) architecture using spring, JSF and also implementing JSTL (JSP standard tag library), custom tag development and tiles.
- Knowledge on Performing memory data processing for batch, real time, and advanced analytics using Apache Spark.
- Good understanding in migrating map reduce programs into Spark RDD transformations, actions to improve performance
- Good Exposure to Web/application servers such as Apache Tomcat, WebLogic.
- Experience in developing database to create its objects like tables, views, functions, triggers and stored procedures packages using PL/SQL in Oracle.
- Strong analytical and logical ability to work independently or in a team.
Big Data Technologies: Hadoop, HDFS, Hive, Pig, HBase, Spark, Scala, Sqoop, Flume, ZooKeeper, Kafka, Cloudera CDH4, CDH5, AWS, HiveQL, Pig Latin.
Java/J2EE Technologies: JSF, Struts, Servlets, JSP,EJB, Junit and JDBC
Programming Languages: C, C++, Java, SQL, PL/SQL, HTML,XML
Web Development: HTML5, DHTML, XHTML, CSS, Java Script, AJAX
Frameworks: Struts, Hibernate, Spring, JSTL
Application/Web Servers: Apache Tomcat, IBM Web Sphere.
Databases: Oracle 10g/11g, SQL Server, IBM DB2.
Methodologies/Design Patterns: OOAD, OOP, UML, MVC
IDEs: Eclipse, Net beans
Tools: MSOffice, Microsoft Visio, XML Spy, TOAD
Operating Systems: Windows 98/2000/NT/XP, UNIX and LINUX
Confidential, Somerset, NJ
- Imported logs from web servers with Flume to ingest the data into HDFS.
- Implemented custom interceptors for flume to filter data and defined channel selectors to multiplex the data into different sinks.
- Retrieved data from HDFS into relational databases with Sqoop.
- Parsed cleansed and mined useful and meaningful data in HDFS using Map - Reduce for further analysis
- Fine tuning hive jobs for optimized performance
- Implemented UDFS, UDAFS, UDTFS in java for hive to process the data that can't be performed using Hive inbuilt functions.
- Designed and implemented PIG UDFS for evaluation, filtering, loading and storing of data.
- Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.
Environment: Hadoop, Hive, Pig, Sqoop, Spark, Oracle10g, HDFS, Oozio, Flume, Scala
Confidential, Edison, NJ
- Gathered all the business requirements from the business Partners.
- Wrote Map reduce jobs using various input and output format. Also design custom format as per the business requirements.
- Importing and exporting data from HDFS from Oracle database and vice versa using SQOOP.
- Experience working on processing unstructured data using Pig and Hive.
- Wrote PIG UDFs to perform data cleansing and transforming for ETL activities.
- Wrote HIVE UDF, UDAF, UDTF for Data analysis and Hive table loads.
- Responsible for analyzing and cleansing raw data by performing hive queries and running pig scripts on data.
- Monitored Hadoop cluster job performance and performed capacity planning and managed nodes on Hadoop cluster.
- Involved in converting the data to HL7 Format.
- Developed workflow on Oozie for automating the job flows.
Environment: Hadoop, Pig, Sqoop, Oozie, Oracle 10g, Linux OS, MS project
Confidential, Edison, NJ
Java / Hadoop developer
- Involved in Design and Development of technical specification documents using Hadoop technology on Amazon cloud computing (EC, S3).
- Developed an integration wrapper to push computations from local server to the Amazon cloud.
- Developed MapReduce programs to parse the raw data using Clusters.
- Deployed the application in Amazon Web Services (EC2).
- Implemented the Naïve Bayes algorithm to analyze the data according to the user’s requirements.
Confidential, Edison, NJ
- Extensively involved in gathering requirements, preparing requirements documents.
- Implemented Struts framework.
- Generated User interface Templates using XML/XSL.
- Developed the Java Code using Eclipse as IDE.
- Implemented Action classes, which contain the logic of selecting and using the right Data Access Object (DAO) classes to create and execute, host transaction(s) applicable to current user operation. The request handlers are also responsible for readying up the data received from the host (via DAOs) to meet presentation requirements.
- Used Log4j for logging and debugging.
Environment: Java/J2EE, Oracle, Struts, JSP, XML, Eclipse and Apache Tomcat.
Confidential, High hills, NJ
- Involved in Design and development of the application.
- Worked with struts validation and mappings
- Developed JSP pages to meet the requirements.
- Worked with MVC Design patters with Struts.
- Participated in work flow design and framework design.
- Generated User interface Template using JSP.
- Implemented Action classes and Data Access Object (DAO) classes.
Environment: Java/J2EE, Struts, JSP, XML, Eclipse, Apache Tomcat.
- Involved in all phases of the application development.
- Wrote procedures to call various stored procedures from database.
- Designed the Database.
- Designed/developed tables, views, materialized view, stored procedures, packages and functions.
- Developed various backend application programs, such as Triggers using SQL and PL/SQL language for the management for decision making.
- Wrote the validation XML files and properties file needed for validating user inputs.
- Developed Presentation layer using JSP, HTML and XML.
- Performed unit testing, system testing and user acceptance test.