Bigdata Lead Resume
Parsippany, NJ
SUMMARY
- 10+ years of overall experience with java, J2EE and hadoop
- 4+ years of experience Hadoop Technologies like MapReduce, Pig, Hive, HBase, sqoop, oozie and zookeeper.
- Ability to optimize the usage of Hadoop to get maximum performance be it in Amazon Web Services, RackSpace or In - House Cluster.
- A passion to learn new things (new Languages or new Implementations) have made me up to date with the latest trends and industry standard.
- Experience in working with Hortonworks environment and MapR.
- Extensive experience in migration of Data Warehouse from RDBMS to Hive.
- Using Sqoop for import multiple table data from RDBMS to Hadoop environment.
- Expertise in developing MapReduce jobs to facilitate the flexibility of ETL.
- Using Pig to aggressively analyse and expose the various facts of the data on the fly.
- Experience using integrated development environment like Eclipse, Net beans, JDeveloper, MyEclipse.
- Experience with writing complex SQL queries in Confidential DB2, Oracle 10g, MS SQL Server 2005/2008, and MySQL.
- Ability to work on diverse Application Servers like JBOSS, APACHE TOMCAT, WEBSPHERE.
- Proficient in adapting to the new Work Environment and Technologies.
- Highly result oriented and pro-active, proven abilities to learn new technologies quickly and implementing them successfully in production.
- Experience in Working with various sizes of team from small to very large.
- Quick learner and self-motivated team player with excellent interpersonal skills.
- Well focused and can meet the expected deadlines on target.
- Experince Working with kafka, Storm and Spark.
- Experience Working with Machine Learning systems like Mahout.
- Excellent Communicational and written skills.
TECHNICAL SKILLS
Hadoop Certification: Cloudera Certification for Apache Hadoop Developer
Hadoop Technologies: HBase, HIVE, Sqoop, Flume, HDFS, Oozie, HCatelog, Mahout
J2EE Technologies: Servlets, JSP, EJB, JDBC, Web Services (WSDL, SOAP), Spring and Hibernate
Web Servers/application servers: Apache tomcat Server, Confidential WebSphere server, JBoss
Web tools and languages: HTML, XML, CSS, DHTML, Java Script
Databases: Confidential DB2, Oracle8i/9i/10g, MS SQL Server 2005/2008, MySQL
Languages: Java / J2EE, HTML, SQL
Operating Systems: Windows 2003/2008/XP/Vista, Unix, Linux (Various Versions)
Tools: MS-Office 2003/2007/2010 , Eclipse3.3/3.4, Eclipse, Net Beans.
Version Control: Confidential RTC.
Others: ASP.NET, VB.NET and C#
IDEs: Eclipse, NetBeans, JDeveloper, MyEclipse
PROFESSIONAL EXPERIENCE
Confidential, Parsippany, NJ
BigData Lead
Responsibilities:
- Using Sqoop to load data from DB2/Mainframe into HBase environment.
- Insert Overwriting the HIVE data with HBASE data daily to get fresh data every day.
- Using SPARK code written in python to perform ETL task where possible.
- Using Kafka with SPARK Streaming to consume messages from other system.
- Creating various views for HBASE tables and also utilizing the performance of Hive on top of HBASE.
- Using XML to get data and get some new information about the claim using XPATH UDF.
- Creating Custom UDF and UDTF to perform java operation in Hive and Pig.
- Using MLlib for detecting correlation for Fraud Detection score.
- Using Pig to transform large datasets into required format.
- Using Distributed Cache to store reference data and Improve performance of MR.
- Actively involved in the decision process of the data flow and utilization.
- Hortonworks 2.3 being used with Hive 1.2.1/HBASE 1.1.1.
- Using Spark SQL where possible to efficiently run the hive queries.
- Extensively used Map-Reduce to process the available terabytes of data.
- Using HCatalog to create on the fly tables and daily refreshing new data.
- Used PVCS for version control and maintaining the code.
- Using various Queue Names to give priority to the job being processed.
- The processed data is then provided for various applications consumption.
- Deployed and tested the application on UNIX based environments.
Environment: Hadoop, HBase, Hive, Map-Reduce, DB2, java, XML,, Pig, PVCS, UNIX, HCatalog.
Confidential, Orem, UT
Sr Hadoop Developer
Responsibilities:
- Using Core-Java to write Map-Reduce Jobs.
- Using Amazon Elastic Map-Reduce for processing data.
- Using Json to get data and store data into HDFS.
- Using Amazon RDS as Database for processing relations.
- Actively participated in the deployment of Hadoop Cluster into individual nodes.
- Involved in the decision making of shifting from MapR framework to HortonWorks.
- Extensively used Map-Reduce to process the available terabytes of data.
- Worked on loading the data from MySql to HBase where necessary using Sqoop.
- Using cassandra to store majority of data which needs to be divided based on region.
- Using Hive to query the exported CSV data containing billions of data records.
- Used Git Stash for source control.
- Using oozie to schedule various jobs on Hadoop Cluster.
- Java multi-threaded environment and quartz scheduler.
- Written JUnits and integration tests for the class developed.
- Developed projects using Maven to build and deploy.
- Deployed and tested the application on UNIX based environments.
Environment: Hadoop, HBase, Hive, Map-Reduce, Eclipse, JUnit, MySql, java, json,, maven, AWS.
Confidential, Salt Lake City, UT
Hadoop Developer
Responsibilities:
- Using Core java Technologies to Manipulate the Data from HDFS and getting desired Results.
- Using hibernate and spring in the Real-Time services using J2EE technologies using MVC design.
- Hadoop Map-Reduce is used to analyze the un-organized text files, searching for data or pattern.
- Using AMAZON RDS to manage the application Data.
- Hadoop HBase was used to store Data in de-normalized way.
- Using oozie to run regular jobs in timely manner.
- Using AMAZON EMR for analysis of large files.
- Hosting RESTful services in the intranet to allow other teams to get access.
- Worked on spring components IOC, AOP and MVC.
- The Back-End data has been maintained in Oracle Database.
- Development of java web services using SOAP.
- Java multi-threaded environment and quartz scheduler.
- Written JUnits and integration tests for the class developed.
- Developed projects using Maven to build and deploy.
Environment: Hadoop, Map-Reduce, HBase, Eclipse, JUnit, J2EE, ClearCase, AMAZON RDS, spring, hibernate.
Confidential, New Jersey
Sr. Java Consultant
Responsibilities:
- Developing functionalities and services with technologies like spring, Hibernate frameworks.
- Using spring components such as IOC, AOP and MVC to adhere to the requirement.
- The data was being maintained in DB2 to use the robust features of Confidential Database.
- Development of java web services using SOAP to enable sharing of data.
- Java multi-threaded environment is maintained to make best of the available resources.
- Written JUnits and integration tests for the class developed.
- The UI was developed using jQuery with HTML5 and ExtJS.
- The Data generated from the Application is carefully logged for later analysis.
- Support the Business Analyst if any issues arise during the production installs.
- Maintained source code versions and merging the code changes with ClearCase.
- Developed projects using Ant to build and deploy.
- Deployed and tested the application on UNIX based environments in test Servers.
Environment: Spring, Hibernate, Eclipse, JUnit, Ant, UNIX, J2EE, ClearCase and Oracle DB.
Confidential, TN
Sr. Java Developer
Responsibilities:
- The WebApplication is based on MVC Design Pattern using Spring MVC.
- The System is built based on Waterfall Model.
- Once the Requirement is assigned, the requirement needs to be thoroughly analyzed and documented with the questions for the customer team using URMS.
- The Different structure of different databases needs to be considered for the sizing of the requirement. Also need to make a list of files that will be affected like JSP, Servlets, HTML, javascript file, Database Tables etc.
- Using Various JavaScript, AJAX and DOJO components achieve the best possible UI which is acceptable for the client
- Each sub System has been assigned an individual Database. So, need to consider the various databases and their tables that will be affected.
- Once the Requirement is established, Test cases need to create for the Customer Approval of Requirement in Confidential Rational Quality Manager.
- Once the Requirement has been approved, it needs to be implemented in Environments like DEV, TEST, PREPROD and PRODUCTION.
- This Application uses Hibernate to connect to the database and maintain the integrity of the data.
- The Database maintained for the data was Confidential DB2 and Display the required field and make it accessible as POJO class.
- Deployment into Confidential WebSphere for production.
Environment: Java, JDBC, DB2, RTC, WebSphere, RCQ/URMS, Hibernate, Confidential DB2
Confidential
Sr. Java Developer
Responsibilities:
- The System is built using MVC Design Pattern.
- The MSC Portal was developed on WebSphere Portal using Java Portlets.
- The System follows Agile Methodology.
- The Portal was accessible for Tele-sales and Sales Managers.
- The Portal and Portlets use JMS and MOM API to communicate.
- The code which is analyzed for effected area such as UI (JSP, HTML, and CSS), Database, Servlets, and Java.
- Once the hours are approved, we need to make the changes accordingly in Developer server and provide the required SQL queries or PL/SQL program to the DBA (triggers if any).
- Once there has been basic testing done on the Achieved requirement, it is moved to PRE - SERVER (For Testing team) as a patch.
- Once the Testing team has approved, the requirement is moved to production.
- Hibernate is used as the default ORM tool for the integration of Database.
- The Project release once completed is deployed onto the Confidential WebSphere.
Environment: Java, JDBC, DB2, RTC, RCQ/URMS, Hibernate, Confidential DB2 and Confidential WebSphere, JNDI
Confidential
Java Developer
Responsibilities:
- Wrote and maintained the Software Requirement Specification (SRS) for the project
- Designed UML diagrams using Confidential Rational Rose 2001 EE,Borland Together, Magic Draw
- Installed and Configured WebSphere Portal Server 5.1, WSAD 5.1 and Portal ToolKit 5.0.2 plug-in.
- Provides work direction, tracks progress, and manages workload to other application developers as required.
- Modified the company’s WebSphere’s Portal themes and skins according to the portal requirements
- Developed SQL queries to implement Struts frame work.
- Successfully implemented the MVC architechture and Object Relational mapping is done using Hibernate
- Developed the start up service interfaces required and run time service implementation classes.
- Extensively used Struts tag libraries and jar files and Custom tags.
- Accountable for the successful execution of all application development activities.
- Wrote and maintained the Ant build script for the project
Environment: Java, JSP, Servlets, JavaScript, JDBC, Confidential Websphere 5.1 Application Server, WSAD, TOAD, Change Man, MS Windows 2000, LDAP, Oracle JTA, JMS and JNDI.
Confidential
Jr. Java Developer
Responsibilities:
- Analyzing and preparing the requirement Analysis Document.
- Deploying the Application to the JBOSS Application Server.
- Requirement gatherings from various parties involved in the project
- Estimate timelines for development tasks.
- Used to J2EE and EJB to handle the business flow and Functionality.
- Interact with Client to get the confirmation on the functionalities.
- Involved in the complete SDLC of the Development with full system dependency.
- Actively coordinated with deployment manager for application production launch.
- Provide Support and update for the period under warranty.
- Monitoring of test cases to verify actual results against expected results.
- Performed Functional, User Interface test and Regression Test
- Carrying out Regression testing to track the problem tracking.
Environment: Java, J2EE, EJB, UNIX, XML, Work Flow, JMS, JIRA, Oracle, JBOSS
