Sr. Big Data Engineer Resume
Ca Foster City, CA
PROFESSIONAL SUMMARY:
- Over 9 years plus of experience in IT industry which includes Big Data Technologies and Java, J2EE Technologies
- Good hands - on experience in Apache Spark with Scala
- Good hands-on Java development skills using J2EE Frameworks like Spring, Hibernate and RestFul Web Services, Micro-services.
- Experience in writing MAPREDUCE programs in java for data cleansing and preprocessing.
- Experience in working with Flume to load the log data from multiple sources directly in to HDFS
- Familiar with importing and exporting data using Sqoop
- Experience in managing and reviewing Hadoop Log files.
- Experience in working on various Spring modules (Spring MVC, DAO and AOP) along with Hibernate as the back-end ORM tool for implementation of persistence layer and mapping of POJOs.
- Experienced with Java Multithreaded programming to develop multithreaded modules and applications.
- Extensive experience in different IDEs like RAD, Eclipse, NetBeans and JBuilder.
- Worked on different Application and Web Servers like WebLogic 7.x, Confidential Web Sphere 6.1/7.0,8.0, BEA Web Logic 9.0/10.0, Apache Tomcat and JBoss.
- Experience in relational and Object oriented database design and implementation using Oracle 8i/9i/10g, MySQL and writing queries in SQL
- Effective independent and team player and self-starter with high adaptability to new technologies.
- Have excellent analytical, problem solving and communication skills.
- A Team Player with good interpersonal skills, decision-making and committed to quality
TECHNICAL SKILLS:
Programming Languages: Java,Scala
J2EE Technologies: Servlets, JSP, JSTL, JMS, JDBC, Web Services, SOAP, WSDL, Restful, Pivotal-Cloud, ANT, JAXB, Spring 4.0, Hibernate 3.0 ORM
Bigdata Ecosystem: Hive, HBase, Apache Spark, PIG, Sqoop, Flume, Oozie, Zookeeper, Kafka, Storm and MapReduce frameworks.
Web/Applications Servers: IntelliJ, Confidential WebSphere 5.x/6.x/7.x/8.x, WebLogic 7.x, Tomcat 4.x/5.x, JBoss 4.x
J2EE IDEs: Confidential WebSphere Studio (WSAD) 4.x/5.x, Eclipse 4, JBuilder 2006,RAD 6.0
Database and Related: Oracle 8i/9i/10g,DB2,SQL, PL/SQL
Operating Systems: Windows NT/2000/XP/2003 Servers, UNIX, Linux
Version Control & tools: GITHUB, CVS,SVN
PROFESSIONAL EXPERIENCE:
Confidential, CA, Foster City, CA
Sr. Big data Engineer
Responsibilities:
- Created Spark jobs to see trends in data usage by users.
- Developed Spark code to using Scala and Spark-SQL for faster processing and testing.
- Developed a data pipeline using HBase, Spark and Hive to ingest, transform and analyzing customer behavioral data
- Experienced in working with Spark eco system using SCALA and HIVE Queries on different data formats like Text file and parquet
- Process data through Kafka and verify the same in Spark as experiment
- Involved in developing Spark using Scala and Spark-SQL for faster processing of data and exploring of optimizing it using Spark Context, Spark-SQL, Pair RDD's, Spark YARN.
- Real time streaming the data using Spark with Kafka.
- Performed performance tuning for Spark Steaming e.g. setting right Batch Interval time, correct level of Parallelism, selection of correct Serialization & memory tuning.
- Developed Spark-SQL for faster testing and processing of data.
- Import the data from different sources like HDFS/HBase into Spark RDD developed a data pipeline using Kafka to store data into HDFS.
Environment: Big data Ecosystem: Spark 2.0, Scala, Python, Kakfa, HBase, Hive, PIG, Zookeeper and Unix.
Confidential, Denver, CO
Sr. Big data Engineer
Responsibilities:
- Used Spark API over Hadoop YARN to perform analytics on data in Hive.
- Process data through Kafka and verify the same in Spark as experiment
- Involved in developing Spark using Scala and Spark-SQL for faster processing of data and exploring of optimizing it using Spark Context, Spark-SQL, Pair RDD's, Spark YARN.
- Real time streaming the data using Spark with Kafka.
- Develop use case to monitor the efficiency of Spark real time processing.
- Exploring with the Spark improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, and Pair RDD's.
- Performed performance tuning for Spark Steaming e.g. setting right Batch Interval time, correct level of Parallelism, selection of correct Serialization & memory tuning.
- Developed Spark-SQL for faster testing and processing of data.
- Import the data from different sources like HDFS/HBase into Spark RDD developed a data pipeline using Kafka and Storm to store data into HDFS.
- Creating Hive tables and Increased the performance of the by using Partitions and Buckets for Hive table
- Using Snappy compression to Hive data.
- Performed complex Joins on the tables in Hive for generating the reports.
- Managed and reviewed Hadoop log files.
Environment: Hadoop Ecosystem: Spark, Scala, Python,Kakfa, Hive, HBase, Hive, PIG, Sqoop, Flume, Oozie, Zookeeper, Storm and MapReduce frameworks, Java 1.7, Unix.
Confidential, Hoffman Estates, IL
Sr. Big data Hadoop Engineer
Responsibilities:
- Developed MapReduce(YARN) jobs for cleaning, accessing and validating the data.
- Loaded the customer profiles data, customer claims information, billing information etc. into HDFS.
- Worked with highly unstructured and semi structured data
- Developed MapReduce programs and Solved performance issues.
- Used different file formats like Sequence files, Text Files
- Followed Agile methodology for the entire project.
- Managed and reviewed Hadoop log files.
- Performed unit testing using MRUnit
- Involved in fixing issues arising out of duration testing.
Environment: Hadoop Ecosystem: Hive, HBase, Hive, PIG, Sqoop, Flume, Oozie, Zookeeper and MapReduce frameworks, Java 1.7, Unix.
Confidential, Detroit, MI
Sr. Java/J2EE Developer
Responsibilities:
- Interacted with business managers to transform requirements into technical solutions.
- Followed Agile software development with Scrum methodology.
- Involved in Java, J2EE, Struts, Spring, Web Services and Hibernate in a fast paced development environment.
- Involved in adding IKA new line of business for the Confidential .
- Server side coding and development using Spring, Exception Handling, Java Collections including Set, List, Map, Spring, Hibernate, Webservices, etc in Windows & Linux environment.
- Involved in defect tracking as well as planning using JIRA.
- Resolved a complicate production issue for business managers, where the number of records where displaying wrong.
- Created and modified Struts actions.
- Worked with struts validations.
- Worked on Spring application framework features IOC container and AOP and integrated Spring with Hibernate using the HibernateTemplate.
- Developed enterprise inter-process communication frame work using Spring REST-ful Web Service.
- Developing SOAP Webservices and REST Webservices(JAXB, JSON, JAX-RS,JAX-WS)
- Developed Hibernate persistent layer.
- Implemented Spring MVC controllers like Controller, Simple Form Controller.
- Implemented Spring MVC framework in the presentation tier for all the essential control flow, business level validations and for communicating with the business layer.
- Created build/deployment scripts, using Ant /Maven.
- Taken care of multi threading in back end java beans.
- Conducted code reviews using eclipse plug-in for JProbe/JCR/PMD/Check-Style.
- Analyzing the issues and proactively identifying potential problem and its solution.
- Respond and close technical escalation from QA team and from the analysts, closing problems by trouble shooting and involving in root cause analysis.
- Developed Java Helper classes for Action Code and Work Assigned.
- Used Web Services (SOAP) for transmission of large blocks of XML data between two interfaces.
- Used Junit for Testing in a Test Driven Environment.
- Performed root-cause analysis of issues, some coding changes, and testing of changes, processes.
- Also involved in bug fixing.
Environment: Core Java, Servlet, Spring 3.0,Struts 2.0, Apache Axis2.0,Hibernate,Webservices,log4j, JIRA, JUnit, FLEX,JDBC, XML, XSTL,Websphere7.5.3, DB2, Windows XP, Linux, Oracle 10g.
Confidential
Sr. Java/J2ee Developer
Responsibilities:
- Involved in requirements gathering meetings for CORE Application.
- Server side coding and development using Spring, Exception Handling, Java Collections including Set, List, Map
- Involved in developing new connector for CareerBuilder, Rally, JobVite .
- Involved in understanding the API for the CareerBuilder, Rally, JobVite.
- Used ANT scripts to build JAR, WAR and EAR files.
- Application deployment on Tomcat Web Server & Weblogic application server.
- Involved in defect tracking as well as planning using JIRA.
- Developed enterprise inter-process communication frame work using Spring REST-ful Web Service.
- Developed UI work flow using Spring MVC
- Developing SOAP Webservices and REST Webservices(JAXB, JSON, JAX-RS,JAX-WS)
- Developed Hibernate persistent layer.
- Taken care of multi threading in back end java beans.
- Followed Agile software development with Scrum methodology.
- Involved in parsing XML/JSON for data received.
- Created all the services to work with the various entities provided and restified the services using REST APIs.
- Designed and Developed Servlets for authentication.
- Used Junit for Testing in a Test Driven Environment.
- Used Web Services (SOAP) for transmission of large blocks of XML data between two interfaces.
- Performed root-cause analysis of issues, some coding changes, and testing of changes, processes.
- Also involved in bug fixing.
- Developed the application using Eclipse as IDE tool.
Environment: /Platform: Core Java, Servlet, JSON,JDBC, Apache CXF,Spring 3.0, XML, XSTL,Websphere7.5.3, DB2, WebServices, Windows XP, Linux
Confidential, Charlotte, NC
Lead/Sr.Java Developer
Responsibilities:
- Involved in requirements gathering meetings for CORE Application
- Involved in analysis and define the proper error code for the user in WCC.
- Developed the Servlets, Java Beans, and DAO’s to handle the backend data.
- Involved in Java, J2EE, Struts, Spring, Web Services and Hibernate in a fast paced development environment.
- Developed enterprise inter-process communication frame work using Spring REST-ful Web Service.
- Developed UI work flow using Spring MVC
- Created build/deployment scripts, using Ant /Maven.
- Used Junit for Testing in a Test Driven Environment.
- Created and modified Struts actions.
- Worked with struts validations.
- Developed various helper classes needed following Core Java multi-threaded programming and Collection classes.
- Used multithreading in programming to improve overall performance.
- Involved in analysis and finding root cause for the interdependent apps teams.
- Involved in writing code for code IDV time out error with WCC.
- Used Data Access Object (DAO) design pattern to abstract and encapsulate data access from relational database.
- Analyzed the defects and implemented the functionality with Business Logic
- Done impact analysis of the application under development
- Prepared the design and technical documents for the enhancements
- Involved in production support (bug fixing) and performance tuning of the existing project.
- Build and Deployed Applications in Web Sphere and JBoss Application Server.
- Deployed the code in the test/prod environments.
- Provided required inputs/support to the QA.
Environment: Core Java, SOA,JDBC, JEE, Servlet, Struts2.0, Spring 2.5, XML, Apache CXF,XSTL,Websphere7.5.3, DB2, XML, WebServices, Windows XP, Linux, AIX, JBoss Application Server
Confidential
Lead/Sr. Java Developer
Responsibilities:
- Responsible for Gathering the system requirements, feasibility study, prototyping, and suggesting the best solutions to customers.
- Responsible for writing detailed functional and technical design documentation for the requirements owning and controlling the documents.
- Designed and developed framework components, involved in designing MVC pattern using Struts Framework.
- Involved in the Development, testing and maintenance phases of Software Development Life Cycle (SDLC).
- Developed new services like NAS, ADD SAN
- Involved in designing of NAS service workflow.
- Used multithreading in programming to improve overall performance.
- Use Clear case for code version management
- Involved in manual testing as part of the regular updates
- Involved in the maintenance work like CR(Change Request)
- Developed user interface using Struts, JSP, HTML, CSS and Java Script.
- Used Data Access Object (DAO) design pattern to abstract and encapsulate data access from relational database.
- Used Junit for Testing in a Test Driven Environment.
- Involved in the deployment of the code in prod.
Environment: Java, J2EE, Servlets, JSP, Struts, Java Beans, HTML, CSS, WebSphere, Apache Axis2.0 XML, XSTL, WebServices, DB2, AIX, Linux
Confidential
Lead /Sr.Java/J2EE Developer
Responsibilities:
- Developed use case models, UML class diagram and sequence diagram for Pricer Worklist
- Involved in gathering the system requirements and prototyping.
- Responsible for writing detailed functional and technical design documentation for the requirements.
- Followed Agile software development with Scrum methodology.
- Using Data Access Object (DAO) pattern, created classes for implementing the access logic to database using JDBC and calls to DB2
- Responsible for implementing business logic using Servlet and Struts.
- Created and modified Struts actions.
- Worked with struts validations.
- Involved in code reviews, test case reviews.
- Developing SOAP Webservices and REST Webservices(JAXB, JSON, JAX-RS,JAX-WS)
- Developed Hibernate persistent layer.
- Implemented Spring MVC framework in the presentation tier for all the essential control flow, business level validations and for communicating with the business layer.
- Used Junit for Testing in a Test Driven Environment.
- Used multithreading in programming to improve overall performance.
- Use eclipse extensively for code development and debugging.
- Responsible for effective communication between the project Team leads and clients.
- Involved in handling the tickets which have been raised by the clients.
- Involved in analysis of the code for trouble shooting.
- Analyzing and working on various issues raised by the customers and business people
- Proactively monitoring the processes and the tool availability.
- Involved in bug fixing and performance tuning of the existing project.
- Involved in the testing of the application.
Environment: Platform: Java1.5, Servlets1.4, Windows2000,Spring 2.5, DB2, RSA 7.5, WebSpere 7.5, Web Services, Apache Axis1.x.x,EAD4J Framework, Windows XP
Confidential
Lead /Sr. Developer
Responsibilities:
- Involved in gathering the system requirements and prototyping.
- Responsible for writing detailed functional and technical design documentation for the requirements.
- Using Data Access Object (DAO) pattern, created classes for implementing the access logic to database using JDBC and calls to DB2
- Responsible for designing and Coding for Case Folder.
- Responsible for writing detailed functional and technical design documentation for the requirements.
- Responsible for implementing business logic using Servlet and Java.
- Involved in code reviews, test case reviews.
- Involved in resolving the DRs.
- Involved in code reviews, test case reviews.
- Use eclipse extensively for code development and debugging.
- Responsible for effective communication between the project Team leads and clients.
- Involved in handling the tickets which have been raised by the clients.
- Involved in analysis of the code for trouble shooting.
- Analyzing and working on various issues raised by the customers and business people
- Involved in bug fixing and performance tuning of the existing project.
- Involved in the testing of the application.
Environment: /Platform: Java1.5, Servlets1.4, Windows2000, SQL Server, RSA 7.5, WebSpere 7.5Web Services, PanDOORA Framework, Windows XP
Confidential
Lead /Sr.Java Developer
Responsibilities:
- Work with Business Analyst in translating business requirements into Functional Requirements Document and to Detailed Design Documents
- Responsible for writing detailed functional and technical design documentation for the requirements.
- Implement the Model View Controller (MVC) design pattern using J2EE, JSP, Servlets to control the
- Responsible for writing detailed functional and technical design documentation for the requirements.
- Ensure that development is performed as per requirements
- Involved in code reviews, test case reviews.
- Use eclipse e xtensively for code development and debugging.
- Responsible for effective communication between the project Team leads and clients.
- Involved in analysis of the code for trouble shooting.
- Analyzing and working on various issues raised by the customers and business people
- Involved in bug fixing and performance tuning of the existing project.
- Involved in the testing of the application.
Environment: /Platform: Java 1.5, Struts 1.2, Informix DB, Servlets, Windows XP, JBOSS4.0.4.GA
Confidential
J2EE Developer
Responsibilities:
- Work with Business Analyst in translating business requirements into Functional Requirements Document and to Detailed Design Documents
- Responsible for writing detailed functional and technical design documentation for the requirements.
- Implement the Model View Controller (MVC) design pattern using J2EE, JSP, Servlets to control the
- Responsible for writing detailed functional and technical design documentation for the requirements.
- Ensure that development is performed as per requirements
- Involved in code reviews, test case reviews.
- Use eclipse extensively for code development and debugging.
- Deployed of the code in production servers.
Environment: /Platform: Servlets,Tomcat 5.0, Log4j, XHTML, Java1.4, MySQL 5.0.27,WAP, Windows XP
Confidential
J2EE Developer
Responsibility:
- Developing, maintaining and updating projects Single Level Login, Secret Garden E-Commerce Application .
- Deployed of the code in production servers.
- Involved in code reviews, test case reviews
Environment: /Platform: Struts1.0, JDBC, MS Access, Core Java1.4.2, Windows XP