Sr. Big Data Engineer Resume
Foster City, CA
SUMMARY:
- Over 9 plus years of experience in IT industry which includes Big Data Technologies and Java, J2EE Technologies
- Good hands - on experience in Apache Spark with Scala
- Excellent Java development skills using J2EE Frameworks like Spring, Hibernate, Web Services and RestFul Web Services, Micro-services.
- Experience on Hadoop, Map Reduce, Yarn, Strom, Hive, Pig, HBase, Kafka, Oozie, Zookeeper, Splunk, Scoop, MongoDB.
- Experience in writing MAPREDUCE programs in java for data cleansing and preprocessing.
- Experience in working with Flume to load the log data from multiple sources directly in to HDFS
- Familiar with importing and exporting data using Sqoop
- Experience in managing and reviewing Hadoop Log files.
- Experience in working on various Spring modules (Spring MVC, DAO and AOP) along with Hibernate as the back-end ORM tool for implementation of persistence layer and mapping of POJOs.
- Experienced with Java Multithreaded programming to develop multithreaded modules and applications.
- Extensive experience in different IDEs like RAD, Eclipse, NetBeans and JBuilder.
- Worked on different Application and Web Servers like WebLogic 7.x, IBM Web Sphere 6.1/7.0,8.0, BEA Web Logic 9.0/10.0, Apache Tomcat and JBoss.
- Experience in relational and Object oriented database design and implementation using Oracle 8i/9i/10g, MySQL and writing queries in SQL
- Effective independent and team player and self-starter with high adaptability to new technologies.
- Have excellent analytical, problem solving and communication skills.
- A Team Player with good interpersonal skills, decision-making and committed to quality
TECHNICAL SKILLS:
Programming Languages:: Java,Scala
J2EE Technologies: Servlets, JSP, JSTL, JMS, JDBC, Web Services, SOAP, WSDL, Restful, Pivotal-Cloud, ANT, JAXB, Spring 4.0, Hibernate 3.0 ORM
Bigdata Ecosystem: Hive, HBase, Spark, PIG, Sqoop, Flume, Oozie, Zookeeper, Kafka, Storm and MapReduce frameworks.
Web/Applications Servers: IBM WebSphere 5.x/6.x/7.x/8.x, WebLogic 7.x, Tomcat 4.x/5.x, JBoss 4.x
J2EE IDEs: IBM WebSphere Studio (WSAD) 4.x/5.x, Eclipse 4, JBuilder 2006,RAD 6.0
Database and Related: Oracle 8i/9i/10g,DB2,SQL, PL/SQL
Operating Systems: Windows NT/2000/XP/2003 Servers, UNIX, Linux
Version Control & tools: GITHUB, CVS,SVN
PROFESSIONAL EXPERIENCE:
Confidential, Foster City, CA
Sr. Big data EngineerResponsibilities:
-
Created Spark jobs to see trends in data usage by users.
- Developed Spark code to using Scala and Spark-SQL for faster processing and testing.
- Developed a data pipeline using HBase, Spark and Hive to ingest, transform and analyzing customer behavioral data
- Experienced in working with Spark eco system using SCALA and HIVE Queries on different data formats like Text file and parquet
- Process data through Kafka and verify the same in Spark as experiment
- Involved in developing Spark using Scala and Spark-SQL for faster processing of data and exploring of optimizing it using Spark Context, Spark-SQL, Pair RDD's, Spark YARN.
- Real time streaming the data using Spark with Kafka.
- Performed performance tuning for Spark Steaming e.g. setting right Batch Interval time, correct level of Parallelism, selection of correct Serialization & memory tuning.
- Developed Spark-SQL for faster testing and processing of data.
- Import the data from different sources like HDFS/HBase into Spark RDD developed a data pipeline using Kafka to store data into HDFS.
Environment: Big data Ecosystem: Spark 2.0, Scala, Python, Kakfa, Hive, HBase, Hive, PIG, Oozie, Zookeeper, Storm and Java 1.8, Unix.
Confidential, Denver, CO
Sr. Big data Engineer
Responsibilities:
-
Used Spark API over Hadoop YARN to perform analytics on data in Hive.
- Process data through Kafka and verify the same in Spark as experiment
- Involved in developing Spark using Scala and Spark-SQL for faster processing of data and exploring of optimizing it using Spark Context, Spark-SQL, Pair RDD's, Spark YARN.
- Real time streaming the data using Spark with Kafka.
- Develop use case to monitor the efficiency of Spark real time processing.
- Exploring with the Spark improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, and Pair RDD's.
- Performed performance tuning for Spark Steaming e.g. setting right Batch Interval time, correct level of Parallelism, selection of correct Serialization & memory tuning.
- Developed Spark-SQL for faster testing and processing of data.
- Import the data from different sources like HDFS/HBase into Spark RDD developed a data pipeline using Kafka and Storm to store data into HDFS.
- Creating Hive tables and I Confidential eased the performance of the by using Partitions and Buckets for Hive table
- Using Snappy compression to Hive data.
- Performed complex Joins on the tables in Hive for generating the reports.
- Managed and reviewed Hadoop log files.
Environment: Hadoop Ecosystem: Spark, Scala, Python,Kakfa, Hive, HBase, Hive, PIG, Sqoop, Flume, Oozie, Zookeeper, Storm and MapReduce frameworks, Java 1.7, Unix.
Confidential, Hoffman Estates, IL
Sr. Big data Hadoop Engineer
Responsibilities:
-
Developed MapReduce(YARN) jobs for cleaning, accessing and validating the data.
- Loaded the customer profiles data, customer claims information, billing information etc. into HDFS.
- Worked with highly unstructured and semi structured data
- Developed MapReduce programs and Solved performance issues.
- Used different file formats like Sequence files, Text Files
- Followed Agile methodology for the entire project.
- Managed and reviewed Hadoop log files.
- Performed unit testing using MRUnit
- Involved in fixing issues arising out of duration testing.
Environment: Hadoop Ecosystem: Hive, HBase, Hive, PIG, Sqoop, Flume, Oozie, Zookeeper and MapReduce frameworks , Java 1.7, Unix.
Confidential, Detroit, MI
Sr. Java/J2EE Developer
Responsibilities:
-
Interacted with business managers to transform requirements into technical solutions.
- Followed Agile software development with Scrum methodology.
- Involved in Java, J2EE, Struts, Spring, Web Services and Hibernate in a fast paced development environment.
- Involved in adding IKA new line of business for the Nterflow.
- Server side coding and development using Spring, Exception Handling, Java Collections including Set, List, Map, Spring, Hibernate, Webservices, etc in Windows & Linux environment.
- Involved in defect tracking as well as planning using JIRA.
- Resolved a complicate production issue for business managers, where the number of records where displaying wrong.
- Created and modified Struts actions.
- Worked with struts validations.
- Worked on Spring application framework features IOC container and AOP and integrated Spring with Hibernate using the HibernateTemplate.
- Developed enterprise inter-process communication frame work using Spring REST-ful Web Service.
- Developing SOAP Webservices and REST Webservices(JAXB, JSON, JAX-RS,JAX-WS)
- Developed Hibernate persistent layer.
- Implemented Spring MVC controllers like Controller, Simple Form Controller.
- Implemented Spring MVC framework in the presentation tier for all the essential control flow, business level validations and for communicating with the business layer.
- Created build/deployment scripts, using Ant /Maven.
- Taken care of multi threading in back end java beans.
- Conducted code reviews using eclipse plug-in for JProbe/JCR/PMD/Check-Style.
- Analyzing the issues and proactively identifying potential problem and its solution. Respond and close technical escalation from QA team and from the analysts, closing problems by trouble shooting and involving in root cause analysis.
- Developed Java Helper classes for Action Code and Work Assigned.
- Used Web Services (SOAP) for transmission of large blocks of XML data between two interfaces.
- Used Junit for Testing in a Test Driven Environment.
- Performed root-cause analysis of issues, some coding changes, and testing of changes, processes.
- Also involved in bug fixing.
Environment: Core Java, Servlet, Spring 3.0,Struts 2.0, Apache Axis2.0,Hibernate,Webservices,log4j, JIRA, JUnit, FLEX,JDBC, XML, XSTL,Websphere7.5.3, DB2, Windows XP, Linux, Oracle 10g.
Confidential, American Fork, UT
Sr. Java/J2ee Developer
Responsibilities:
-
Involved in requirements gathering meetings for CORE Application.
- Server side coding and development using Spring, Exception Handling, Java Collections including Set, List, Map
- Involved in developing new connector for CareerBuilder, Rally, JobVite .
- Involved in understanding the API for the CareerBuilder, Rally, JobVite.
- Used ANT scripts to build JAR, WAR and EAR files.
- Application deployment on Tomcat Web Server & Weblogic application server.
- Involved in defect tracking as well as planning using JIRA.
- Developed enterprise inter-process communication frame work using Spring REST-ful Web Service.
- Developed UI work flow using Spring MVC
- Developing SOAP Webservices and REST Webservices(JAXB, JSON, JAX-RS,JAX-WS)
- Developed Hibernate persistent layer.
- Taken care of multi threading in back end java beans.
- Followed Agile software development with Scrum methodology.
- Involved in parsing XML/JSON for data received.
- Created all the services to work with the various entities provided and restified the services using REST APIs.
- Designed and Developed Servlets for authentication.
- Used Junit for Testing in a Test Driven Environment.
- Used Web Services (SOAP) for transmission of large blocks of XML data between two interfaces.
- Performed root-cause analysis of issues, some coding changes, and testing of changes, processes.
- Also involved in bug fixing.
- Developed the application using Eclipse as IDE tool.
Environment: Core Java, Servlet, JSON,JDBC, Apache CXF,Spring 3.0, XML, XSTL,Websphere7.5.3, DB2, WebServices, Windows XP, Linux
Confidential, Charlotte, NC
Lead/Sr.Java Developer
Responsibilities:
-
Involved in requirements gathering meetings for CORE Application
- Involved in analysis and define the proper error code for the user in WCC.
- Developed the Servlets, Java Beans, and DAO’s to handle the backend data.
- Involved in Java, J2EE, Struts, Spring, Web Services and Hibernate in a fast paced development environment.
- Developed enterprise inter-process communication frame work using Spring REST-ful Web Service.
- Developed UI work flow using Spring MVC
- Created build/deployment scripts, using Ant /Maven.
- Used Junit for Testing in a Test Driven Environment.
- Created and modified Struts actions.
- Worked with struts validations.
- Developed various helper classes needed following Core Java multi-threaded programming and Collection classes.
- Used multithreading in programming to improve overall performance.
- Involved in analysis and finding root cause for the interdependent apps teams.
- Involved in writing code for code IDV time out error with WCC.
- Used Data Access Object (DAO) design p Confidential ern to abstract and encapsulate data access from relational database.
- Analyzed the defects and implemented the functionality with Business Logic
- Done impact analysis of the application under development
- Prepared the design and technical documents for the enhancements
- Involved in production support (bug fixing) and performance tuning of the existing project.
- Build and Deployed Applications in Web Sphere and JBoss Application Server.
- Deployed the code in the test/prod environments.
- Provided required inputs/support to the QA.
Environment: Core Java, SOA,JDBC, JEE, Servlet, Struts2.0, Spring 2.5, XML, Apache CXF,XSTL,Websphere7.5.3, DB2, XML, WebServices, Windows XP, Linux, AIX, JBoss Application Server
Confidential
Lead/Sr. Java Developer
Responsibilities:
-
Responsible for Gathering the system requirements, feasibility study, prototyping, and suggesting the best solutions to customers.
- Responsible for writing detailed functional and technical design documentation for the requirements owning and controlling the documents.
- Designed and developed framework components, involved in designing MVC p Confidential ern using Struts Framework.
- Involved in the Development, testing and maintenance phases of Software Development Life Cycle (SDLC).
- Developed new services like NAS, ADD SAN
- Involved in designing of NAS service workflow.
- Used multithreading in programming to improve overall performance.
- Use Clear case for code version management
- Involved in manual testing as part of the regular updates
- Involved in the maintenance work like CR(Change Request)
- Developed user interface using Struts, JSP, HTML, CSS and Java Script.
- Used Data Access Object (DAO) design p Confidential ern to abstract and encapsulate data access from relational database.
- Used Junit for Testing in a Test Driven Environment.
- Involved in the deployment of the code in prod.
Environment: Java, J2EE, Servlets, JSP, Struts, Java Beans, HTML, CSS, WebSphere, Apache Axis2.0 XML, XSTL, WebServices, DB2, AIX, Linux