Sr.big Data Engineer Resume
Kansas, MI
SUMMARY:
- 10+ years of experience in software development with deep business acumen and technical expertise in Big Data technologies.
- Hands - on experience in working with Big Data technologies i.e. HDFS, MapReduce, HBase & Hive, Oozie, Sqoop, Spark, Kafka.
- Deep Knowledge in Hadoop architecture and various components such as HDFS, Map Reduce and other Ecosystem components.
- Worked on all phases of data warehouse development life cycle, ETL design & implementation & support of new & existing applications.
- Experience in Git (distributed version control system).
- Experience integrating Jenkins with spring boot for continuous delivery.
- Experience in building Spark Scripts using Python.
- Experience in Elastic Search.
- Worked on AWS cluster with six nodes.
- Certified IBM Info Sphere Streams Technical Professional (000-N08 v1).
- Certified SPLUNK Sales Representative.
- Experience in creating and validating blocks in block chain using JavaScript.
- Experience in creating IOT (Internet of Things) based applications using Python on Arduino IDE.
- Experience in working RESTful web services.
- Expertise in designing and developing enterprise applications for J2EE platform using JAVA (JDBC, Servlets, JSP), Angular, Hibernate, Spring, HTML, JavaScript, AJAX, XML and Oracle.
- Experience in working with build tools like Maven and Gradle
- Expertise in various frameworks: JSF, spring and ORM tools like Hibernate 2.0/3.0.
- Strong hold on OOPS concepts and Web-based technologies with good understanding of various phases such as Requirements, Analysis, Design, Development and Testing.
- Extensive experience in deploying and working with Tomcat, JBOSS and Node.js.
- Hands on experience in implementing the application logic using Spring Boot, MVC.
- Significant experience with design pattern implementation (MVC, Session Façade, Singleton).
- Having experience in using Oracle, SQL and MySQL.
- Excellent communication, interpersonal, analytical skills and strong ability to perform in a team.
- Understanding, flexible, and adopt at working in high paced environments, adapting quickly to different business organization’s needs.
- Recognized for on-time Planning, Development and Delivering Quality Solutions.
TECHNICAL SKILLS:
Operating Platforms: Windows Family, Linux, Horton works, Cloudera
Big Data: HDFS, Map Reduce, Hive, Oozie, Kafka, Spark, IBM INFOSPHERE
Streams, Elastic Search, Splunk
NOSQL: Mongo DB, HBase
Languages: Java, Python, Scala, SPL (Streams Programming Language)
Frameworks: Spring, Spring Boot, Angular and Hibernate
RDBMS: Oracle, MySQL, MS-SQL Server
Servers: Jakarta-Tomcat, Jetty, Node.js and JBOSS
IDE s/Tools: SpringToolSuite, eclipse, visual studio code, Streams Studio
PROFESSIONAL EXPERIENCE:
Confidential, NJ
Sr.Big Data Engineer
Responsibilities:
- Analyzing the existing system process.
- Identifying the business-critical Measures by closely working with the SME.
- Streamlined the migration process from the existing system to Big Data architecture.
- Worked on ETL using Spark, Spark Streaming, Hive, HBase, and Oozie on Hadoop.
- Processing of incoming files using Spark native API, using Spark scripts
- Developed Python scripts using both Data frames/SQL and RDD in Spark 1.x/2.x for Data Aggregation.
- Performed significant role in upgrading the system to Spark 2.0 with Data frames and optimizing the jobs to best utilize of Tungsten Engine.
- Handled large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformations and other during ingestion process itself
- Designed & implemented HBase tables, Hive scripts.
- Experienced in managing and reviewing Spark log files for troubleshoot & debug
- Written Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data
- Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs and then upgrading to Data frames
- Written complex UDFs to handle the various multiple missing functionalities in Hive for analytics
- Performed various optimization techniques on Hive QL scripts to use HDFS & processing resources efficiently
- Designed various dimension tables using HBase and written scripts to automate the data loading to dimension tables
- Worked closely on increasing system performance by reducing the I/O by identifying the process gaps and tuning the queries
- Designed workflows & coordinators for the task management and scheduling using Oozie to orchestrate the jobs
- Developed Automation scripts using UNIX/Python which incorporates the business process for data processing
- Written cron jobs to handle performance checks in the files system and data
- Involved in writing parsers using Python
- Worked with requirements team to calculate the complex KPIs
- Worked with admin team to collaboratively resolve the issues, wherever required.
- Ensuring the adherence to the Agile Project Management methodologies and practices
- Involved in scheduling Oozie jobs
- Involved in Unit Testing
Environment: Horton works HDP, Java, HDFS, Map Reduce, Hive, Oozie,PySpark,Scala, Tableau.
Confidential, Kansas,MI
Sr.Big Data Engineer
Responsibilities:
- Involved in developing API’s to fetch the data from Hadoop using Spring Boot Rest Controller using Java
- Implemented service classes to fetch data from API
- Involved in designing Reporting UI using HTML and CSS.
- Involved in fetching data from API and plot the data on charts using angular.
- Involved in form validation using Angular
- Involved in creating charts using chart.js
- Involved in Unit Testing
- Involved in peer Testing
Environment: Horton works HDP, Java, Spring Boot, Angular, JavaScript, Node.js, Chart.js, HTML and CSS
Confidential, NJ
Sr.Software Engineer
Responsibilities:
- Involved in developing API’s to fetch the data from Database using Spring Boot Rest Controller
- Implemented http service to fetch data from API using angular http module
- Involved in creating charts using chart.js
- Involved in Unit Testing
- Involved in peer Testing
- Delivering a complete front-end application using Angular 2
- Ensuring a seamless high-performance application on mobile and desktop
- Writing tested, idiomatic, and documented JavaScript, HTML and CSS
- Building RESTful API (JSON) frontend
- Code reusable templates
Environment: Horton works HDP, Teradata, Java, Spring Boot, Tableau, JBOSS and Angular
Confidential
SPL Developer
Responsibilities:
- Involved in coding by using SPL (Streams Programming Language).
- The GUI was designed by using HTML, CSS, JavaScript and High Charts.
- Involved in Unit Testing.
- Used Jetty Server to develop, test and deploy all the SPL components.
Environment: StreamsStudio-3.0, IBM INFOSPHERE Streams, SPL, Jetty, JavaScript, VOCI Speech to Text Machine and High Charts
Confidential
Java Developer
Responsibilities:
- Involved in creating action classes using struts.
- The GUI was designed on the base of MVC design-patterns.
- Involved in Designing and Coding.
- Responsible for developing Application Front end using JSP’s.
- Used Struts tag libraries in the JSP pages.
- Used Tomcat Server to develop, test and deploy all the Java components.
- Used WEBLOGIC Application server to test the application.
- Involved in the System integration testing and regression testing of the implemented.
Environment: Servlets, JSP, JDBC, Struts, Tomcat and Oracle
Confidential
Java Developer
Responsibilities:
- Understanding existing business model and client requirements.
- Implemented the JSP to design the user-friendly front-end screens.
- Involved in the development of the form pages using JSP.
- Implemented the Java Servlets to process the client request on server side.
- Implementing the Persistence logic using JDBC
- Application Development and Implementation.
- Created configuration files in XML for deployment descriptors.
- Involved in Unit Testing for developed components.
Environment: Servlets, JSP, JDBC, Tomcat and MYSQL
