We provide IT Staff Augmentation Services!

Big Data Engineer Resume

OrlandO

SUMMARY

  • An astute IT professional with 11 Years of experience in IT Industry.
  • 4+ years of Experience in Big Data Hadoop echo systems - Hadoop, Spark, Machine Learning, Scala Yarn, Pig, Hive, HBase, Akka, Oozie, Flume, Zoo Keeper.
  • Around 6 Years of Core Java, Python, Web Services Experience in traditional and distributed environments.
  • Experience with messaging & Complex event processing systems using Kafka and Spark Streaming.
  • Extensively worked on Spark Core, Spark Streaming and Spark SQL and Spark ML Modules using both Scala and Java.
  • Experience in importing and exporting the data using Sqoop/Flume from HDFS to Relational Database systems/Logs and vice-versa.
  • Good Knowledge on Hadoop Analytics using R.
  • Collected logs data from various sources and integrated in to HDFS using Flume.
  • Developed Oozie workflow for scheduling and orchestrating the ETL process.
  • Extensively worked with Kerberos secured clusters on Hadoop platform.
  • Worked with different data formats like ORC, Parquet, Avro etc.
  • Experience in multiple java technologies like JDBC, Servlets, Quartz Scheduler, EJBs, JNDI, JMS, Guava APIs, Apache Commons.
  • Experience in implementing MVC, Singleton, Session Fa ade, DAO, DTO, Front Controller, Business Delegate and Factory Method patterns.
  • Experience in developing and consumingRest and Soap Web Services.
  • Worked on building continuous integration and test driven development environments.
  • Worked on Apache NIFI to stream line the data flow.
  • Good knowledge on Spark Mlib, Python.
  • Good Knowledge on AWS Cloud QuickSight, AWS S3, RedShift, AWS EMR and Athena and EC2.
  • Worked on Development, Enhancement, and production support on different technologies like Big data, Oracle, peopleSoft and core Java, Rest and SOAP Web Services.
  • Have created Oracle Pl/sql procedures, functions and triggers.
  • Oracle Pl/sql trainer for High Tech Account employees.
  • Experience on High Availability and high traffic applications.
  • Worked on PeopleSoft Development.
  • Oracle 10g OCA DBA Certified. OCP DBA Certification (Exam).
  • Excellent Knowledge of UNIX and shell scripting, Auto sys jobs creation.
  • Experience analyzing logs using Splunk queries.
  • Experienced in release management and release process improvements.
  • Strong analytical skills, disciplined delivery standards and dedication to develop and enjoy making high quality software systems.

PROFESSIONAL EXPERIENCE

Confidential, Orlando

Big Data Engineer

Responsibilities:

  • Created Big Data pipeline using Kafka, Spark to ingest different events from VGS system into HDFS for analysis.
  • Store ticketing archival data in the Data Lake and generate Reports.
  • Implemented Datamart along with analytical reporting capabilities in Hdfs.
  • Developed Kafka publisher to fetch Sales, Product, Usage Events from Rabbit MQ to Kafka
  • Developed Spark Streaming subscriber to fetch data from Kafka Topics to HDFS.
  • DevelopedSparkPrograms for Data validations and to implement business logic and worked on Spark Tuning.
  • Created Hive External tables/views on top of Events data for reporting purpose.
  • Discussions with Business to plan Report generation and exection.
  • Discussions with Architects to choose right technology for each requirement.
  • Hive Performance Tuning on various parameters and configurations.
  • Hive Report generation for Business Objects.
  • Extensively worked on Json and ORC file formats for Hive storage.
  • Worked on other Horton works components like Zeppeline.
  • Extensively used Snappy and mc4 compressions.
  • The environment is secured with Kerberos authentication, SSL/SSL SASL.
  • Java API to store and validate the duplicate events using Hbase
  • Storing and accessing kafka offsets on Hbase tables.
  • Python scripts to connect hive through ODBC and run Hive queries.
  • Spark scheduler Jobs to concatenate small orc files.

Environment: Horton works, Hive, Spark, Kafka, Scala, core java, Hbase, NIFI, Business Objects, Python, rest Web Services, uc4, AWS,Kerberos, Business Objects.

Confidential, NJ

Big Data Engineer

Responsibilities:

  • Created Big Data pipeline, to fetch different source data from different sources like ENodes, LMS, Custer mobile log data to HDFS though various approaches like Flume, sqoop, Server copy etc.
  • Data retrieval from HDFS, hive using Spark Sql with Scala.
  • Implemented Spark SQL to access hive tables into spark for faster processing of data.
  • Implemented Flume data consumption APIS using Java.
  • Developed a solution using Spark, Spark SQL using Scala for analyzing the intermediate data from HDFS and Mongo DB.
  • Created Flume interfaces and worked on Flume configuration to load the raw data into HDFS and Mongo DB, that is coming from enodebs, MMECSL and UEAPP and other publishers.
  • Used Sqoop for importing and exporting data from Oracle Database into HDFS and Hive.
  • Designed and Created Hive databases, tables and views. Worked onHive Performance tuning and created UDFs.
  • Incremental data movement using sqoop and Oozie jobs.
  • Implemented Spark SQL to access hive tables into spark for faster processing of data.
  • Involving on designing the application.
  • Fixing the bugs and implementing enhancements.

Environment: HDFS, Cloudera, Flume, sqoop, Hive, Spark, Scala, Mongo DB, jcraft, p4c, chef, core java, Apache Commons, Servlets, Quartz job scheduler, Guava APIs, Python, rest Web Services, Kerberos, Angular Js, Java Script, java, high charts.

Confidential, CA

Big Data Engineer

Responsibilities:

  • Created data pipeline, to fetch different source data from different sources to HDFS though Kafka.
  • Incremental data movement using sqoop and Oozie jobs.
  • Worked on Hive Optimization.
  • Developed Spark Streaming APIs using scala.
  • Developed Kafka Producer Service in Java, using Kafka Publisher APIs.
  • Designed and developed real time event processing of data from multiple sources using Spark Streaming by integrating with Apache Kafka and Zookeeper, Flume, Akka.
  • Responsible for data movement from existing relational databases, Teradata, external websites to HDFS using sqoop and flume.
  • Have created Kafka producer and consumers using Java, oracle Pl/sql.
  • Developed Hive UDFs using Java.
  • Developed Oozie workflow for scheduling and orchestrating the ETL process
  • Production cutover activities and migration of the changes to Production.
  • Customer interactions on a daily basis - for Knowledge Transfer, status Reporting, clarifications.
  • Getting signoff from business users on completion of User Acceptance Testing phase of the project.
  • Business Discussions and Onshore, offshore co-ordination.

Environment: HDFS, Yarn, Unix, Hadoop core, Spark, Scala, Java, Map Reduce 2, Sqoop, Flume, Oozie, Pig and Hive, Akka, Kafka, Eclipse, Github.

Confidential, CA

Java Developer

Responsibilities:

  • Designing and developing the CSX Application for providing common REST and SOAP services for all AppleCare applications like Mobile Genius, iLog, online applications and Service Modules.
  • Working with different application teams for design and integration on the new CSX Application with different dependent systems.
  • Discussions with business, to gather requirements and Design the new Tool with several common services, which replaces the individual services across Confidential Care Modules.
  • Implementation of REST, SOAP APIs.
  • UsedSOAP UI toolto create test cases and quickly unit test the web service code.
  • Complete involvement in D2P for the Services.
  • Supporting the application, which went live and fixing the bugs/enhancing the services.
  • UsedANTas build tool and developed build file for compiling the code of creatingWAR files.
  • Production cutover activities, Planning and Deployment of the CSX Services to Production.
  • Usedconfluencefor managing the project related notes and docs.
  • Created PL/SQL scripts, pl/sql code enhancements and query plan analysis and tuning.
  • Working with business to support new Setups, Scheduling jobs, and enhancements for the existing services.

Environment: IBM AIX Servers, OSX Server, Mountain Lion, Unix, Java, J2EE, Ajax, Spark, Akka, REST, SOAP Web Services, Ant, Jetty Web Services, Scala, Oracle Pl/sql, Splunk, Fog Light, Oracle pl/sql, Radar, Espresso, Service-Now, Mongo DB, SQL developer, Xcode, Eclipse

Confidential, CA

Production Support Analyst

Responsibilities:

  • Supporting Confidential Care applications like Mobile Genius, ILog, online applications and Service Modules.
  • Working on the support and maintenance of the above applications and data fixes.
  • Working with Application team for deployments on the new enhancements to the application.
  • Attending meetings with business, to give a report on the system health, performance and will provide a report on the current issues and investigation status.
  • Creating monitoring tools to raise alerts if any functionality breakages in the system.
  • Working with business to support new product installations, and expansion projects for data setups and end-to-end golden data transactions.
  • Working on the scheduled job setting changes for Daylight saving changes.
  • Active interaction with the teams integrated to Mobile Genius, iLog, and Online Apps and SM applications.
  • Production cutover activities and migration of the changes to Production.

Environment: Mac OSX Servers, IBM AIX Servers, Java, J2EE, Ajax, Web Services, Scala, Oracle Pl/sql, Splunk, Fog Light, Mac OSX Yosemite, Mountain Lion, Unix, Radar, Espresso, SQL developer, Eclipse.

Confidential, CA

Java Developer

Responsibilities:

  • Analysis of the Requirements and Effort estimation of enhancements.
  • Designing the final requirements by analyzing the existing system and business objects involved.
  • Developed REST and SOAP web services to interact with external systems.
  • Usedspringfor Dependency Injection for plugging in theHibernateDAO objects for the business layer.
  • Technical and functional mentor to the team.
  • Review and modification of test cases and monitoring of the results.
  • Getting signoff from business users on completion of User Acceptance Testing phase of the project.
  • Production cutover activities and migration of the changes to Production.
  • Go Live and Post Production Support.

Environment: Mountain Lion, Unix, Mac OSX Servers, IBM AIX Servers, REST and SOAP Web Services, Radar, SQL developer

Confidential, CA

Peoplesoft Developer

Responsibilities:

  • Created new Synchronous API’s to get request from Frontend UI Using Integration broker.
  • Prepared Technical and functional design documents of the same
  • Efforts estimated and provided the timelines for the same.
  • Documentation of test cases and execution of test cases.
  • Performed unit testing and load testing on the application.
  • Tuned the code and Oracle queries for performance gain.
  • Developing the Oracle Procedures, Packages, and Triggers.
  • Production cutover activities and migration of changes to Production.
  • Warranty Support.

Environment: People Code, Application Designer, Mac OSX Servers, IBM AIX Servers, Mountain Lion, Unix, Radar, Espresso, SQL developer

Confidential, CA

Oracle pl/sql Developer

Responsibilities:

  • Developing the Oracle Procedures, Packages, and Triggers.
  • Support for Integration and User acceptance testing.
  • Interacting with external teams for setting up the connectivity between the systems for different interfaces.
  • Cutover activities and production migration.
  • Production support.

Environment: Mountain Lion, Unix, IBM AIX Servers, Radar, SQL developer, pl/sql

Hire Now