Sr. Bigdata Engineer Resume
Eden Prairie, MN
SUMMARY
- Sr. Bigdata Engineer with nearly 14 yearsofsoftware development experience in Healthcare, Manufacturing and Retail domain.
- Nearly 4 years in developing Bigdata solutions with Java and Scala on Hadoop ecosystem and over 10 years in NetWeaver BI, Portal, and Java Technologies.
- Hands on experience with Hadoop eco system (Map reduce, Pig, Hive, Sqoop, HBase, Flume, Oozie, Zookeeper, Spark Streaming, Spark SQL using Scala.)
- Strong knowledge on Hadoop HDFS architecture and Map - Reduce framework.
- Hands on experience in analyzing data using Hive SQL, Pig Latin, and MapReduce programs.
- Hands on experience in NoSQL databases like HBase, Cassandra.
- Good Understanding of Cloud Computing like AWS, Google Cloud and Azure.
- Managed full lifecycle application development and demonstrated ability in delivering solutions that include architecture, analysis, programming, design, product installation, testing and support.
- Communicate effectively with both technical and non-technical project stakeholders.
- Versed in both agile and waterfall development techniques.
TECHNICAL SKILLS
Hadoop/Big Data: MapR & CDH, HDFS, MapReduce, Pig, Hive, YARN, Sqoop, Flume, Spark, Confluent Kafka (5.0)
Scripting Language: Scala, PowerShell
Java & J2EE technologies: Core Java, Servlets, JDBC, EJB, Web services.
SAP: SAP HANA, SAP UI5, XS Engine, SAP BI, Java WebDynpro, ECC, SAP MDM.
Databases: Oracle, MySQL, Cassandra
PROFESSIONAL EXPERIENCE
Confidential, Eden Prairie MN
Sr. Bigdata Engineer
Responsibilities:
- Involve in SCRUM planning, Sprinting and daily Scrum calls to discuss teh progress and challenges in teh sprints.
- Replatform existing ETL from DataStage to Hadoop.
- Provide architectural solutions for teh replatform activities.
- Design and Develop a process to load data into MapR Eco System from RDBMS and flat files for Structured Data.
- Import large volumes of data using Sqoop from Oracle, MySQL to Hive and HBase.
- Design and DevelopStreamingapplications with Scala for incremental data load from confluent Kafka topics to Hive and HBase.
- Perform cleaning and filtering on imported data using Hive and Map Reduce.
- Develop Common Data Model to perform transformations and load data from multiple sources systems.
- Develop Spark applications with spark-sql to perform in-memory data computation and apply transformations to load data into Hive and HBase Tables.
- Install and Configure Confluent Kafka OSS cluster.
- Configure monitoring and alerting for confluent Kafka with Prometheus.
- Configure and secure data in Hive and HBase Tables with POSIX Groups.
- Design and implement data model with PHI and PI data restrictions according to teh company policy.
- Configure Jenkins for CI/CD Pipeline.
- Provide solutions to team on challenges faced.
Environment: MapR Hadoop 6.0, ApacheSpark 2.2.1, Yarn, Hive, Pig, Sqoop, Flume, Confluent Kafka 5.0, Zookeeper, Java, Scala, TWS, Oozie, MapRFS, MapRDB, HBase.
Confidential, NJ
Sr. Hadoop Developer
Responsibilities:
- Actively Involved in SCRUM planning and discuss teh development/progress of teh sprints.
- Install and Configure Hadoop Eco System.
- Exploring withSpark toimprove teh performance and optimization of teh existing algorithms in Hadoop usingSparkcontext,Spark-SQL, Data Frame, pair RDD’s,SparkYARN.
- DevelopedSparkapplications for faster processing in Hadoop eco system.
- Design and DevelopSpark Streamingapplications using Scala from Kafka to process nearly real time data in Hadoop eco system.
- Perform transformations, cleaning and filtering on imported data using Hive and Map Reduce.
- Load data intoSparkRDD and perform in-memory data computation to generate teh output response.
- Create Data Refinement Pig Scripts and Hive Queries.
- Create Data pipeline with produces and consumers for data streaming using Kafka.
- Hands on experience in runningHadoopstreaming jobs to process terabytes of xml format data. Load and transform large sets of structured, semi structured and unstructured data.
- Hands on experience with handling administration activities using Cloudera manager.
- Support MapReduce programs running in teh cluster environment.
- Used Scala collection framework to store and process teh complex consumer information and slick to query and storing in database in a Scala fashion using teh powerful Scala collection framework.
Environment: CDH Eco system 5.8, ApacheSpark, MapReduce, Yarn, Hive, Pig, HBase, Sqoop, Flume, Kafka, Zookeeper, Java, Scala, Oozie, HDFS, Cassandra, MongoDB, Tableau.
Confidential, NJ
Architect (EMS Project/ Support)
Responsibilities:
- Actively Involved in SCRUM planning and discuss teh development/progress of teh sprints.
- Design teh landscape for accessing applications from Pharma and Consumer MDM repositories in Portals.
- Design and develop Java web services, UI5 applications for portal on device by using MDM JAVA API’s for Material Reservations, GTIN Enumeration etc. for accessing data from MDM.
- Design and develop java programs using MDM Java API to import data from source systems and assign enterprise fields in MDM.
- Enhanced pricing Java application to meet teh new business requirements.
- Actively involved in creating Informatica mappings by adhering to teh coding standards and making use of teh in-house developed validation process to figure teh correctness of teh data.
- Developed mappings, sessions and workflows in Informatica Power Center.
- Extracted data from SAP system to Staging Area and loaded teh data to teh target database by ETL process using Informatica Power Center.
- Developed web application using JSP custom tag libraries, Java Servlets and Objects using J2EE standards.
Environment: Java, SAP NetWeaver 7.4, Informatica MDM, Java Webdynpro, MDM7.1SP10, J2EE
Confidential, KY
Technical Lead
Responsibilities:
- Interact with Business Users, gather requirements and prepare design documents.
- Prepare teh Technical specs and Technical design, perform SIT and fix defects in UAT.
- Design and Develop Webdynpro Java Applications for Creating MDM applications etc.
- Create teh Webdynpro Java application by calling ECC Function Modules for teh data exchange.
- Develop Web services to create sales order and maintenance requests from third party applications to SAP backend systems.
- Prepare teh Technical specs and technical design.
- Enhance standard ESS applications like Time, Travel using Java.
- Developed BPM workflows using Java and J2EE.
- Developed Java applications for Project Systems (Planning).
Environment: NetWeaver 7.3SP3, Web AS 7.3, SAP ECC 6.0 Ehp7, NWDS, WebDynpro Java, MDM7.1SP8, Visual Composer 7.3, Java, EJB, J2EE
Confidential, Texas
Technical Lead
Responsibilities:
- Demonstrated ability in delivering portal solutions that include architecture, analysis, programming, design, testing and support.
- Interact with Business Users and gather requirements, Review functional Specs.
- Developed prototype to check teh functionality using current architecture.
- Design teh Architecture for teh process flow.
- Prepare teh Technical specs and technical design.
- Design and develop services using Java to interact back end system.
- Design and develop Adobe forms for Order Management.
Environment: Java, J2EE, Web AS 7. 0, SAP ECC 6.0, Java WebDynpro, Interactive Forms (Online and Offline), LCDS and LCES.
Confidential, Ohio
NetWeaver Consultant
Responsibilities:
- Demonstrated ability in delivering NetWeaver Portal solutions that include architecture, analysis, programming, design, product installation, testing and support.
- Responsible for upgrading teh Portal from EP 6.0 to EP 7.0.
- Work with application teams to implement coding standards for maximum architecture efficiency.
- Managed and monitored data loads with full upload and delta mechanism.
- Automated data load processes, maintaining BI/BW statistics, rollup of BI/BW aggregates using Process Chains.
- Extensively used BEX Analyzer to build queries, queries with charts and worksheets to access data from BI Info providers.
- Enhanced query performance using BI/BW Statistics, query read modes and aggregates.
- Responsible for fixing teh upgrade issues related to HCM applications (HR Personnel Administration, Payroll, Time, Benefits, Org Mgt. and e-Recruiting).
- Developed applications based on service-oriented architecture (SOA) principles.
Environment: SAP NetWeaver 7.0 SP15, NetWeaver CE 7.1, Web AS 7. 0, SAP ECC 6.0, NetWeaver BI 7.0, Java WebDynpro, Java, EJB, CRM. 2007.
Confidential
Portal Architect
Responsibilities:
- Analyze and recommend technologies to enhance current systems and support overall business goals and global client relationships.
- Designed a Webdynpro App for ESS and MSS pertaining to attendance, Leave and Claims.
- Created Functional and Technical Specs for architectural review. Participated in unit Testing, QA/UAT bug fixes, Implementation and support documentation.
- Developed Webdynpro Java application for ESS (Employee Self-Service) and MSS (Manager Self-Service) pertaining to iTime Payroll and attendance purposes.
Environment: SAP EP 7.0 SP 14, WAS 7.0, Java, NWDI, SAP BI 7.0, CRM 6.0 and ECC 6.0
Confidential, TX
Java/Portal Developer
Responsibilities:
- Interact with IT team and super users in business to finalize overall look and feel of Enterprise Portal system.
- Implemented single SIGN-ON with Portal, ECC, BW, and CRM Systems.
- Responsible for Testing of Central Finance Portal for its Functionality as expected.
- Creation of various users, roles, groups and role assignment. Development of JSP front end for SAP Transactional reports.
Environment: SAP EP 6.0 SP 11, WAS 6.20, Java, ECC 5.0, CRM 4.2 and BW
