SUMMARY:

• Overall, 18 years of software development experience
• 11 years of experience in software development with deep business acumen and technical expertise in Big Data technologies
• Hands - on experience in working with Big Data technologies i.e. HDFS, Yarn, HBase, Hive, Oozie, Sqoop, Spark and AWS (S3 and EMR)
• Good knowledge in Spark streaming modules Kafka and kinesis
• Deep knowledge in Hadoop architecture and various components such as HDFS and other Ecosystem components
• Worked on all phases of data warehouse development lifecycle, ETL design & implementation & support of new and existing applications
• Experience in Git (distributed version control system)
• Experience in project management tools such as Jira, Rally and TFS
• Experience integrating Jenkins with spring boot for continuous delivery
• Experience in building Spark scripts using Python
• Good knowledge in Elastic Search
• Worked on AWS EMR cluster with eight notes
• Experience in creating and validating blocks in block chain using JavaScript
• Experience in RESTful webservices
• Experience working with build tools such as Maven, SBT and gradle
• Experience working in Agile environment and SAFe (Scaled Agile Framework) Release Train
• Strong hold on OOPs concepts and Web-based technologies with good understanding of various phases such as Requirements, Analysis, Design, Development and Testing
• Extensive experience in deploying and working with Tomcat, JBOSS and Node.js
• Hands on experience in implementing teh application logic using Spring Boot, MVC
• Significant experience wit design pattern implementation (MVC, Session, Façade, Singleton)
• Experience using SQL and MySQL
• Worked extensively with Hadoop Distributions - Hortonworks, MapR
• Designed Voice Activated Hands-free reporting solution using NLP assistants like Alexa Assistant
• Excellent communication, interpersonal, analytical skills and strong ability to perform in a team
• Ability to quickly adapt to different business/organizational needs
• Recognized for consistent delivery of commitments in a timely fashion

TECHNICAL SKILLS:

Operating Platforms: Windows Family, Linux, Horton Works, Cloudera

Big Data: HDFS, Map Reduce, Hive, Oozie, Kafka, Spark, IBM INFOSPHERE Streams, Elastic Search and Splunk

Cloud Technologies: AWS

NOSQL: HBase

Languages: Java, Python, Scala, SPL (Streams Programming Language)

Frameworks: Spring, Spring Boot, Angular and Hibernate

RDBMS: Oracle, MySQL, MS-SQL Server

Servers: Jakarta-Tomcat, Jetty, Node.JS and JBOSS

IDEs/Tools: IntelliJ, SpringToolSuite, Eclipse, Visual Studio code, Streams Studio

PROFESSIONAL EXPERIENCE

Confidential, Charlotte, NC

Sr. Big Data Engineer

Responsibilities:

• Design data processing pipelines with spark scala
• Performed Spark memory and query optimizations to improve performance.
• Implemented Custom Scala UDF s.
• Written queries to validate data stored in RDS.
• Produce Unit tests for spark data frames using Cucumber.
• Produce acceptance tests for lambda, spark data frames using Cucumber.
• Monitoring teh production pipelines.
• Monitoring Spark Jobs Using AWS Cloudwatch.
• Written scripts to validate teh data.
• Worked with admin team to collaboratively resolve teh issues, wherever required.
• Ensuring teh adherence to teh Agile Project Management methodologies and practices
• Automated teh pipelines by using Jenkins.

Environment: ApacheSpark, Scala, AWS EMR, Postgres RDS, Servicenow and Jenkins, Hive, HBase and Airflow

Confidential, Nashville, TN

Sr. Big Data Engineer

Responsibilities:

• Design data processing pipelines with spark scala
• Produce Unit tests for spark data frames.
• Creating cluster to spin EMR on AWS.
• Monitoring Spark Jobs Using AWS Cloudwatch.
• Using Ganglia to monitor Cluster while executing teh jobs.
• Optimized teh performance by monitoring teh application in ganglia cluster monitor tool.
• Worked with requirements team to calculate teh complex KPIs
• Written scripts to validate teh data.
• Worked with admin team to collaboratively resolve teh issues, wherever required.
• Ensuring teh adherence to teh Agile Project Management methodologies and practices
• Automated teh jobs by using Airflow.

Environment: ApacheSpark, Scala, MapR Distribution, AWS EMR, Hive, HBase and Airflow

Confidential, Bridgewater, NJ

Sr. Big Data Engineer

Responsibilities:

• Analyzing teh existing system process.
• Identifying teh business-critical Measures by closely working with teh SME.
• Streamlined teh migration process from teh existing system to Big Data architecture.
• Worked on ETL using Spark, Hive, HBase, and Oozie on Hadoop.
• Processing of incoming files using Spark native API, using Spark scripts
• Developed scripts using both Data frames/SQL and RDD in PySpark (Spark with Python) 1.x/2.x for Data Aggregation.
• Performed significant role in upgrading teh system to Spark 2.0 with Data frames and optimizing teh jobs to best utilize of Tungsten Engine.
• Handled large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformations and other during ingestion process itself
• Designed & implemented HBase tables, Hive scripts.
• Experienced in managing and reviewing Spark log files for troubleshoot & debug
• Written Hive jobs to parse teh logs and structure them in tabular format to facilitate effective querying on teh log data
• Involved in converting Hive/SQL queries into Spark transformations using PySpark RDDs and then upgrading to Data frames
• Written complex UDFs to handle teh various multiple missing functionalities in Hive for analytics
• Performed various optimization techniques on Hive QL scripts to use HDFS & processing resources efficiently
• Designed various dimension tables using HBase and written scripts to automate teh data loading to dimension tables
• Worked closely on increasing system performance by reducing teh I/O by identifying teh process gaps and tuning teh queries
• Designed workflows & coordinators for teh task management and scheduling using Oozie to orchestrate teh jobs
• Developed Automation scripts using UNIX/Python which incorporates teh business process for data processing
• Written cron jobs to handle performance checks in teh files system and data
• Involved in writing parsers using Python
• Worked with requirements team to calculate teh complex KPIs
• Worked with admin team to collaboratively resolve teh issues, wherever required.
• Ensuring teh adherence to teh Agile Project Management methodologies and practices
• Involved in scheduling Oozie jobs
• Involved in Unit Testing

Environment: Horton works, HDP, Java, Spring Boot, Angular, JavaScript, Node.Js, Charrt.js, HTML and CSS

Confidential, Norwalk, CT

Senior Software Engineer

Responsibilities:

• Worked with Product, Design, and Engineering teams on requirements gathering and evaluation.
• Participate in product design reviews to provide input on functional requirements, product designs, test estimates, schedules and potential risks.
• Performed Regression testing to make sure that teh operations staff can see teh aircraft-to-ATC communication in clear text.
• Experience in automating regression test cases using Selenium WebDriver.
• Tested Angular 2 Application with Jasmine testing framework and Protractor. Used TypeScript for automation.
• Implemented automation using Selenium Grid to perform testing on multiple devices.
• Application Screenshots are captured using TakeScreenshotAs and copied teh images whenever an exception occurs and at any checkpoint while execution.
• Handled various Alerts using different methods such as dismiss, accept, getText in effective way depending on teh requirement.
• Responsible for developing teh scripts to support Jenkins (Continuous Integration) of teh scripts with teh build server.
• Developed Test Scripts to implement Test Cases, Test Scenarios, and features for BDD (Behaviour Driven Development), TDD (Test Driven Development) using Cucumber in Gherkins format.
• Used GitHub to maintain and manage teh build scripts, test data used and other documents related to teh project.
• Implemented TestNG unit testing framework for teh smoke test and used all Annotations in TestNG effectively.
• Good hands-on experience on bug tracking tool like Jira.
• Worked on Data Driven/Page Object Model Framework and extracted data from external Excel files using Apache POI and loaded into teh variables in teh scripted code.
• Worked on Maven for build and dependency management where I has added different dependencies such as apache poi/TestNG/Maven-plugin-api etc. according to teh requirement.
• Performed functional testing of web services using SOAPUI and RESTful web services.
• Performed Mobile Testing of Android based Apps using Appium. Connected Real Devices.
• Performed Cross Browser and parallel testing to test teh applications are working as desired in different browsers and environments.
• Expertise in Smoke testing, Regression testing, System Testing and UAT.
• Delivered Quality Engineering services for 3 enterprise applications
• Worked with business analysts to interpret business requirements
• Developed automated test scripts in HP Winrunner and LoadRunner
• Performed functional, load and regression testing
• Triaged defects with developers and business analysts