We provide IT Staff Augmentation Services!

Spark & Hadoop Developer Resume

0/5 (Submit Your Rating)

Cincinnati, OH

SUMMARY:

  • 14 years of IT experience which includes 1.5 years of experience on Hadoop environment.
  • Good experience with Big Data Ecosystems, ETL.
  • Experience in data architecture including Data ingestion, Data analysis and Data Analytics, advanced Data processing. Experience optimizing ETL workflows.
  • Hands on experience with Hadoop Core Components and Hadoop Ecosystem (Sqoop, Flume, HDFS, Hive, Pig, Impala, Oozie).
  • Hands on experience in ingesting real time/near realtime data using Flume, Kafka, Spark Streaming.
  • Hands on experience in importing and exporting the data using Sqoop between Relational Database and HDFS.
  • Hands on experience on Linux systems.
  • Experience in developing jobs using Spark framework modules like Spark - Core, Spark-SQL and Spark Streaming using Scala
  • Expert in working with Hive data warehouse tool-creating tables, data distribution by implementing partitioning and bucketing, writing and optimizing the HiveQL queries.
  • Experience in Unix shell scripting.
  • Experience in using Sequence files, AVRO file, Parquet file formats.
  • Good knowledge on Statistics and Machine Learning.
  • Very good experience in Data warehouse architecture (Conceptual, logical, and physical), solutioning (Dimensional Modeling) and implementations.
  • Hands Experience in building effective Data warehouse and Data Marts utilizing Ralph Kimball Star Schema Dimensional Modeling methodology and Bill Inmon Relational Modeling methodology.
  • Good knowledge on BI tools like Tableau, OBIEE.
  • Demonstrated ability in defining project goals and objectives, prioritizing tasks, developing project plans and providing framework for effective communication while maximizing responsiveness to change.

TECHNICAL SKILLS:

Database: DB2, Oracle 9i/10g/11g using SQL Developer and TOAD, Oracle Exadata, SQL Server Teradata, PostgreSQL.

Data warehouse/ BI tools: Informatica, OBIEE, Tibco Spotfire, Reveleus (OFSAA).

Others: OFSAA Reveleus, CA Erwin 7.3, MS Visio, Adobe Acrobat, HP Mercury Quality Center, Lotus notes, VB.

WORK EXPERIENCE:

Spark & Hadoop Developer

Confidential .Cincinnati, OH

Responsibilities:

  • Involved with ingesting data received from various relational database providers, on HDFS for analysis and other big data operations.
  • Written Spark jobs in Scala to analyze the engineering data.
  • Used Spark to perform necessary transformations and actions on the fly for building the common learner data model which gets the data from Kafka in near real time.
  • Used Spark API over Cloudera to perform analytics on data in Hive.
  • Optimizing of existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames and Pair RDD's.
  • Created Hive external tables to perform ETL on data that is generated on daily basics.
  • Created SQOOP jobs to handle incremental loads from RDBMS into HDFS to apply Spark Transformations and Actions.
  • Developed job flows in Oozie to automate the workflow for pig and hive jobs.

Tools: Used: Apache Hadoop, Hive, HDFS, Scala, Spark, Linux, MySQL, Eclipse, Oozie, Sqoop, Kafka, Cloudera Distribution, Oracle.

Technical Lead

Confidentia, Cincinnati, OH

Responsibilities:

  • Design, Build Informatica process for any data requirements which is not readily available in GE systems.
  • Building spotfire reports for Engineering department.
  • Lead team of BI resources.
  • Design, Build BI solution using Spotfire and ADS tools to develop reports as per client expectations.
  • Responsible for Planning, Scheduling, Task Distribution, Tracking and Delivery.
  • Lead team to create best practices and automation across multiple process improvements.
Tools Used: Tibco Spotfire 6.5.2, Cisco ADS Studio 7.0, Informatica 9.6.1, Oracle SQL Developer, Putty, WinScp, ServiceNow, pgadmin iii - postgreSQL tool.

Technical Lead

Confidential, Pittsburgh, PA

Responsibilities:

  • Manage and provide Technical Solutions to projects.
  • Lead team of data analyst, Informatica, Java, Manual Testing, PL/SQL developers.
  • Responsible for Planning, Scheduling, Task Distribution, Tracking and Delivery.
  • Coordination/leading on different project activities across systems.
  • Lead team to create best practices and automation across multiple process improvements.
  • Team coordination of more than 40 people including mix of Business, Senior Management, IT Team (BA, Development, and Testing Teams) resides across different locations/countries.

Tools: Used:Informatica 9.5.1, Informatica Analyst, Data Profiling, Toad for Oracle 11, Oracle RDBMS 11g, Oracle Exadata Database, SQL Server 2008, Teradata Studio 14.10.01, Tableau, Java, Microsoft Visio, Putty, Harvest, HP ALM, Clarity, Information Technology Service Management (ITSM), CA7 Scheduler, Shared Services Manager (SSM) Tool, ACORN Tool.

Technical Lead/ Data Modeler

Confidential, Pittsburgh, PA

Responsibilities:

  • Lead team of data analyst, SAS analyst, Informatica, OBIEE, OFSAA (Reveleus), PL/SQL developers to complete implementation of ALLL re-engineering system.
  • Dimensional Data Modeling and maintenance for large data warehouse using Ralph Kimball Star Schema Dimensional Modeling methodology.
  • Responsible for Planning, Scheduling, Task Distribution and Tracking.
  • Coordination/leading on different project activities across systems.
  • Integrated business area's in single platform which was previously distributed across SAS, SQL Server, Desktop application.
  • Team coordination of more than 20 people including mix of Business, Senior Management, IT Team (BA, Development, and Testing Teams) resides across different locations/countries.
Tools Used:Informatica 9.5.1, Erwin Data Modeler 7.3, Toad for Oracle 11, Oracle RDBMS 11g, Oracle Exadata Database, OBIEE, Teradata Studio, OFSAA Reveleus, Microsoft Visio, Putty, Harvest, HP ALM, Clarity, Information Technology Service Management (ITSM), CA7 Scheduler.

Technical Lead

Confidential, Pittsburgh, PA

Responsibilities:

  • Lead team of Informatica, PL/SQL to build a Reveleus Datamart.
  • Worked on entire environment set-up and successful implementation of solution.
  • Involved in all the discussions with the client during requirements phase.
  • Designing ETL’s as per the requirements and Creating/Updating High Level and low level design for the changes made to the components to improve the performance.
  • Co-ordination with Onshore/ Offshore teams.
  • Reviewing the Low level design as per the Design checklists for the changes made to the components to improve the performance.
Tools Used: Informatica 8.6, Informatica MDM,Toad for Oracle 11, Oracle RDBMS 11g, Oracle Exadata Database, OBIEE, Teradata Studio, Microsoft Visio, Putty, Harvest, HP ALM, Clarity, Information Technology Service Management (ITSM), Autosys Scheduler.

ETL Designer

Confidential, Pittsburgh, PA

Responsibilities:

  • Creating Solution Design to populate the warehouse tables.
  • Part of Enterprise Data warehouse using IBM BDW Model, Reveleus Datamart.
  • Responsible for ETL Architecture to load data into Ralph Kimball Star Schema Dimensional Modeling tables.
  • Involved in all the discussions with the client during requirements phase.
  • Designing ETL’s as per the requirements and Creating/Updating High Level and low level design for the changes made to the components to improve the performance.
  • Co-ordination with Onshore/ Offshore teams.
Tools Used:Informatica 8.6, Informatica MDM, Toad for Oracle 11, Oracle RDBMS 11g, Oracle Exadata Database, OBIEE, Teradata Studio, Microsoft Visio, Putty, Harvest, HP ALM, Clarity, Information Technology Service Management (ITSM), Autosys Scheduler.

ETL Developer

Confidential, Cleveland, OH

Responsibilities:

  • Part of Informatica Upgrade Project of Version 7.1.1 to Version 8.1.6.
  • ETL Developer in building Customer Billing System(CBS).
  • Involved in the daily status call discussion on requirements and change request. Gather all new requirements and understand the business logic for new change request from Subject Matter Experts(SME).
  • Analysis of defects reported for identifying the root cause and validate the need for code or design changes.
  • Reviewing the Low level design as per the Design checklists for the changes as per the requirements.
  • Developing mappings using Informatica Designer Tool for the required changes as per the defect/CR’s. Creating the workflows/sessions and executing the same using the Informatica Workflow Manager Tool. Monitoring the execution of workflow/sessions using Workflow Monitor Tool.
  • Preparing/Executing the test cases as per the change requests received by creating the test data.
  • Check - out and Check - in of the modified objects through Change Management tool.
Tools Used:Informatica 8.6/ 7.1.1, Toad for Oracle 11, Oracle 1\g, Microsoft Visio, Putty, Serena Change Management, HP ALM, Autosys.

ETL Developer

Confidential

Responsibilities:

  • Creating Low Level design documents as per the mapping specifications provided and referring high level design documents.
  • Developing ETL mapping using Informatica Designer Tool. Creating the workflows/sessions and executing the same using the Informatica Workflow Manager Tool. Monitoring the execution of workflow/sessions using Workflow Monitor Tool.
  • Preparing/Executing the test cases for the code by creating the test data.
  • Check - out and Check - in of the modified objects through Serena ChangeMan DS.
  • Defects handling: Defect tracking and data fix using IBM-Rational. Triaging defects and fixing the same in code if required.
Tools Used: Informatica 7.1, Oracle 9i, Oracle SQL Plus, Serena Change Management, Autosys.

VB Developer

Confidential

Responsibilities:

  • Involved in Application Development
  • Preparing low level design
  • Creating all the screens formats
  • Design the reports using Data Reports
  • Unit testing & Debugging.
  • Triaging the defects.
Tools Used: VB, Oracle 9i, Oracle SQL Plus. Bachelor in Computer Applications (2002)Osmania University, Telangana, India Additional Information EXPERTISE:

We'd love your feedback!