We provide IT Staff Augmentation Services!

Enterprise Data Architecture - Hadoop & Spark Developer Resume

0/5 (Submit Your Rating)

Torrance, CA

SUMMARY:

  • Over 12 years of IT Consulting experience that includes Data Engineering, Data Modelling, Data Analysis, Data Analytics & BI, Software Development, Project Delivery and Operations for key enterprise programs in the Retail, Insurance & Automotive industries
  • Expert in designing and implementing enterprise data solutions including batch / real - time processing
  • Experience in working with large operational databases / large tables / Transactional Data
  • Proficient in writing PL/SQL and T-SQL scripts for the database objects like tables, views, cursors, procedures, functions, packages, database triggers, indexes, sequences and flashbacks
  • Experience in working with ETL tools like DTS, SSIS, Informatica
  • Proficient in performing Data analysis with the RDBMS systems and exposure to Data Analysis in Big Data (Hadoop) environment using Sqoop, Hive, Impala, Spark - pyspark
  • Exposure to real time data streaming & processing using Flume, Kafka and Spark Streaming
  • Exposure to working with Mongo DB - No SQL database
  • Proficient in Performance tuning of ETL processes using Explain Plan, SQL Profiler, AWS Reports
  • Experience with generating reports & dashboards using Power BI, Tableau, Salesforce, Hyperion Brio & BO
  • Proficient in Marketing Analytics
  • Successful management of projects & programs ensuring quality & timely delivery
  • Proficient in client & vendor relationship management

TECHNICAL SKILLS:

Statistical Programming: R, Python

Desktop / Web Technologies: VB, ASP, C#.Net, ASP.Net

Data Warehouse/ ETL Processing: PL-SQL, T-SQL, DTS, SSIS, Informatica, SF Data Loader, Shell Scripts

Data Visualization / BI / Reporting: Power BI, Tableau, Hyperion - BRIO, Business Objects

Big Data Framework / Querying Tools: Hadoop, Spark, Hive, Pig, Impala

Data Pipes: Flume, Kafka

NO SQL Data bases: MongoDB

Development Tools: R-Studio, Sublime Text, HUE, Microsoft SQL Server, Toad, Microsoft Visual Studio

Configuration Management Tools: GitHub, Bit Bucket, VSS

Project Management / Agile Tools: Microsoft Project, JIRA, Confluence, SharePoint

CRM: Salesforce

PROFESSIONAL EXPERIENCE:

Confidential, Torrance, CA

Enterprise Data Architecture - Hadoop & Spark Developer

Responsibilities:

  • Loaded Structured Data from MySQL Database into HDFS and Hive and Impala Databases using Sqoop
  • Loaded data from files (text, CSV, Sequence, Avro, Parquet) in to HDFS
  • Performed Data Analysis and Data Transformation on the data stored in HDFS and in Hive Database by creating RDDs and Data Frames in Spark - Python shell
  • Performed Data Analysis using Hive QL
  • Loaded transformed data into HDFS / Hive Database from RDDs
  • Performed real time data analysis on Twitter feed data using Spark - Streaming
  • Implemented POCs on Flume real time Data streaming

Environment: Cloudera Distributed Hadoop Cluster, HDFS, Sqoop, Hive, Spark, Python, Spark Streaming, Flume

Confidential, Torrance, CA

Data Analytics & Standards Consultant

Responsibilities:

  • Data Analytics
  • Design and capture PMO Operations KPIs in SQL Server 2005
  • Generate Reports and Dashboards (Operational dashboards, Sourcing dashboards, Project Compliance Dashboards, Project Specific Dashboards) in Power BI by connecting to multiple data sources
  • 1. RDBMS (SQL Server / Oracle)
  • 2. OData feeds (SharePoint, PWA)
  • 3. GitHub
  • 4. JIRA
  • Evaluated Tableau and Generated POC reports using Tableau Public
  • Process Adoption and Standardization
  • Meet with Project teams periodically to ensure PMO processes / Standards / Guidelines are adhered
  • Tools: Power BI, Tableau, Sql Server 2005, SharePoint 2013, Bit Bucket, Microsoft Project Online, JIRA

Confidential, Louisville, KY

Data Architect - Project Lead

Responsibilities:

  • Meet with Business / Product teams and understand the project ask
  • Identify the required Data munging and Data transformations
  • Provide Data Integration oversight - review and tweak the design & data models using tools like Toad / Erwin
  • Conduct Estimation workshops / WBS sessions
  • Resource Loading and Project Planning using Microsoft Project
  • Budget tracking - Track and report Project Financial data
  • Tools: Erwin, Toad, SharePoint 2010, Microsoft Project

Confidential, Framingham, MA

Data Engineer / Data Analyst / Data Architect / Project Lead (Consultant)

Responsibilities:

  • Lead the architecture, design and delivery of multiple Data products / solutions in the Customer Loyalty, Customer Hub, Telesales, Campaign Management, Retail Analytics and Point of Sales domain areas
  • Designed database models to store huge volume of Sales / transactional data
  • Developed Oracle stored procedures, Packages, functions, sequences, indexes, triggers
  • Designed and Developed ETL Scripts / packages using PL/SQL scripts, Shell Scripts, Oracle Stored Procedures / Packages, SSIS, DTS and Informatica
  • Designed and Developed reports using Hyperion Brio and Business Objects
  • Monitored and improved performance of ETL scripts using Explain Plan, SQL Profiler, AWS reports
  • Designed and migrated ETL jobs scheduling from Crontab to Tidal
  • Gathered Statistics and Analyzed Tables and Indexes for Performance Tuning
  • Consumed a SOAP based real time Java web services from PL/SQL using UTL HTTP package for real time data updates
  • Proposed process improvements - Automation / Migration / Performance Tuning / Dedupes / Redundant processes
  • Did Feasibility analysis for migrating Data from Oracle DB to Mongo DB
  • Worked with QA team for Load Testing and Performance Testing

Environment: / Tools: Unix / Linux, Oracle 10g, Sql Server 2008, Mongo DB, Shell Scripting, SSIS, DTS, Informatica, SalesForce Data Loader, Salesforce, VB/ASP, ASP.Net, Hyperion Brio, Business Objects, Crystal reports, JIRA

Confidential

Tech Lead / Senior Software Engineer

Responsibilities:

  • Gathered the Business requirements from offshore by interacting with the client
  • Estimated the effort by reaching out to the Technical leads and published the project schedule
  • Lead the design and development of Oracle PL/SQL stored procedures and packages for customer ranking
  • Worked with Software Quality Assurance Group for the code Review and System Testing
  • Worked with the Confidential Business users for the user acceptance testing (UAT) and adhered to change management process for the Produce release
  • Construction management System - This Project was for Toll Brothers a leading home building company in US. This Project aimed at providing a web based application and a PDA based (hand held) app for the Project managers for tracking all the tasks associated with building a lot
  • Assisted requirements gathering from off shore
  • Created Design Documents and System Test Plans
  • Designed the ASP.Net web application and the .Net 2.0 compact Framework smart client application using Rational Rose
  • Developed the proof of concept for all the possible ways of data synchronization from the handheld with the central database using ASP.Net, ADO.Net and C#.Net
  • Developed a proof of concept for the smart client application - Data synchronization module using the .Net compact framework 2.0
  • Involved in analysis of the requirements and created design documents (High level and Detailed)
  • Developed the windows based application using C#.Net and used Web Services to get authenticated to connect to the UK government secure site. Used XML Serialization methods to process the downloaded forms and involved in unit testing & system testing and fixed all the test issues

We'd love your feedback!