We provide IT Staff Augmentation Services!

Senior Data Engineer Resume

Menlo Park, CA


Developer/QA with over 7 years of experience in ETL/BI. Expert in Oracle, Hadoop, Hive/Sqoop, SQL, Python, Perl, Shell Scripting, MicroStrategy, Toad, Autosys. Experience with Full Life Cycle Software Management, Agile, Jira, Git, and Jenkins. Experience in Tech, Health, Medical, Financial, Retail, Sales, and CRM fields.


Datawarehouse Experience: ETL, BI, Data Integration, Analytics, Dev/QA

Programming Languages: Hive/SQL, C/C++/Java, Python/Perl/Ruby/PHP/Bash

Databases: Oracle, Teradata, MySQL, Postgres, Vertica, Hive, Hadoop

Operating Systems: UNIX: Solaris, AIX, HP - UX, Linux Redhat, Centos, Ubuntu and variants; Windows: NT, Mac OSX


Confidential, Menlo Park, CA

Senior Data Engineer


  • Collect business requirements and map requirements to data constructs
  • Implement Data Pipelines/ETL using Dataswarm and Presto for Tealium raw Json files to Hive cluster. Dataswarm is Confidential 's home-built ETL framework.
  • Implement BI and Analytics reports using MetricFactory home-built reporting tool
  • Develop Analytical SQL queries to validate reports

Environment: DataSwarm, MetricsFactory, Tealium, Presto, Hive, Python, Bash, Linux

Confidential, San Diego, CA

Senior QE Data Engineer


  • Modify Parity Checker tool to verify sync of two production DBs in Vertica.
  • Extract and Verify Tax analytics data from Hadoop source and Vertica database using hive and vsql/psql
  • Develop Analytical SQL queries to validate ETL/Tableau reports
  • Automated QA jobs with Python and Bash scripts on Linux (Redhat and Centos) and automate scheduling with Tidal.

Environment: Vertica, Tableau, Tidal, Hadoop, Python/Panda/Numpy, Bash, Linux, Bash, Awk, Sed

Confidential, Foster City, CA

Senior QA Analyst


  • Design and write SQL and Bash scripts to load QA database from Production.
  • Integrate data from Unica marketing database into VDM project.
  • Develop Analytical SQL queries to validate MicroStrategy BI Reports.
  • Automate QA jobs with Bash scripts on Linux.
  • Develop scripts in Bash and Python for Data loading and Data Profiling

Environment: DB2, Netezza, Unica, MicroStrategy, AutoSys, Splunk, SQL, Python, Shell Scripts

Confidential, San Francisco, CA

Senior BI/ETL Engineer


  • Key Team Memnber for migrating Confidential Analytics Platform from Oracle/Postgres Datawarehouse to Hadoop/Hive using Hive HQL/Sqoop, Oracle PL/SQL, and Python scripts for Confidential Data Services Department in an Agile/Scrum environment.
  • Design and code SQL to validate Source to Target Mappings.
  • Code Regression tests to verify ETL migration from Oracle to Hadoop.
  • Code API Tests for RESTFUL Services for LC Monthly Statements.
  • Develop Analytical SQL queries to validate Tableau BI Reports.

Environment: Hadoop/ Sqoop , Hive, Oracle, Postgres, Tableau, Centos, Java, Groovy, Python.

Confidential, San Francisco, CA

Senior ETL/QA Tester


  • Design and code SQL to validate 120 simple and derived BI Sales/Revenue metrics.
  • Develop Analytical SQL queries to validate MicroStrategy BI Reports.
  • Automate QA tasks using Teradata BTEQ, SQL, Perl and Shell Scripts.
  • Code custom ETL used for MicroStrategy BI cubes using BTEQ and SQL.

Environment: Teradata 13, SQL Assistant, MicroStrategy 9.2, Splunk, Excel, AIX, Windows 7.

Confidential, CA

Senior ETL Developer


  • Design and code ETL for Data.com Datawarehouse using Kettle and MySQL procedures.
  • Develop Java-based RESTful API in Pentaho/Kettle.
  • Develop and integrate web Omniture API in Pentaho/Kettle.

Environment: MySQL 5.5, Apache HBase, SQL Workbench, Pentaho PDI, MicroStrategy 9.3, Ruby 1.9, Rails 3.2, Java 1.7, Perl, Shell Scripts, XML, SOAP, REST, JSON.

Confidential, San Francisco, CA

Database QA Engineer


  • Design and code SQL validation for Confidential Order Management Datawarehouse using PL/SQL.
  • Develop Analytical SQL queries to validate MicroStrategy Reports.
  • Automate QA tasks with Ruby and Shell scripts.

Environment: Oracle 11g, Toad 11.0, MicroStrategy 8, SQL, Ruby 1.9, Shell Scripts, TCL/Expect.

Confidential, San Diego, CA

Database QA Engineer


  • Design and code SQL validation for Confidential Consumer Money Card daily loads.
  • Validate Email Campaign data with Responsys email vendor data.
  • Validate site traffic and email campaign data in Omniture SiteCatalyst with backend Netezza datawarehouse. Develop queries for Omniture datawarehouse and validate results with Omniture SiteCatalyst including visits, visitors, pageviews, conversions, and custom events.
  • Create pivot queries in Netezza DWH to for campaign marketing effectiveness and conversion reports.

Environment: Oracle 11g, Netezza Release 6, SQL Developer 3.0, Aginity Workbench 2.1, SiteCatalyst v14, SQL, Perl, Java, Shell Scripts.

Confidential, Burbank, CA

Data Warehouse Engineer - QE


  • Design and code PL/SQL and Perl scripts for testing ETL processes of multi-dimensional datawarehouse comprised of several hundred facts/dimensions used for advertiser reporting.
  • Perform statistical database profiling of source data, range, max, count, outliers.
  • Analyse data requirements and design rule-based software to validate and flag source data.
  • Design and code TestNG functional validation tests for Mondrian BI reporting application using Java web API.
  • Design and code web services automation using Perl SOAP and web API.
  • Participate in design reviews of datawarehouse ETL and modeling.

Environment: Oracle 11g, SQL Developer, Eclipse, Mondrian 3.0, Perl, Java, PL/SQL, Hadoop, PIG, MDX, SOAP/XML, XQuery, JBoss 7, TestNG, Bugzilla, SVN.

Confidential, Los Angeles, CA

Unica Consultant


  • Setup and configure Unica NetInsight for over 200 websites.
  • Setup and configure Unica Campaign Management, A/B, and multivariate testing for marketing department.
  • Code preprocessing scripts for web logs to filter out robots, cookie parsing, and base64 transform.
  • Integrate customer profiles into Unica NetInsight datamart for reporting.

Environment: Unica 8, SQL server 2008, TSQL, Windows PowerShell.

Confidential, Los Angeles, CA

Data Warehouse ETL/Developer


  • Design logical and physical model for CRM datamart for Confidential loyalty program using Erwin r7.
  • Map data and design transformations from SQL server source to Oracle.
  • Code ETL extraction scripts using SQL Server 2008 stored procedures and Oracle PL/SQL.
  • Assist DBA in fine tuning SQL Server and Oracle extract queries and recommend indexes.
  • Write SQL diagnostics on SGA and shared pool in Oracle 10g to resolve out of memory MicroStrategy error.
  • Provide technical support to Confidential Asia DW ETL production system.

Environment: SQL Server® 2008 Management Studio, Oracle SQL Developer, DTS, PL/SQL, Bash

Confidential, Omaha, NE

ETL Developer/Data Analyst


  • Interfaced with IT and Car Management teams to collect Empty Return Ratio business causes and requirements.
  • Analyzed UP transportation database and translate business requirements into data requirements.
  • Correlated car empty miles with transportation factors such as reservations, assignments, and directional loading.
  • Coded custom ETL custom scripts to extract data from Teradata V2R6 and Oracle 10g.
  • Predicted High ERR cars using heuristics and data mining techniques. Deployed daily actionable Excel reports to Car Management resulting in potential savings of 50K empty miles per day.

Environment: SQL 99, Clementine 12.0, Weka 3.5, Oracle 10g, Teradata V2R6, Java, Excel 2007, Oracle PLSQL Developer 7.1, Teradata SQL Assistant 7.1

Confidential, San Diego, CA

Data Warehouse Developer


  • Interfaced with Engineering, Accounting, Marketing, QA, and IT teams for data reconciliation and validation.
  • Configured and Administered Unica’s Affinium NetInsight product and data warehouse for Confidential website. The system processed over 5 million page views per day and stored over 2 TB of weblog data using an Oracle 10g backend.
  • Developed custom visit sessionizer using object-oriented Perl. The algorithm buckets weblog requests by visitor based on session cookie stored in flash object. The sessionizer parses large data sets and pre-fills the cookie parameter field in weblog requests before handing off to NetInsight Update module.
  • Created custom SQL and pivot reports to measure stickiness, visitor to listener conversion, tracks plays, session length, visitor Dollar value, and various ROI metrics for CEO and Marketing Department.
  • Responsible for merging, analyzing, and correlating page and Ad click-through data and return rates with demographics, sales, and shopping cart data to calculate buyer propensity and recommend Ad design and placement.
  • Published Organic and Paid referrals and sales by referrer reports for Ad campaign effectiveness.
  • Constructed trending and forecasting summary reports using Excel with ODBC-connected Oracle back-end.
  • Specified Unica’s based JavaScript tags for tracking flash and WMP media file usage/download.
  • Aggregated daily metrics data for high-performance reporting using Oracle Stored Procedures.
  • Fine Tuned Oracle SQL queries using Oracle’s TKPROF and Explain Plan.

Environment: Unica NetInsight, Oracle 10g, SQL, DW, ETL, Perl, Web Analytics, Excel, ODBC, Shell

Hire Now