Sr. Data Engineer Resume Foster City - Hire IT People

SUMMARY:

Over 14 years of experience in Design, Development and implementation of BigData, Client/Server applications on Windows and UNIX platforms using Hadoop, JAVA, C++, C, SQL, VB, Perl and Shell Scripts.
3+ years of experience on Big Data technology stack (Hadoop, MapReduce, HDFS, YARN, Hive, Pig, Sqoop, Oozie, Hbase, MongoDB et al) - Programming and Analytics.
Experienced in performance tuning JAVA MapReduce code and HIVE queries.
Extended Hive and Pig core functionality by writing custom JAVA User Defined Functions (UDF) and User Defined Aggregating Functions (UDAF); implemented DAG workflows with error handling and mailing through Oozie.
Around 10 years of experience as an Oracle PL/SQL Developer in Analysis, Design and Implementation of Business and ETL Applications along with 4+ years of experience in JAVA application development and maintenance.
In depth understanding of MapReduce concepts and its critical role in data analysis of huge and complex datasets.
Proficiency with mentoring and on-boarding new engineers on Hadoop and taught courses on Hadoop and its eco system, setup multi node cluster on Amazon Cloud AWS, installation and configuration of VMs.
Good knowledge on NoSQL databases like MongoDB and Cassandra.
Good knowledge of Data warehousing concepts and ETL processes
Expertise in Client-Server application development using Oracle 11g/10g/9i, PL/SQL, SQL *PLUS, TOAD and SQL*LOADER.
Experience in converting existing ETL flows into Hadoop using Pig, Hive, Sqoop, Java MapReduce and oozie.
Strong UNIX Shell Scripting skills.
Experienced in large development projects involving object oriented technologies and worked with leading firms in Banking Financial Systems, Medical Systems and Insurance.
Sound knowledge in Fixed Income Securities, Trading, Trade Lifecycle, Market/Reference Data.

TECHNICAL SKILLS:

Hadoop, MapReduce, Hive, Pig, Sqoop, Oozie, MongoDB
C, C++, VB 6.0, JAVA, Python, SQL, C#, VB.Net
Oracle 9i/ 10g/ 11g, Sybase 12.5.1, MS-Access
MFC 6.0, COM, Windows SDK
XML, HTML, Perl, Shell Scripts
Windows XP, Windows 2000, Windows 7, Linux, HP-UX, Solaris

PROFESSIONAL EXPERIENCE:

Confidential, Foster City

Sr. Data Engineer

Environment: Cloudera CDH 5.3.2, Hive 13, Pig 12, Mongo DB, Sqoop 1.4.5, DB2, Kerberos, RedHat 6.5

Managing and working on end-to-end data pipelines involving data extraction, transformation, and loading using Hive for GCAR project to recover counterfeit fraud losses.
Worked closely with data analysts, product managers and software engineering teams to understand data needs from product perspective.
Prepare test data and execute data transformation and data validation testing using pig and hive scripts.
Convert the Abinitio ETL jobs to Pig scripts for Data Lake ingestion.
Aggregate, Analyze and export the Market Insights Data to EDW and reported through Micro Strategy which provides KPIs by Merchant category, predefined market segments.

Confidential, Las Vegas

Sr. Hadoop/Data Engineer

Environment: Hadoop 2.4, Hive 13, Tez, Sqoop 1.4, Oozie 4.x, RedHat 6.5, XAsecure 3.5

Streamlined the data pipelines to separate out the stage, pre-stage and load handling.
Implemented the JAVA MapReduce XMLParser to handle both gas and electric XML data and improved the performance using AVRO.
Performance tuning in Hive.
Extract data from custom format files using Python and ingest them into Hive tables.
Capture business requirements from the users and lead the team by providing technical solutions.
Reconciled data from different sources and ingested them into a stage table.
Created JAVA UDFs for data specific processing and extended HIVE and PIG functionalities.
Streamlined SQOOP import from Oracle to Hive and implemented scripts to export the aggregations back to Microstraregy.
Lead the onsite-offshore team and be responsible for deliverables.

Confidential, Pittsburgh, PA

Hadoop Developer

Environment: Hadoop 1.x, Hive, Pig, Mongo DB,Sqoop, Oozie, CentOS

Responsibilities:

Involved in development of Metadata Management Framework POC to document and accelerate the sourcing and distribution of data utilizing the Big Data technologies such as Hadoop, Hive, Pig Scripts, Sqoop and Oozie.
Coded Pig Latin, Hive and Map Reduce scripts for data feeds to determine Data Quality checks, metrics and perform Data Profiling, scrubbing and analysis tasks and run them through the Analytic warehouse system.

Confidential

Oracle/ C++ Developer

Environment: Oracle 9i/ 11g, Linux, Shell Scripting, C++

Responsibilities:

Worked as C++/oracle application developer on proprietary application used for trading, portfolio management, compliance checking, conducting Investment Research analysis and fund accounting operations relating to investment securities and mutual funds.

Confidential

Software Engineer

Responsibilities:

Worked with different companies like Confidential, Confidential, Confidential, Confidential as client-server application developer using technologies like C++, Java, Oracle, PL/SQL, VB, MFC, COM, ATL, XML, Autosys, Unix scripting, Perl and Solaris.

We provide IT Staff Augmentation Services!

Sr. Data Engineer Resume

Foster, CitY

We'd love your feedback!

Resume Categories

Client Services

Job Seekers

Visa Sponsorship