Big Data Engineer Resume

SUMMARY

Overall 4.5 + years of experience as a Sr. Data Engineer at Confidential with expertise in Ecommerce/Consumer Goods domain.
Experienced in developing and managing databases, Business Intelligence (BI) applications, ETL applications using SSIS, Informatica, MS SQL, Python, SQL Server Agent Jobs.
Experienced in analyzing large data sets on Hadoop using HDFS commands, Hive, Sqoop, Pig, HBase and MapReduce
Skilled in Jenkins, Git, Python & BASH Shell.

TECHNICAL SKILLS

Mongo DB, Couch DB, Hive, MS SQL, HBase
MS SQL, MySQL, PL - SQL, C#, ASP.Net, Shell, python, Spark, Pig, Sqoop: SSIS, SSRS, Informatica
Dream viewer, Jenkins, Selenium, JIRA Visual studio, SSMS, Kafka, Studio TD, BitBucket, Git, Source Tree: Data Base and Basics of Automation testing in Selenium

PROFESSIONAL EXPERIENCE

Confidential

Big Data Engineer

Responsibilities:

Responsible for designing, building and architecting the Cloud Based (Software as a Service) Platform for Confidential storage products using Hadoop techniques.
Implemented promotion schemes and real-time intraday coupon catalogs for stapels.com/ca websites using HBase, Kafka and spark, this enabled regular updates on pricing and increased the revenue by 10%.
Created Hive queries that helped market analysts for dynamic pricing trends by comparing fresh data with EDW reference tables and historical metrics.
Worked on ETL logic and performance tuning on Spark to improve SLA from 15 minutes to 6 minutes.
Designed and delivered high-performance event-driven data ingestion platform for handing massive data collection, storage, and analytics using Python, Spark, RabbitMq, and Hive.
Enabled speedy reviews and fast mover advantages by using Scoop to automate data loading into the Hadoop Distributed File System and HBase to pre-process the data.
Experienced in using Kafka to collect the data from source to multiple destinations.
Facilitated insightful daily analysis of order fulfilment data (60-80 GB) extracted from post transaction team using python scripts and suggested recommendations that increased traffic by 38% and advertising revenue by 16%.

Data Engineer

Confidential

Responsibilities:

Led a team of 5 and enabled data-driven decision making for clients across verticals (Retail) and horizontals (Sales, Marketing). Recognized and awarded for consistently deriving creative solutions and prototypes for business expansion.
Measured the impact of marketing spend on various site and recommended the marketing team to channelize their efforts on cost effective campaigns which saved $2M revenue in 2015 Q3 using data APIs.
Segmented customers based on purchasing behavior and established a Quarterly self-serve reporting tool in SSRS enabling business users to employ differential targeting strategies that reduced attrition rate by 13%.
Designed and developed Unified Publishing Catalog Platform using Service Oriented Architecture (SOA) which includes UI, Confidential B2B & B2C Catalog Data Publishing & Management, high volume Data publication to multi-channel consumer (Content-as-a-service), ETL .
Owned and implemented complex SSIS packages to migrate data from multiple vendors (Wayfair, Ingram, United, SP Richards, etc.) to Confidential Data Warehouse using for-each Loop/Sequence containers, condition splits, and aggregates and derived columns.
Developed stored procedures, SQL Agent jobs, input interfaces and fine-tuned queries using SQL Profiler, SQL Query Analyzer, Index tuning wizards and supported Ad-hoc requests to ensure stability and integrity of Confidential Product DB.

DevOps

Confidential

Responsibilities:

Closely working with agility team to automate the build and deployments of Platform components.
Responsible for Cluster tuning, monitoring and troubleshooting.
Worked on configuration management tools like Puppet and continuous integration processes with Jenkins.
Extensively used Python, Shell Script to automate CI/CD (Jenkins) application.
Involved in Cluster Monitoring, Troubleshooting, Manage and review Hadoop log files.

SOFTWARE GRADUATE TECHNICAL INTERN ,

Confidential, NJ

Responsibilities:

Designed and implemented real-time Hotel Booking application in C#, a WCF web service portal that allows customers to book room's online, login, register, send emails, calculate revenue, check booking status and validate bank card payment.
Developed SOAP and RESTFUL Web Services, DLLs, User Controls with MySQL Database and Session State Management.