SUMMARY

• Result Oriented, Proactive, and Industrious Professional with around 8+ years of hands - on experience in the field of Data insight, Data Analysis and Data Engineering.
• Strong knowledge and experience on ETL, Data Analysis, Data Lineage, Big Data pipelines, Data quality, Data Reconciliation, Data transformation rules, Dataflow diagram including Data replication, Data integration and Data orchestration tools.
• Excellent backend skills in Hive, creating, optimizing & performance tuning objects like Tables, Stored Procedures, Functions on multiple databases SQLServer, MySQL, Oracle, SQLite, Snowflake (SnowSQL).
• Solid Experience and understanding of Implementing large scale Data warehousing Programs and E2E Data Integration Solutions on Snowflake Cloud, AWS Redshift, Informatica Intelligent Cloud Services and databases (MySQL,Teradata, Oracle, Sybase, SQL server, DB2)
• Knowledge and experience on AWS services like Redshift, Redshift spectrum, S3, Glue, Athena, Lambda, CloudWatch and EMRs like HIVE, Presto
• Strong experience in core Java, Scala, SQL, PL/SQL and Restful web services.
• Proficient in writing complex SQL for data analysis in Data Bricks.
• Good Knowledge in Amazon AWS S3 concepts like EMR and EC2 web services which provides fast and efficient processing of Big Data.
• Designing and highly implementing performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks.
• Hands on Experience on Python programming for data processing and to handle Data integration between On-prem and Cloud DB or Datawarehouse.
• Hadoop & Spark Developer experience as software developer in design, development, deploying and large scale supporting large scale distributed systems.
• Expertise in using various Hadoop infrastructures such as Map Reduce, Pig, Hive, Zookeeper, Hbase, Sqoop, Oozie, Flume, Drill and spark for data storage and analysis.
• Work on NoSQL large databases Hadoop (HDFS), MongoDB to store data from heterogeneous sources to provide Customer Experience.
• Extensive experience in creating Business Intelligence Solutions and expertise in designing and implementing Relational & Non-Relational Database model as per business needs.
• Strong in writing Complex SQL queries in RDBMS to Analyze large amount of data, good knowledge in OLAP and Aggregated tables concept.
• Experience across a wide spectrum of industries including Telecommunications, Financial Services, Pharmaceutical, Bio-Tech, Supply Chain, Retail, Operations and Manufacturing.
• Expertise in developing SQL and PL/SQL codes through various Procedures/Functions, Packages, Cursors and Triggers to implement the business logics of database.
• Delivering and presenting proofs of concept to of key technology components to project stakeholders.
• Build reports using Python, HTML, JavaScript & web technologies for Global Systems.
• Extensive experience in Master Data management and Extracting, Transforming & Loading (ETL) using SSIS, bulk insert, DTS packages into Data Warehouse from heterogeneous.
• Worked on the GCP architecture according to the needs and requirements of the company.
• Worked with GCS (Google Cloud SQL) for handling the user data or client data.
• Involved in building Tableau dashboards to provide insight to the data.
• Ability to collaborate with peers in both business and technical areas, to deliver the optimal business process solutions, in line with corporate priorities
• Experience in all stages of Software Development Life Cycle (SDLC) with proficiency in Various Methodologies including Waterfall and Agile (SCRUM).
• Familiarity with Jenkins, Continuous Integration, Continuous Deployment, Pipelines.
• Expertise in working on Agile/Scrum methodology and used tools like JIRA, Rally, SharePoint, Confluence for project collaboration.
• Experience in gathering business requirements from business/user, creating Process Flows and Data Flow Diagram (DFD)
• Adept in creating Use cases, Test Cases and GUI (Graphic User Interface) using MS WORD.
• Strong analytical skills in data analysis and technical skills in using Excel modeling, Excel PowerPoint motivated professional with leadership qualities and extensive skills in interacting with subject matter experts, stakeholders, development team, users, and various levels of management.
• Team leads skills encompassing user interviews, coordination with Technical Leads, developers, QA/QC analyst during the design and testing phase of the product.
• Experience in maintaining support documents, Test Plans, QA Sign off Documents and Maintaining Weekly Status Reports.
• Experienced in handling concurrent projects and providing expected results in the given timeline.
• Excellent Communication, Time Management, and presentation skills.

TECHNICAL SKILLS

Big Data Tools: HDFS, MapReduce, HBase, Pig, Hive, Sqoop, Flume, MongoDB, HBase, Oozie, Zookeeper, spark, Airflow, storm & Kafka, Snowflake, PySpark

Database Systems/Tools: SQL Server, MySQL, Teradata, Oracle, Microsoft Access, Snowflake, Sybase, DB2, SAP HANA, SNOWSQL

Methodologies: Agile (Scrum, Kanban), Waterfall

Project Management: Microsoft Project, PowerPoint, Excel, Jira

Scripting Tools: JavaScript, jQuery, AJAX, JSON, Node.js, Python

Operating Systems: Windows, UNIX, Linux

BI Tools: Tableau, Salesforce 4

PROFESSIONAL EXPERIENCE

Confidential - Plano, TX

Sr. Big Data Engineer

Responsibilities:

• Responsible for SOX controller for TFS ( Confidential Financial Services), MSF (Mazda Financial Services), TFSB ( Confidential Financial Services Bank) and BASS PRO.
• Using complex SQL created daily monthly, and quarterly evidence for audit purpose as per the request.
• Coordinating with business and data factory owners to get evidence.
• We use Autosys to find out the Last Modified code dates and find the workflows and built ETL pipelines in Snowflake.
• Implement One time Data Migration of Multistate level data from SQL server to Snowflake by using Python and SnowSQL.
• Hands on experience in suing Netezza and Snowflake to do Starschema join to build the data pipelines at Confidential
• Day to-day responsibility includes developing ETL Pipelines in and out of data warehouse, develop major regulatory and financial reports using advanced SQL queries in snowflake.
• Stage the API or Kafka Data (in JSON file format) into Snowflake DB by Flattening the same for different functional services.
• Hands on experience in Visio to find data workflows.
• Worked on Architecture Design for Multistate implementation or deployment.
• Build Docker Images to run airflow on local environment to test the Ingestion as well as ETL pipelines. Building/Maintaining Docker container clusters managed by Kubernetes. Utilization of Kubernetes and Docker for the runtime environment of the CI/CD system to build, test and deploy.
• Created Airflow DAGs to schedule the Ingestions, ETL jobs and various business reports.
• Support Production Environment and debug issues using Splunk logs. On call support for production job failures and lead the effort on working with various teams to resolve the issues.
• PerformedData validationon updated raw data as and when required.

Confidential

Sr. Big Data Engineer

Responsibilities:

• Oversee the performance of Design to develop technical solutions from Analysis documents.
• Developed Merge jobs in Python to extract and load data into MySQL database.
• Expertise in Snowflake data modeling and ETL using Snowflake SQL, implementing complex stored procedures and best practices with data warehouse and ETL concepts.
• Worked onSQL Loaderto load data from flat files obtained from various facilities every day.
• Developed multi-threaded standalone app in Python, PHP, C++ to view Circuit parameters and performance.
• Developed Python batch processors to consume and produce various feeds.
• Developed entire frontend and backend modules using Python on Django Web Framework.
• Developed Business Logic using Python on Django Web Framework.
• Used snowflake as database to connect to tableau using the secure connections.
• Automated data loading, extraction, reports generation using python scripting
• Responsible fordata mappingof raw datasets according to data standard.
• Generated the test plan specifying an overview of testing approach, testing strategy, roles, responsibilities, and its scope
• Assisted in preparingBusiness Requirement Documents (BRD)and documented software requirements for the application
• Managed and developed financial reportsthat aligned and simplified monthly business reviews, annual planning, and forecasting processes
• Performed Parallel Testing or Production Testing, which ensures that the new system will perform correctly in a production environment and interface correctly with other production systems.
• Participated in the User Acceptance Warranty Testing. Involved in doing security and negative testing of the application
• Generated ad hoc and monthly recurring financial and operational performance analysis, based on the needs of the stakeholders
• Appliedsubject matter expertiseand advanced level of understanding ofbest practices and standardsof using Tableau.
• Createdtechnical documentation and presented informationto senior management
• Partially involved in all phases (SDLC) of this project from planning, requirement gathering, designing, development, testing, and implementation.
• Created Aggregations, calculated Fields, Table calculations, Totals, percentages using Key Performance Indicators (KPI) and Measure
• Created organized, customized analysis and visualized projects anddashboardsto present to senior level executives
• Created variousAd-hoc reports and analysesas per user requests for managerial meetings and discussions
• PerformedData validationon updated raw data as and when required.

Confidential

Big Data Engineer

Responsibilities:

• Worked on building predictive analytic solutions
• Developed, executed, and reviewed unit test plans and results using SQL Server tools.
• Devising BI architecture, utilizing Tableau Desktop and Server and SQL interface, to provide platform for accurate reporting and analysis of key metrics for the teams based out of Americas, Asia Pacific, Africa, and Europe
• Extensive experience using framework APIs like Struts, Spring, iBatis and hibernate.
• Interfaced with the Oracle back-end using Hibernate Framework and XML config files.
• Developed Servlets and JSPs based on MVC pattern using Struts framework and Spring Framework.
• Expertise in MVC Architecture using JSF and Struts framework and implementing custom tag libraries.
• Developed the GUI module DMT using Struts framework, JavaScript, DOJO, Ajax, HTML and DHTML.
• Building actionable and data-driven analytics by evaluating Offers (products) utilized across different geographies
• Involved in various stages of data preparation for model training and inferencing
• Developed python scripts to cleanse and automate the data retrieval process
• Created Segments, custom metrics, and dimensions for advanced analytics
• Developed SQL queries to get data from Database
• Co-ordinated with offshore development teams
• Assigned roles and permissions for all the users based on the security protocol defined by the key stakeholders

Confidential

Data Engineer

Responsibilities:

• Performed in-depth analysis for multiple IT and non- IT programs which includes understanding the business needs and translating to analytic requests, extracting data from a variety of sources like Teradata, oracle, manipulate and analyze using quantitative, statistical, and visualization using tools like Tableau, Excel, and SQL assistance.
• Built, published customized interactive reports and Tableau dashboards, report scheduling using Tableau server. Created action filters, parameters, and calculated sets for preparing dashboards and worksheets in Tableau.
• Met with various groups, including business owners, SMEs (subject matter experts) and marketing team, for requirements gathering in definition Stage
• Built end to end reporting layer in Tableau Dashboard for analytical, operational, real- time and adhoc reporting environment. Defined historical and incremental refresh frequency in tableau for the data refresh
• Developed Tableau workbooks from multiple data sources using data blending and interactive views, trends, and drill downs.
• Responsible for creating /delivering monthly reports to business users.
• Worked with the project manager to estimate best/worst case scenarios, track progress with weekly estimates of remaining work to do, conducting informal meetings ad hoc and as needed.
• Wrote adhoc queries for business needs. Adapted agile scrum methodology.