Go Fast Using Data Virtualization

During a recent house move I discovered an old notebook with metrics from when I was in the role of a Data Warehouse Project Manager and used to estimate data delivery projects. For the delivery a single data mart with supporting Reports and Dashboards, with 6–8 dimensions and a handful of measures or facts, the delivery cost was approximately $200K AUD per data initiative and it needed a delivery team of 8–10 resources over a duration of roughly 12 weeks.

In this example we used more traditional methods of data delivery. We used an ETL approach for all data movement (in this case DataStage) and an agile, iterative methodology for development and deployment. The data would move from the source system to a staging area (making it easy to reconcile the data being captured), then the data would move to an ODS or Operational Data Store (to service the business from an operational reporting perspective), then the data was modelled and moved to the EDW or Enterprise Data Warehouse (for historical storage and time-series analysis), then the data was moved into subject matter specific data models (quite often star-schema designs) and then finally multi-dimensional cubes (used for more in-depth analysis and reporting).

Read more in https://www.datavirtualizationblog.com. Originally published on January 14, 2022




We do #DataVirtualization We care about #AgileBI, #BigData #Analytics, #Dataservices, #DataManagement, Logical #DataWarehouse Web, #SaaS and #Cloud integration.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Docker Best Practices: Images

Highly Expert In Magento 2

The 16 Best Application Integration Tools to Consider for 2021

How to Utilize your Scrum Master Part 2: Master of Scrum Ceremonies

A quick recap of the Event Sourcing concept

Mocking External Dependencies in Go

env0 and Terraform Cloud: A Difference of Opinion

About MSIGHT Team: How Data Product Team is Organized

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store


We do #DataVirtualization We care about #AgileBI, #BigData #Analytics, #Dataservices, #DataManagement, Logical #DataWarehouse Web, #SaaS and #Cloud integration.

More from Medium

Why are Data Warehouses evolving to Lake Houses? Part2 — Access raw data after ETL

Ensure data quality by using Great Expectations

dbt Setup and Installation

High level overview of how dbt aids in the ELT process