Back to the cube

Warehouse efficiency / 2 min read

Does a dashboard need to refresh every five minutes?

The right refresh interval follows the business decision, source timing, and downstream dependencies.

A busy schedule is a reason to investigate

A finance dashboard may refresh throughout the day even when its users make a daily decision. That pattern can justify an investigation, but the viewing schedule alone is insufficient evidence for slowing the pipeline. Another report, operational queue, or control may depend on the same tables.

The workload owner first identifies who consumes the output, when the source actually changes, and how late information can arrive before the decision is affected. A five-minute schedule may be justified for one consumer and unnecessary for another. The goal is an explicit service requirement.

An illustrative comparison

Consider a hypothetical dashboard that runs continuously, with no failed runs, backfills, or additional consumers. A five-minute schedule triggers 288 runs in 24 hours. An hourly schedule triggers 24. That is 264 fewer scheduled runs, or about 91.7% fewer runs.

This is scheduling arithmetic, not a measured client result. It does not establish a 91.7% reduction in cost, electricity, or emissions. Query shape, caching, capacity commitments, source volume, retries, and any catch-up processing affect the actual result. An hourly refresh is only a candidate if it satisfies the business requirement.

Separate display refresh from warehouse work

A dashboard reload may use a cached result, while a background transformation may execute independently of the dashboard. Query history and orchestration records help identify which activity actually causes processing. The investigation should follow the relevant job chain instead of treating each page refresh as a warehouse query.

The team can then examine whether the workflow reads unnecessary data or repeats transformations. BigQuery guidance on query computation provides a technical companion for that review. The selected change may involve the query, the schedule, the dependency chain, or a combination of them.

Reference: Google Cloud: optimizing BigQuery computation.

Test one change against the service promise

A bounded trial compares equivalent outputs and representative workload volumes before and after a proposed change. The record includes scheduled and actual runs, bytes scanned or compute time, attributable cost, output quality, and the time information becomes usable. Peak periods and late-arriving corrections need representation.

The owner agrees on the maximum acceptable delay and a rollback condition before the trial starts. A lower processing count has limited value if it causes missed reporting deadlines or creates extra manual reconciliation.

A useful first step

A data leader can begin with one dashboard and a short inventory of its consumers, source update times, upstream jobs, and freshness requirements. A Data Waste & Efficiency Diagnostic turns that evidence into a prioritized change and measurement plan. Any expected savings remain qualified until the operating comparison is complete.

A relevant next step

Data Waste & Efficiency Diagnostic

Tera Earlywine offers a paid, focused review of one warehouse and a defined workload set. The diagnostic produces a baseline, ranked improvements, explicit assumptions, and a measurement plan. Billing and query history, architecture context, and a workload owner establish the starting point; implementation is scoped separately.

Discuss a warehouse efficiency decision