Home / Blog / Data & dashboards

Real time reporting: when live data is worth it, and when hourly is enough

By Philip van den Berge, PL-30010 November 20268 min read
a clock on a pole in front of a building
Photo by Imad Alassiry on Unsplash

Real time reporting is worth paying for when someone acts on the number within minutes: a planner moving a truck, a team lead shifting people to the phones, a marketer pausing a campaign that is burning budget. For most other reports, a refresh every hour or once a night supports exactly the same decisions, costs less to build and gives you fewer things that can break. So before you ask for live data, work out how quickly anyone acts on each number, because that answer sets the refresh rate.

What real time reporting means in practice

When a client asks me for real time reporting, my first question is what they would do differently if the number were ten minutes old. Quite often the answer is nothing. What they want is a report that is current when they open it, without someone exporting, pasting and emailing it first. That is a scheduled refresh problem, and it is much cheaper to solve than a streaming one.

Strictly speaking, real time means the screen changes within seconds of the event, so an order comes in and the counter moves. Near real time means a lag of one to a few minutes. Most of what gets sold as a live dashboard sits in the second group, or refreshes every 15 minutes, and for nearly every business decision I have worked on that is plenty. The label matters less than the delay you can accept, so I write that delay into the spec as a number per metric, for example no older than 15 minutes during working hours, and build to it.

Four refresh levels, from streaming to nightly

Each step up adds cost to the build, to the monthly running bill and to the attention the setup needs when something fails. Going from nightly to hourly is usually a small change to a schedule. Going from hourly to streaming often means a different architecture, with different tools, and that is where budgets get surprised.

Say a 30-person distributor asks for a live sales dashboard. If the sales director reads it each morning and discusses it at a weekly meeting, an hourly refresh from the order system covers every decision that dashboard will ever support. If the same company also runs a dispatch desk that promises same-day shipping, the open orders page for that desk is the one place where a five minute refresh makes a difference. Same company, same data, two different answers, and only one of them needs the more expensive setup.

When a live dashboard is worth the cost

A live dashboard earns its cost when the decision is made during the working day, when acting late costs money, and when the person looking at the screen can actually do something about it. Take away any one of those and a slower refresh does the same job. These are the cases where I think it is usually justified.

The logistics control tower example shows the kind of screen I mean, with on-time delivery by hour against a target, lanes coloured by status and a table of live shipments with their ETA. It runs on sample data for a fictional carrier, so nothing streams into it, but a real version would pull from the transport management system and truck telematics every few minutes, because that is how quickly a planner can respond.

When hourly or nightly reporting is enough

Most management numbers move more slowly than the meetings that discuss them. Revenue by month, gross margin, pipeline value, days sales outstanding and headcount are read weekly or monthly and acted on over weeks. A dashboard that refreshes at six every morning gives a finance lead everything they need, and it avoids a problem that live finance figures create, which is that they change while you are looking at them. A revenue total that shifts halfway through a meeting starts an argument about which figure is right, and that argument is a worse use of the hour than the late data ever was.

Some sources can't deliver faster anyway. Bank data often arrives once a day, many accounting packages post entries in batches, and the month isn't final until the close. If a source only changes overnight, refreshing the dashboard every minute just reads the same rows again. A business intelligence dashboard for management is almost always in this group.

There is also a middle ground that I use a lot. Refresh every hour during working hours and once overnight. That puts the morning's orders on screen by lunch, keeps API calls modest and costs next to nothing extra.

My rule of thumb: refresh about as often as someone can act on the number. Anything faster costs money and shows no one anything new.

What a real time dashboard costs to build and run

The build cost rises with freshness because of the plumbing around the dashboard, not the charts. Many APIs cap how many calls you can make per minute or per day, so polling every minute across a few hundred records can hit the limit before lunch. Streaming needs the source to push events, and plenty of business systems simply don't offer that. And the more often a job runs, the more often it can fail, so monitoring becomes part of the work.

If your reporting lives in Power BI, the platform sets some limits for you. Microsoft's documentation on data refresh says semantic models on shared capacity (the normal Pro setup) can be scheduled to refresh up to eight times a day, and models on Premium, Premium Per User or Fabric capacity up to 48 times a day. For anything fresher you switch to DirectQuery, where the report queries the source on each interaction, and that mode has its own documented limits, including a one million row cap on data returned and a 225 second query timeout. Automatic page refresh, which re-queries the visuals on a timer while the page is open, only works with DirectQuery and similar sources, and on shared capacity its shortest interval is 30 minutes.

In practice, near real time in Power BI usually means a source database that can take the query load, a Premium or Fabric capacity, or both. When a client wants live numbers on a wall screen, or for viewers who don't have a Power BI licence, I often build a small web dashboard on Postgres instead, with a scheduled job pulling from the APIs. If your problem is a report that takes ages to load rather than one that is out of date, that is a different fix, covered in how to fix a slow Power BI report.

Choosing a refresh rate for a live dashboard: five questions

  1. Who opens it, and when? A screen on the warehouse wall is looked at all day. A board pack is opened once a month.
  2. What do they do when a number moves? If the honest answer is that they raise it at the next meeting, hourly or nightly is enough.
  3. What does an hour of delay cost? Try to put a rough figure on it. If you can't think of one, live data probably won't change much.
  4. How often does the source really change? Check what the API or export allows, and how often the underlying records are updated.
  5. Who notices when it breaks? A live feed that fails silently does more harm than a nightly one that is honest about its date.

I write the answers into the spec next to each metric. One dashboard often ends up with mixed rates, say open orders every five minutes, revenue hourly and margin overnight. That is fine, as long as every tile shows how old its data is.

Making a live data dashboard people can trust

A dashboard that shows stale data as if it were current is worse than no dashboard, because people make decisions on it. Dashboards fed by APIs tend to break without telling anyone. A token expires, a provider renames a field, a rate limit kicks in at month end. These are the things I put into every live build.

Sometimes the better answer is no screen at all. If what you need is a message when a KPI crosses a line, a KPI alert that emails or messages you can be cheaper than a dashboard someone has to keep watching.

Where to start

If you're unsure, start at hourly. It is cheap to build, it covers most operational needs, and moving one feed to every five minutes later is a small change when the data model is built properly. Going the other way, from a streaming setup back to hourly, usually means throwing work away. After a few weeks you can see which numbers people actually watch during the day, and only those need to get faster.

If you'd like help deciding, I build custom live dashboards on your own data and APIs, and the first conversation is usually about exactly this: which numbers need to be live and which can wait until morning. For reports that should stay in Microsoft's tools, have a look at Power BI dashboard development.

Frequently asked questions

What is real time reporting?

Real time reporting means a report or dashboard that updates within seconds or minutes of the underlying event, so the numbers reflect what is happening now. In practice, most business dashboards described as real time refresh every few minutes or every hour. The right delay depends on how quickly someone acts on the number, which for most management reporting is hours or days, not seconds.

Does Power BI support real time dashboards?

Yes, with limits. According to Microsoft, import models on shared capacity can be scheduled to refresh up to eight times a day, and models on Premium, Premium Per User or Fabric capacity up to 48 times. For fresher data you use DirectQuery, optionally with automatic page refresh, or push data into the model. Each option adds cost, load on the source or query limits.

How often should a live dashboard refresh?

As often as someone can act on the change. A dispatch screen or support queue may need updates every one to five minutes. Sales and operations overviews are usually fine hourly. Finance and management numbers are best refreshed overnight, after the day is closed, so figures don't move during a meeting. One dashboard can mix these rates if each tile shows its last update.

Is a real time dashboard more expensive than a daily one?

Usually, yes. A faster refresh means more API calls, more load on the source system, more monitoring and sometimes a higher licence tier, as with Power BI Premium or Fabric capacity. Streaming can also need a different architecture. Moving from nightly to hourly is normally a small step, while moving from hourly to streaming is a large one.

Can I start with hourly refresh and make it live later?

Yes, and it is the approach I recommend when you are unsure. If the data model and history table are built properly, shortening the refresh interval for one feed later is a small change. Start hourly, watch which numbers people check during the day, and make only those faster. That way you pay for live data only where it changes a decision.

Want reporting your team actually uses?

Tell me what you report on today and where the data lives, and I will send a fixed quote. Usually within a few hours on working days.

Get a free quote →