German Grid Forecasts: Where the Uncertainty Bands Fail

The Transparency Platform publishes each TSO’s day-ahead point forecasts alongside realized values. That pairing makes an audit possible without anyone’s permission: the forecast was committed before delivery, the outcome is measured after it, and the difference (the error e = actual − forecast) accumulates into a track record nobody curates.

Two rules govern everything below. First, any quantity fitted from history (a mean, a standard deviation, a quantile) uses a trailing 90-day window that ends the day before the observation it is scored on. No method ever sees its own exam. Second, every construction is evaluated per quarter-hour delivery slot, because grid errors have a schedule: a method that is right on average and wrong every noon should be caught, not excused.

The aggregate numbers look respectable: load bias −385 MW on a ~55 GW system, solar −56 MW, offshore wind +176 MW. The clock tells another story.

Load is over-forecast by 1.8–2.0 GW at hours 11–14: roughly two large power stations of demand that is predicted daily and daily fails to materialize, five times the size of the aggregate bias. The likely mechanism is well known (behind-the-meter solar eroding metered load faster than the forecast model learns); the size is something you only learn by measuring. Offshore wind leans the other way, under-forecast by ~240 MW through the night hours; solar’s midday under-forecast peaks around −420 MW.

The TSO publishes points, not ranges. Any desk wanting a range constructs it, and the standard construction is parametric: fit mean and standard deviation to recent errors, quote $\mu \pm z\,\sigma$. The textbook distribution-free alternative replaces the Gaussian fit with trailing empirical quantiles: a rolling split-conformal band. We ran both, identically windowed, strictly out-of-sample; and a third method introduced in § 04.

A desk with a quantitative team will say it does not build bands this way, and it is right. It runs quantile regression on weather features, or it buys a probabilistic forecast with bands already attached. That objection does not weaken the audit; it describes the normal case for it. What is under test is not how a band was built but whether the number printed on it is true. The procedure here scores whatever band you already quote, from whatever construction, against the outcomes you already have: it counts how often the band contained the answer, splits that count by regime, and prices the misses. Nothing in it needs to know the model. The two constructions audited below are in the paper because they are public and reproducible, not because they are the state of the art, and a better-built band is a more interesting subject for the same test rather than an exemption from it.

At nominal 90%, the Gaussian band delivered 84.8–88.9% depending on series; rolling conformal delivered 83.2–86.6%. Neither is a 90% band. The aggregate conceals worse: solar’s nominal-80% construction covered 56–63% through spring 2026, precisely when new solar capacity made the error distribution grow faster than any trailing window could learn.

Adaptive conformal inference (Gibbs & Candès, 2021) treats the miss rate as a control problem: after every observation, the target miss rate $\alpha_t$ is nudged,

so the very next interval widens after a miss and tightens after a long run of hits. The step size γ is a plain tuning constant (larger reacts faster and wobbles more), and it was fixed at γ = 0.02 before any evaluation, not tuned to the results. Same trailing windows, same no-lookahead discipline, same data.

The result, visible in Fig. 02 for every series and level: achieved coverage lands at 89.0–89.7% against a 90% promise, 79.3–79.9% against 80%, 93.1–94.6% against 95%. The price is stated rather than hidden, and stated in the tail, not just the mean, because an average width in an article about the sins of averaging would be an irony. The adaptive load band at 90% averages 8.8 GW against the Gaussian’s 7.5 GW; at the 95th percentile of widths it quotes 14.2 GW against the Gaussian’s 11.1 GW, and its single widest interval reached 19.3 GW (onshore wind’s worst: 25.8 GW). Those wide moments are the method telling you it recently missed and does not yet trust the window. That is information, not noise. The Gaussian’s narrower quotes at those same moments were never actually 90% bands. Calibration does not make uncertainty larger; it stops you understating it.

Where the aggregate table still flatters everyone is regime. The month-by-month cut is the figure this article’s own Recipe demands:

The month-by-month cut also keeps us honest about what adaptivity is. When the solar shift hits in March 2026, all three methods get hurt: ACI’s worst month is 65% against an 80% promise, because a reactive method cannot see a regime break coming. The difference is what happens next: ACI walks back to 74% in April, 79% in May, 82% by June, while the static constructions sit in the mid-50s to low-60s for four consecutive months, still quoting “80%” the whole time. Adaptivity is not immunity; it is recovery measured in weeks instead of quarters, and a stated miss you can see happening, instead of a silent one.

What do these percentage points cost? German imbalance answers, because reBAP is published quarter-hourly by the four TSOs on netztransparenz.de. Pricing every escape from the trailing Gaussian “90%” band at that quarter-hour’s adverse imbalance-versus-day-ahead spread (favourable spreads clipped to zero: risk side only), scaled to a portfolio with 100 MW mean load whose errors move proportionally to the system’s (an assumption, stated): €90,000 of un-reserved adverse exposure over the 13 months, against €44,500 under the adaptive band: a ≈€45,500 difference per 100 MW. The stated band missed 15.2% of quarter-hours. And the property the averages hide: the mean adverse spread over all intervals was €24.6/MWh, while on the intervals the Gaussian band missed it was €34.8/MWh. The band fails disproportionately in exactly the intervals where failing is expensive. Miscalibration and market stress share causes; that correlation is the audit’s sharpest commercial fact.

Beyond that, this analysis can show whether a published point forecast is conditionally biased, and whether a band constructed around it holds its promised frequency. Those are checkable against outcomes. It cannot show whether your forecasts share these failure modes: portfolio forecasts differ from system forecasts, and a desk’s decision layer consumes uncertainty differently than a TSO’s.

One consequence deserves naming, because it is where this method has teeth. If you buy forecasts from a vendor who quotes confidence bands, everything above applies to them exactly as it applies to the TSO. Coverage is checkable by the customer, from data the customer already holds (committed forecasts and realized outcomes) without the vendor’s cooperation or their model. A vendor whose “95%” bands blow through weekly in a windy quarter is not suffering “unprecedented weather”; they are failing an audit nobody has run. Recipe 4.1 doubles as the four questions to put in your next RFP or quarterly vendor review.

The audit methodology (the pairing discipline, per-slot scoring, regime-conditional coverage, the adaptive repair) transfers unchanged to any stack that can export twelve months of forecasts and outcomes.