Performance
How we keep score
A forecasting product that grades its own homework is worth nothing. This page describes exactly how accuracy is measured and published — before there is a record worth boasting about.
No track record yet. Sheppi is in early access and has not accumulated enough settled predictions to publish a meaningful accuracy figure. Rather than show you a number we invented, this page shows the method. Results will appear here as they settle.
Four commitments
These are the rules the record will be kept under. They are published first so they cannot be adjusted afterwards to suit the results.
Every forecast is timestamped before settlement
A prediction is recorded when it is made, not after the result is known. Nothing is edited retroactively, and the timestamp travels with the call.
Results are published win or lose
Settled predictions keep their original probability next to the outcome. A model that only shows its good weeks is not showing you a model.
Calibration, not a win rate
The question is whether things we call 70% happen about 70% of the time. A win rate rewards forecasting only the obvious; calibration does not.
Human review is labelled as human
Where an analyst has adjusted or disagreed with the model, you see both figures and who said what. The two are never blended into one number.
Calibration, not a win rate
A win rate can be manufactured by only forecasting things that were never in doubt. Calibration cannot.
When enough predictions have settled, this page will show a calibration curve: of every forecast made at roughly 70%, what share actually happened. A well-calibrated model sits near the diagonal. One that sits above it is underconfident; below it, overconfident — and an overconfident forecaster is a dangerous thing to follow.
Alongside it: the sample size, the window, and every settled prediction with its original probability and timestamp intact. Losses included, at the same size as the wins.
Nothing on this page will ever be a projection, a backtest presented as a record, or a figure with the losing period removed.
Watch it being built
Join early access and you will see the record accumulate from the beginning — including the parts that do not flatter us.