Why we publish our Brier score (including when we miss target)

Visit ten random football-prediction websites and try to find their Brier score. Or their log-loss. Or any quantitative claim about how accurate their model has been over the past month. The exercise won't take long. Most of them never publish anything.

At MatchMind we publish ours on a public page that doesn't require a signup. Here's the reasoning — and the consequence, which is that we'll sometimes have to publish numbers we don't love.

Why most sites don't

Posting your accuracy publicly comes with one big risk and one big benefit. The risk: you can be measured. If you said the home team had a 70% chance and they lost, that's in the dataset. If you do that twenty times in a row, your Brier score gets worse and visitors can compute it. There's nowhere to hide.

The benefit is the same thing: you can be measured. Sites that publish honestly are the only ones who can credibly claim their probabilities mean something. A 70% probability that turns out to be wrong 50% of the time is not a 70% probability — it's a marketing number.

Tipster sites usually pick the risk-free option: post big confident calls, dump the misses, count the hits in screenshots. The math is shameful but the business model works because almost no visitor goes looking for the receipts.

Why we picked the other option

Three reasons.

One:the audience MatchMind is for is the data-curious fan. That fan already knows tipster sites are hollow. If we don't show the receipts, we look like the same thing they've been ignoring for years. Calibration transparency is table stakes for this audience, not a flex.

Two:it's a moat competitors can't cross. A traditional tipster site can't suddenly start publishing Brier scores without admitting their model has never been calibrated. That admission would torch their existing user base, who came for “sure picks” not probabilistic estimates. So the position is defensible: we hold a piece of ground they can't take from us without burning their own house down.

Three: it forces us to actually be good. Knowing the Brier score will be on a public page next week changes how you build the model. It kills the temptation to over-fit, to cherry-pick the metric that flatters you, to ship a feature before checking calibration drift. The visibility is a discipline.

What it costs us

Honestly? Sometimes a bad week. The Brier score we're publishing is the average over our live evaluations. When a round of upsets lands, the number goes up. It went up two weekends ago. It'll go up again. That's the deal we made when we put the page online.

We've thought about whether to show a rolling window (last 30 days) or a cumulative number (all live evaluations to date) on the landing page. Right now it's cumulative because the live sample is still small. As the sample grows, we'll switch to a rolling window so the headline number reflects the model's current behaviour, not its first month forever. Whatever we display, the methodology stays on the page.

What this means for you

Two practical things. First: if you visit the Track Record and the number is worse than you remember, that's real. It went up. We'll tell you why in the next post if it stays bad. Second: don't take our word for the methodology — read the “How to read this page” section on Track Record. Brier's mean is a known thing, the uniform baseline is a known thing, and you can sanity-check our claims against external football statistics literature.

That's the entire pitch: we're willing to be wrong in public so you can decide whether to trust us when we're right.