Backtest a betting idea before you trust it

Sports betting backtesting software runs a betting rule over past games, using only the prices and information that existed when each bet would have been placed, and tells you whether the result could be luck. The quant+ Backtest Lab does that for NFL, college football, NBA, college basketball, MLB and NHL sides, totals and moneylines. It blocks look-ahead in the form itself, reports intervals instead of a single number, and scores the rule season by season with walk-forward testing.

Most backtests that look great are wrong in one of three ways. They peek at the future, they test too many ideas, or they ignore how small the sample is. This page covers how the lab handles each.

How you define a strategy

A backtest in the lab is a rule you build from a form, not code:

• Scope: leagues, season range, optional date range, regular season or playoffs.
• Market: spread, total or moneyline.
• Side: home, away, favourite, underdog, over, under, with the line move or against it.
• When the bet is placed: when lines open, or at kickoff (the closing price).
• Which price: the consensus, one named book, or the best available.
• Filters: week, venue, favourite or underdog, divisional game, neutral site, rest advantage, spread and total ranges, how far the line or price moved, temperature, wind and roof.
• Staking: flat units, a percent of bankroll, or fractional Kelly.

Two presets show the idea: "Home dogs +3 to +7" and "Unders in 15+ mph wind". Change one field and run it again.

How the lab prevents look-ahead bias

Look-ahead bias means the backtest uses something you couldn't have known at bet time. In betting it usually hides in one of these:

• Using the closing line to choose a bet, then pretending you got the opening price.
• Filtering on line movement while betting at the open, before the move happened.
• Using game-day weather for a bet placed on Tuesday.

Machine learning has the same trap. The scikit-learn documentation warns that ordinary cross-validation on time-ordered data would lead to training on future data and evaluating on past data, which is why time-series splits train only on what came before.

In the quant+ lab, every input is tagged with when it becomes known. The schedule and the line are known at the open. Line movement, weather and roof status are only known near kickoff. If you choose "bet when lines open", the form marks movement and weather filters as "close only" and won't run them. The price taken can never come from before the moment your rule could have fired. That check is enforced by the engine and covered by tests, so it doesn't depend on you remembering.

Why one ROI number isn't enough

A rule that won 56 percent of 80 bets sounds good. It mostly tells you that 80 bets is a small number.

The lab reports:

• Profit and ROI with a 95 percent bootstrap interval: the range of results you'd plausibly see if the same edge produced a different run of games. If that range comfortably includes zero, you haven't found anything yet.
• A p-value against break-even, so you know how surprising the result is if the rule had no edge at all.
• Win rate against the break-even rate for the prices you actually took.
• Closing line value, when the rule bets before the close. A rule that wins without beating the close is usually a lucky rule.
• Max drawdown and the longest losing and winning streaks, plus the equity curve.
• Walk-forward by season: the rule is scored on each season using an expanding window of earlier seasons, the way you would actually have used it.

The interval and p-value are hidden on very small samples. Showing a confidence interval on a dozen bets would be false precision.

The multiple-testing trap

The quickest way to fool yourself is to try twenty filters and keep the best one. If you test 20 unrelated rules that each have no edge, the chance that at least one passes a 5 percent significance test by luck alone is 1 − 0.95^20, about 64 percent (our arithmetic).

Finance has studied this for years. Bailey, Borwein, López de Prado and Zhu found that standard defences such as hold-out samples tend to be unreliable and inaccurate in the context of investment backtests, and proposed measuring the probability that a backtest is overfit. Betting rules suffer the same problem with far smaller samples.

What we suggest:

1. Write the rule down before you run it, with the reason you expect it to work.
2. Count every variation you tried. The lab keeps "How this run was made" (engine version, seed, resamples, de-vig method, data sources) with each run, so you can retrace it.
3. Trust walk-forward over the full-sample number. If the rule only works in one season, it doesn't work.
4. Check the close. A real edge usually shows up as positive closing line value.

Where the data comes from

The lab uses historical closing lines from nflverse for the NFL, opening and closing main lines from OddsPapi across the six leagues, college lines from CollegeFootballData and CollegeBasketballData, and quant+'s own price collector, which stores every change on the board. Each run lists the sources it used. NFL history goes back furthest; other leagues have shorter histories, and the lab tells you when a sample is thin.

Player props are not in the lab yet. Prop prices and box scores are on the props board, but we won't publish prop backtests until the historical prices are deep enough to be honest about.

Limits that apply to every backtest

Past prices aren't future prices. Markets adapt. A 2024 study of NFL money lines from 2007 to 2016 found inefficiencies, but noted they were unpredictable and varied by season.

You may not get the stake. Books limit accounts that keep beating them; researchers betting a market-based strategy had stakes capped within months, one to $10.45 on a $50 bet. A backtest assumes you got on in full.

A model can be worse than the market and still useful, or better on paper and useless. Hubáček and Šír showed a model can find value while less accurate than the market if it's decorrelated from it. The backtest is where you check which one you have.

quant+ never places bets or connects to sportsbook accounts. Read the risk disclosure before acting on any result.

Run your first backtest

Pick one idea you already believe in. Run it with walk-forward on, bet at the open, and look at three things: the interval, the closing line value and the worst drawdown. Then log any live bets in the journal so you can compare the real thing with the test. The quant+ guides explain closing line value and fair odds if those are new, and the +EV scanner uses the same de-vig methods as the lab.

Start free to open the Backtest Lab, or see it on the home page.

Quant+ is analytics for informational purposes. 21+. If you or someone you know has a gambling problem, call 1-800-GAMBLER.