AI-generated forecasts fail in specific, predictable ways. I have watched three major vendor tools (Anaplan, Pigment, Mosaic) produce revenue forecasts that were within 3% of actuals for six consecutive months, and then miss the seventh by 22% on the same P&L line. Not because the tool broke. Because the input regime shifted and nothing in the model noticed.

If you are the CFO signing off on an AI-generated forecast that goes to the board, you need a framework for catching the miss before it happens. This is mine.

The three failure modes

Every bad AI forecast I have seen falls into one of three failure modes. Not many. Three.

Mode 1: Regime change the model cannot see

Across the mid-market finance teams I have worked with running AI-assisted forecasts, median revenue forecast error typically lands in the low- to mid-single digits over a twelve-month horizon, with a long right tail where individual companies miss badly. Almost every 90th percentile miss was a regime change.

Examples: a competitor exited the market, your top customer got acquired, tariffs changed on a raw material, a new product cannibalized an old one, your VP of Sales got poached with three AEs.

The model does not know these happened. It sees the historical pattern and projects forward. Six months in, revenue was fine because the change had not shown up in the data. Month seven, it shows up all at once.

The check: before you accept any AI forecast, write down a list of the 3 to 5 external or internal changes since the last forecast cycle. Ask whether the model has been re-trained or re-briefed on these. If not, adjust manually.

Mode 2: Overfit to seasonality

The second most common failure is a model that has learned a false seasonal pattern from noisy history.

I saw this at a subscription business where the model kept projecting a February revenue dip because two of the last three Februaries were down. Both February dips were caused by a specific product outage that had nothing to do with seasonality. The model treated it as pattern.

The check: for every recurring dip or spike the model projects, ask what caused the historical version. If the cause was a one-time event, the model is wrong. Datarails and Cube both have “annotate historical events” features. Use them. If you are on a spreadsheet-based forecast, keep a log.

Mode 3: Compounding small errors

The third failure is subtler. Individual line-item errors are small (1 to 3% each). The forecast rolls up to a total that is 8 to 12% off because the errors compound directionally.

I have seen this most often in cost forecasts. If the model overprojects headcount growth by 2%, salaries by 1%, benefits by 3%, contractors by 4%, and G&A by 2%, your total OpEx forecast is off by 7% and everyone on the finance team was checking individual lines.

The check: cross-foot every category. Forecast total OpEx as a percent of revenue. Compare to trailing four quarters. If the percent is more than 200 basis points off recent trend without a stated reason, dig.

The five sanity checks I run

Before I accept an AI forecast:

1. The pattern break check. Look at the last 8 quarters of each major line. Does the forecast break any established pattern? If yes, is there a stated reason? If no stated reason, reject.

2. The driver reasonableness check. For revenue, back into the implied driver assumptions. Salespeople x quota x attainment rate = new bookings. Does that match what your CRO believes? For cost of revenue, back into unit economics. If the model is projecting a 4-point gross margin improvement, what changed to enable it?

3. The “what would have to be true” check. For each material variance vs. prior period or plan, ask: what would have to be true in the operating business for this to happen? If the answer is “nothing has to change,” the forecast is probably right. If the answer is “we would need to hire 12 new AEs, and we have not started recruiting,” the forecast is wrong.

4. The 13-week cash overlay. Rebuild the forecasted cash flow in a 13-week model. If the forecasted P&L implies cash flow that does not match your 13-week model, one of them is wrong. See How to Build a 13-Week Cash Flow Forecast.

5. The last-forecast delta. Compare the current forecast to the one you produced last month. If the change is significant (greater than 3% on revenue, greater than 5% on EBITDA), what specifically drove the delta? “The model updated” is not an answer. If nobody can explain the delta in one sentence, the delta is wrong.

The human-in-the-loop pattern

The pattern I use: the AI produces the base forecast. I produce three adjustments in a top-sheet: (1) explicit assumptions I am overriding, (2) known events not in the model, (3) sensitivity range. The adjusted forecast is what goes to the board.

At the $340M distribution company I mentioned in Why Most CFOs Are Using AI Wrong, the CFO’s top-sheet typically has 5 to 8 adjustments. Total effect on the forecast: 2 to 4% up or down. Every adjustment is one sentence explaining the override and the source.

The board sees both the AI-generated base and the CFO-adjusted forecast. They see the deltas. They see the reasoning. If the CFO is systematically over-adjusting or under-adjusting, that shows up over quarters and it is a good conversation to have.

What good looks like

In conversations with finance leaders using AI in forecasting, my rough split is that about half say their accuracy has improved, roughly a third say it is unchanged, and the rest say it is worse. And in both the “better” and “worse” camps, a large share attribute the change directly to their AI tools.

Same tool, opposite outcomes. That is because the tool is only as good as the sanity-check discipline sitting on top of it. The ones who improved had a framework. The ones who declined trusted the model.

The counter-argument

“If I am going to sanity-check every AI forecast this thoroughly, why bother with AI?”

Because the alternative is worse. The pre-AI CFO forecast at the same $180M SaaS business I mentioned in The Weekly AI-Audit Every Controller Should Run had a median revenue forecast error of 11.4% over the prior three years. The AI-assisted forecast (with sanity checks) had a median error of 3.7% over the last twelve months. The sanity checks take about 90 minutes per forecast cycle. Worth it.

The second objection: “The vendors say their model catches regime changes with the anomaly detection module.” Some do, some do not. In practice, the anomaly detection catches historical anomalies, not future ones. A regime change is a future anomaly. Your job is to catch it.

The third objection: “Are you saying AI forecasting is not ready?” No. I am saying it is ready with a human in the loop and dangerous without one. See 6 Things You Should NEVER Ask an LLM as a CFO for related territory.

The read

If you are building or buying an AI forecasting stack, the vendor bake-off comparison in The 2026 AI CFO Benchmark is where I would start. Real pricing, real feature comparison, no vendor slop.

The full sanity-check framework, plus the 5-question override template, is in The AI-Native CFO Prompt Pack Pro.

Note: Company details in this piece have been anonymized. Any figures drawn from Spencer’s advisory work with middle-market and PE-backed finance teams are directional.