“Measuring no CoT math time horizon (single forward pass)” by Ryan Greenblatt

Subtitle: Opus 4.5 has around a 3.5 minute 50%-reliablity time horizon.

A key risk factor for scheming (and misalignment more generally) is opaque reasoning ability. One proxy for this is how good AIs are at solving math problems immediately without any chain-of-thought (CoT) (as in, in a single forward pass). I’ve measured this on a dataset of easy math problems and used this to estimate 50% reliability no-CoT time horizon using the same methodology introduced in Measuring AI Ability to Complete Long Tasks (the METR time horizon paper).

Important caveat: To get human completion times, I ask Opus 4.5 (with thinking) to estimate how long it would take the median AIME participant to complete a given problem. These times seem roughly reasonable to me, but getting some actual human baselines and using these to correct Opus 4.5's estimates would be better.

Here are the 50% reliability time horizon results:

I find that Opus 4.5 has a no-CoT 50% reliability time horizon of 3.5 minutes and that time horizon has been doubling every 9 months.

In an earlier post (Recent LLMs can leverage filler tokens or repeated problems to improve (no-CoT) math performance), I found that repeating the problem [...]

---

Outline:

(02:41) Some details about the time horizon fit and data

(04:29) Analysis

(07:11) Appendix: scores for Gemini 3 Pro

(11:51) Appendix: full result tables

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
December 26th, 2025

Source:
https://blog.redwoodresearch.org/p/measuring-no-cot-math-time-horizon

---

Narrated by TYPE III AUDIO.

---