“GPT-6.1 Sol nearly matches the no-CoT performance of GPT-6 Astra and is likely a looped transformer” by Rauno Arike

Summary

I ran GPT-6 Sol and GPT-6.1 Sol on the task suite from Think Fast. Surprisingly, 6.1 Sol performs substantially better than 6 Sol, almost matching the performance of GPT-6 Astra. The plot below gives a quick overview of the results:

Measured by mean accuracy across the 27 tasks, GPT-6.1 Sol closes 80% (95% CI: 72–87%) of the gap between GPT-6 Sol and GPT-6 Astra, and is closer to Astra than to GPT-6 Sol on 24 of 27 tasks.

The likely reason behind this gap is that, like GPT-6 Astra and unlike GPT-6 Sol, GPT-6.1 Sol is a looped transformer. After providing a more detailed overview of the benchmark scores, I'll briefly discuss the evidence for this, as well as the implications.

Detailed results

Similarly to Astra, GPT-6.1 Sol saturates many of the benchmarks in the task suite, rendering the time horizon estimates highly uncertain. For this reason, I mainly focus on per-benchmark performance, which already provides a sufficient demonstration of the gap between 6 Sol and 6.1 Sol on its own. The time horizon estimates were 4.0 minutes for GPT-6 Sol (bootstrap median 3.8 min, 95% CI [1.2 min, 20 min]) and 35 minutes for GPT-6.1 Sol (bootstrap [...]

---

Outline:

(00:15) Summary

(01:27) Detailed results

(04:26) What caused the jump?

(05:41) Appendix: Full per-task results

(05:46) GPT-6 Sol: per-benchmark 50% no-CoT time horizons

(06:12) GPT-6.1 Sol: per-benchmark 50% no-CoT time horizons

(06:37) Appendix: Logistic fits

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
September 30th, 2026

Source:
https://www.lesswrong.com/posts/LqSZZAriGqgsGDQe3/gpt-6-1-sol-nearly-matches-the-no-cot-performance-of-gpt-6

---

Narrated by TYPE III AUDIO.

---