“Lessons from building an automated research scaffold” by Alejandro Aristizabal, Josh Hills, Dewi Gould, ma-rmartinez, Falko Galperin, Denis Federico Lim, Aleksandr Bowkis

TL;DR. We built a scaffold to speed up our own research and gather data on automated alignment research (AAR). It turned out to not be valuable for researcher uplift, but was useful for gathering certain failure modes of AAR. Going forward, we plan to study the broader failure modes of AAR and how these automated research systems can be monitored and analyzed.

We’d like to thank Sid Baines, Andrew Draganov, Cameron Holmes and Daniel Tan for helpful comments.

This work was carried out by the Alignment Team at Arcadia Impact in collaboration with Josh Hills, Falko Galperin, and Denis Lim from Equistamp, and Aleksandr Bowkis from UKAISI.

There's a details box here with the title "The scaffold". The box contents are omitted from this narration.

What blocked researcher uplift?

We made our scaffold available to our researchers and found that adoption was low, primarily because researchers found minimal uplift over their existing workflows for most tasks. This was for three main reasons:

Our scaffold wasn’t helpful for conceptual work. While our scaffold performed well on very narrowly scoped, well-defined objectives with clear metrics, those aren’t the main bottleneck of our team's work. By the time a project has been reduced [...]

---

Outline:

(01:01) What blocked researcher uplift?

(03:07) The scaffold was useful for gathering failure modes

(05:08) Next Steps

---

First published:
October 2nd, 2026

Source:
https://www.lesswrong.com/posts/zGaQ3SS9C6So9NXFg/lessons-from-building-an-automated-research-scaffold

---

Narrated by TYPE III AUDIO.