AISN #6: Examples of AI safety progress, Yoshua Bengio proposes a ban on AI agents, and lessons from nuclear arms control .

Examples of AI safety progress

Training AIs to behave safely and beneficially is difficult. They might learn to game their reward function, deceive human oversight, or seek power. Some argue that researchers have not made much progress in addressing these problems, but here we offer a few examples of progress on AI safety.

Detecting lies in AI outputs. Language models often output false text, but a recent paper suggests they understand the truth in ways not reflected in their output. By analyzing a model’s internals, we can calculate the likelihood that a model believes a statement is true. The finding has been replicated in models that answer [...]

---

Outline:

(00:13) Examples of AI safety progress

(03:56) Yoshua Bengio proposes a ban on AI agents

(07:19) Lessons from Nuclear Arms Control for Verifying AI Treaties

(10:02) Links

---

First published:
May 16th, 2023

Source:
https://newsletter.safe.ai/p/ai-safety-newsletter-6

---

Want more? Check out our ML Safety Newsletter for technical safety research.

Narrated by TYPE III AUDIO.