Tracing a Large Language Model's Thoughts

Anthropic's research explores the inner workings of their large language model, Claude, using novel interpretability techniques likened to an "AI microscope". This work aims to understand how Claude processes information, forms thoughts, and makes decisions by examining its internal computations. Their findings reveal surprising insights into Claude's multilingual abilities, planning for future words in poetry, strategies for mental math, and the distinction between faithful and fabricated reasoning. Furthermore, the research investigates how Claude handles multi-step questions, why it sometimes hallucinates, and the internal tensions exploited by jailbreaking prompts. Ultimately, this "AI biology" seeks to increase the transparency and reliability of advanced AI systems.