Sveriges mest populära poddar
Interconnects
VetenskapTeknologi

(Voiceover) Building on evaluation quicksand

17 min•16 oktober 2024

Read the full post here: https://www.interconnects.ai/p/building-on-evaluation-quicksand

Chapters

00:00 Building on evaluation quicksand

01:26 The causes of closed evaluation silos

06:35 The challenge facing open evaluation tools

10:47 Frontiers in evaluation

11:32 New types of synthetic data contamination

13:57 Building harder evaluations

Figures

Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/manual/openai-predictions.webp



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.interconnects.ai/subscribe

Fler avsnitt av Interconnects

Visa alla avsnitt av Interconnects
I wrote an AI textbook — how long until AI can do it better?
•13 min
Open models recap: more on Kimi K3, Qwen 3.8, Xi's WAIC speech, distillation, the open-closed gap, and what's next
•49 min
Kimi K3: The open-weights escalation
•20 min
GLM-5.2 is the step change for open agents
•9 min
Banning Open Source AI Would Be A Mistake
•7 min
State of the blog, mid-2026
•6 min
Frontier post-training recipe review with Finbarr Timbers
•57 min
Claude Fable 5 and new AI safety fables
•12 min
Farewell Ai2
•16 min
Open and closed models are on different exponentials
•7 min

Interconnects med Nathan Lambert finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.