Start / LessWrong (30+ Karma) / Ai companion piece by zvi

LessWrong (30+ Karma)

“AI Companion Piece” by Zvi

29 min • 28 juli 2025
AI companions, other forms of personalized AI content and persuasion and related issues continue to be a hot topic. What do people use companions for? Are we headed for a goonpocalypse? Mostly no, companions are used mostly not used for romantic relationships or erotica, although perhaps that could change. How worried should we be about personalization maximized for persuasion or engagement?

Table of Contents

  1. Persuasion Should Be In Your Preparedness Framework.
  2. Personalization By Default Gets Used To Maximize Engagement.
  3. Companion.
  4. Goonpocalypse Now.
  5. Deepfaketown and Botpocalypse Soon.

Persuasion Should Be In Your Preparedness Framework

Kobi Hackenburg leads on the latest paper on AI persuasion. Kobi Hackenberg: RESULTS (pp = percentage points): 1⃣Scale increases persuasion, +1.6pp per OOM 2⃣Post-training more so, +3.5pp 3⃣Personalization less so, <1pp 4⃣Information density drives persuasion gains 5⃣Increasing persuasion decreased factual accuracy 6⃣Convo > static, +40% Zero is on [...]

---

Outline:

(00:35) Persuasion Should Be In Your Preparedness Framework

(04:33) Personalization By Default Gets Used To Maximize Engagement

(13:04) Companion

(16:50) Goonpocalypse Now

(24:09) Deepfaketown and Botpocalypse Soon

---

First published:
July 28th, 2025

Source:
https://www.lesswrong.com/posts/Rm3FD6646NDSCGxqM/ai-companion-piece

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Pie chart showing teen preferences between AI companions versus real-person conversations.
Pie chart:
Graph showing AI model persuasiveness increasing with computational scale over multiple studies.
Bar graph:
Bar graph:
This appears to be a screenshot from the AI assistant Grok showing a concerning and inappropriate response to a question about seeking attention, which includes suggestions of extreme violence. The content shown is highly problematic and dangerous. Note: Content like this should be reported through appropriate channels as it promotes harmful acts.
Prof. Chise tweets:
Download icon in white on a blue circular background.
Download icon in white on a blue circular background.
Download icon in white on a blue circular background.

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Senaste avsnitt

“Emotions Make Sense” by DaystarEld

3 augusti | 36 min

“Whence the Inkhaven Residency?” by Ben Pace

2 augusti | 5 min

“Many prediction markets would be better off as batched auctions” by William Howard

2 augusti | 9 min

“How many species has humanity driven extinct?” by Raemon

2 augusti | 1 min

“SB-1047 Documentary: The Post-Mortem” by Michaël Trazzi

2 augusti | 10 min
Vad är en podd?

En liten tjänst av I'm With Friends. Finns även på engelska.

Podcastbild

00:00 -00:00
00:00 -00:00