
GPT-2 as Step Toward General Intelligence
Om avsnittet
A machine learning researcher writes me in response to yesterday's post, saying:
I still think GPT-2 is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.
I resisted the urge to answer "Yeah, well, your mom is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked."
But I think it would have been true.
A very careless plagiarist takes someone else's work and copies it verbatim: "The mitochondria is the powerhouse of the cell". A more careful plagiarist takes the work and changes a few words around: "The mitochondria is the energy dynamo of the cell". A plagiarist who is more careful still changes the entire sentence structure: "In cells, mitochondria are the energy dynamos". The most careful plagiarists change everything except the underlying concept, which they grasp at so deep a level that they can put it in whatever words they want – at which point it is no longer called plagiarism.
GPT-2 writes fantasy battle scenes by reading a million human-written fantasy battle scenes, distilling them down to the concept of a fantasy battle scene, and then building it back up from there. I think this is how your mom (and everyone else) does it too. GPT-2 is worse at this, because it's not as powerful as your mom's brain. But I don't think it's doing a different thing. We're all blending experience into a slurry; the difference is how finely we blend it.
Fler avsnitt
Visa alla avsnitt av Astral Codex Ten PodcastAstral Codex Ten Podcast med Jeremiah finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.