Ep. 9 | Unintended Consequences: Taking Responsibility for What We Build | Tony Fadell

At General Magic, we naively believed that if we built elegant tools and put them in people’s hands, the rest would take care of itself. Tony Fadell was there at the beginning, a young engineer with an uncommon gift for making technology disappear into the life of the person holding it, and he went on to help build the iPod, the iPhone, and Nest. So when he says that every barrier to our vision has since been knocked down—“the pipes are open and they’re flooding right to your eyeballs and your fingertips”—he is describing a demolition he took part in. He does not flinch from that. He calls what followed a frictionless flow of everything, for better and for worse, and he declines to pretend the worse was an accident.

What distinguishes this conversation is where Tony locates the failure: not in a few bad actors, but in structure. Cars became safe when government forced the issue and people had stopped dying; a hands-off posture toward AI and social media is an abdication, and the missing ingredient is political will, in a system where money decides whose voice is heard. He sees the next trap plainly. Consumers will not pay $30 a month for an AI assistant, which means the business model will drift, as it did before, toward ads and data, and we will once again be the product. He also says what few of his peers will say aloud: that the centi-billionaires who now set the tone for an entire industry have forgotten the society and its ethical norms which made their fortunes possible, and that Fortune 500 CEOs, once answerable to their employees and to the outside world, now seem answerable to no one.

The phrase that stuck with us is cognitive surrender. Tony describes people who supervise a hundred software agents but end up like quality-control workers on an Amazon conveyor, moving bins from one machine to the next, at the speed of the system rather than the speed of humans, until the craft is gone and burnout is all that remains. There is grief in that image—the ache of watching work that once carried a person’s whole attention reduced to throughput. His solution is modest and historically grounded: Microsoft once sent salespeople to sit across from customers who disliked the company, because it is very hard to hate someone face to face. Build technology that gathers people in person and amplifies what they say. Put the phone down at dinner. Back only companies that make the world a better place. Tony admits he feels alone in this, a humanistic technologist in a Valley that has moved on.

The unedited episode of our interview with Tony is on YouTube.

This post is public so feel free to share it.

In this episode

* 01:10 – The remorse question: as one of the architects of today’s technology immersion, what does he want to prevent this time?

* 01:43 – Unintended consequences: from splitting the atom to the smartphone platform, why creators must “be the voice of reason,” and why app stores should do age gating

* 04:59 – Privacy as a selling point, giving families real tools, and why trusted identity matters in the age of AI

* 06:44 – Safe cars took government. What about AI? Rule-making in China and why “hands off” isn’t the answer

* 08:13 – What Build Collective invests in: clean farms, fusion energy, AI plus recycling, AI plus textiles, and what it won’t touch

* 10:00 – Is there a straight line from General Magic to today? Why users were never “the product,” and how kids and the climate crisis deepened his view

* 12:03 – The iPod, the Walkman and the personal sound bubble

* 14:03 – Analog vs. digital convenience vs. digital connectivity, and how the iPhone became a “notifications engine”

* 16:49 – In Build: from Nest CEO to Google, and the culture clash

* 19:39 – Near-death experiences from General Magic through Philips to Apple and Nest, and why he calls himself “Tony 4.0”

* 20:29 – Founders around the world: what changes from India to Thailand, and why local talent and right-sized technology matter

* 22:46 – Moving his family to Indonesia: a “long vacation from tech” that wasn’t

* 23:44 – The App Store’s genesis: General Magic apps, Steve Jarrett, Philips’s Velo and Nino, and electronic distribution over a phone line

* 27:07 – Will consumers pay for AI assistants? Why he worries the answer is ads and “you’re gonna be the product again”

* 29:25 – Where AI works: domain-specific tools in drug discovery at Orionis Biosciences and why experienced humans stay in the loop

* 32:24 – His biggest AI worry: “cognitive surrender”

* 34:54 – AI and education: the New York Times story about a Stanford grad, and what he hears teaching at MIT

* 36:08 – Does he feel alone as a humanistic technologist? The “centi-billionaires” and who today’s heroes are

* 39:34 – Reasons for hope: up-and-comers in London, Paris and San Francisco, including the Stripe Collison brothers

* 40:22 – Why he’d make a bad politician, and a General Magic story about a four-day infrared data-protocol committee

* 41:21 – What government could do about social media: age gating across states, federal standards, and why voters over 65 “control everything”

* 44:26 – Why having kids is an inflection point, and how younger voters can find their voice sooner

* 45:43 – Can technology make us kinder? Physical interaction, the Microsoft salespeople story, and Deb Roy’s small-group project at MIT

* 48:07 – Tony’s digital habits: phones away from the table, and the case for “nutritional facts” for digital consumption

About Tony Fadell

Tony Fadell joined General Magic in December 1991 as a hardware/software systems architect straight out of the University of Michigan (B.S. Computer Engineering, 1991), where he built the Magic Bus peripheral-bus technology. After taking his product concepts to Philips and a startup that ran out of funding, he went to Apple in 2001 to lead iPod development, became SVP of the iPod division, and led iPhone hardware, firmware and accessories through its first three generations. In 2010 he co-founded Nest Labs with Matt Rogers, creating the Nest Learning Thermostat and Nest Protect; Google acquired the company for $3.2 billion in 2014. Today he is a principal at Build Collective, which invests in and advises deep-tech startups — in this conversation, companies working on clean farming, fusion energy, and AI applied to recycling, textiles and healthcare. He is the author of Build: An Unorthodox Guide to Making Things Worth Making (2022), a frequent speaker on AI, hardware and product culture, and a Designer in Residence at MIT.

Resources & Links

* Tony on LinkedIn: https://www.linkedin.com/in/tonyfadell/

* Tony's book: https://www.buildc.com/the-book

* Build Collective: https://www.buildc.com

* Wikipedia biography: https://en.wikipedia.org/wiki/Tony_Fadell

* Orionis Biosciences: https://www.orionisbio.com

* An Inconvenient Truth (2006 film, Al Gore): https://en.wikipedia.org/wiki/An_Inconvenient_Truth

* Tim Ferriss interview (2019): https://tim.blog/2019/12/23/tony-fadell/

* Wisdom 2.0 with Anderson Cooper (2017), "Take Control of Your Digital Life":

* CogX 2024 Leadership Summit keynote on AI:

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit spellboundproductions.substack.com