Semantic Coupling & the Interconnected AI Stack with Eugene Wu, Professor at Columbia University

Why AI agents fail at scale, and the systems-thinking fix nobody's building yet.

In this episode of AI Radicals, host Satyen Sangani talks with Eugene Wu, Columbia professor and co-founder of the Data Agents and Processes (DAPLab), about why building reliable AI agents is fundamentally a systems problem, not a model problem.

Eugene explains "semantic coupling," the idea that every layer of an agent's stack, from data retrieval to tool calls to reasoning, is interdependent, so a small failure anywhere can quietly corrupt the final output. He and Satyen draw a parallel to early relational databases absorbing the complexity that applications used to handle themselves, arguing that today's computing infrastructure needs to do the same for agents: managing data flows, enforcing rules deterministically instead of hoping a prompt is followed, and giving agents safe room to explore and fail without real-world consequences. Eugene shares research from his lab on search at massive scale, why even top models struggle to find the right evidence in huge datasets, and a "branchable" computing environment that lets agents try, fail, and roll back cheaply.

"You need somewhere to ground reliability and quality. If it's not probabilistic, and you can rely on it and it's guaranteed, then the system doesn't need to think about it at all."

Listen to this episode to learn:

• Why "semantic coupling" makes traditional layered software thinking break down for AI agents
• Why pushing rules into the system (not the prompt) turns probabilistic behavior into guaranteed safety
• Why compute-level innovations like branchable environments could redefine what we expect agents to do on our behalf

--------

“ So many people in the data community and companies are working on data search, but how do you evaluate the end-to-end quality? And how much search is even the critical bottleneck in this kind of end-to-end question? Because what does the agent need to do? It needs to take your question, it needs to figure out how to decompose it into a series of sub-questions. Such as, “Schools near Clinton Hill.” Then it needs to figure out, “Okay, I need to find information about Clinton Hill, and I need to find locations of schools.” And it needs to then search over 40 million documents and datasets that we've collected, and find the right ones. At any given step, if it didn't find the right data, then the whole thing falls apart and you can't answer the question.” – Eugene Wu

--------

Time Stamps

*(01:10): What the Data Agents and Processes Lab is and why it spans multiple research areas

*(04:29): Semantic coupling explained

*(19:31): How agents find the right data in a massive data lake

*(29:18): What trust means when AI can persuade as well as answer

*(42:59): The biggest unlocks for agents over the next 12 months 

*(56:12): What we'll be talking about in 12 to 18 months

--------

Sponsor

This podcast is presented by Alation.

Learn more:

* Subscribe to the newsletter: https://caspianstudios71626.activehosted.com/f/73

* Alation’s LinkedIn Profile: https://www.linkedin.com/company/alation/

* Satyen’s LinkedIn Profile: https://www.linkedin.com/in/ssangani/

--------

Links

Connect with Eugene Wu on LinkedIn: https://www.linkedin.com/in/eugene-wu-b23417290/

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.