S2E5 | What senses do agents need to act? — The Shift Podcast by Microsoft Azure

For AI agents to move from reasoning to action, they need more than text alone. But what “senses” actually matter?

In this episode of The Shift Podcast: Agentic Edition, members of the Microsoft Foundry team discuss how multimodal inputs—such as text, vision, and speech—shape how agents perceive and interact with the world. The conversation explores what’s practical today, rather than assuming fully autonomous systems.

The discussion covers:

· Why multimodal AI expands what agents can understand.

· How vision, voice, and text models are combined in applications.

· The role of tools and APIs in enabling agent action.

· Where modality adds value—and where it introduces complexity.

Rather than framing modalities as future capabilities, the episode focuses on how teams are already working with them in real applications.

This is an honest look at how agents sense, interpret, and respond—based on today’s tooling and constraints.

👉 Read the AI apps and agents e-book: https://aka.ms/AIAppsAgents

👉 Join the Tech Community: https://techcommunity.microsoft.com/

Get to know the team:

· Ronak Chokshi, Director Product Marketing https://www.linkedin.com/in/ronakchokshi/

· Vinod Valloppillil, Partner Product Director https://www.linkedin.com/in/vinodvalloppillil/

· Linda Li, Product Manager II https://www.linkedin.com/in/zhuoqun-linda-li/

The Shift Podcast: Agentic Edition is a place for experts to share their insights and opinions. As students of the future of technology, Microsoft values inputs from a diverse set of voices. That said, the opinions and findings of our guests are their own and they may not necessarily reflect Microsoft's positions as a company. This episode of The Shift Podcast: Agentic Edition was recorded in February 2026. All information about products and offers is relevant to the time of recording. #MultimodalAI #GenerativeAI #GenerativeAgents

#TheShiftPodcast #MultimodalAI #AIAgents #AgenticSystems #MicrosoftFoundry