Building Out GPU Clouds // Mohan Atreya // #317

Demetrios and Mohan Atreya break down the GPU madness behind AI — from supply headaches and sky-high prices to the rise of nimble GPU clouds trying to outsmart the giants. They cover power-hungry hardware, failed experiments, and how new cloud models are shaking things up with smarter provisioning, tokenized access, and a whole lotta hustle. It's a wild ride through the guts of AI infrastructure — fun, fast, and full of sparks!

Big thanks to the folks at Rafay for backing this episode — appreciate the support in making these conversations happen!

// Bio

Mohan is a seasoned and innovative product leader currently serving as the Chief Product Officer at Rafay Systems. He has led multi-site teams and driven product strategy at companies like Okta, Neustar, and McAfee.

// Related Links

Websites: https://rafay.co/

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Mohan on LinkedIn: /mohanatreya

Timestamps:

[00:00] AI/ML Customer Challenges

[04:21] Dependency on Microsoft for Revenue

[09:08] Challenges of Hypothesis in AI/ML

[12:17] Neo Cloud Onboarding Challenges

[15:02] Elastic GPU Cloud Automation

[19:11] Dynamic GPU Inventory Management

[20:25] Terraform Lacks Inventory Awareness

[26:42] Onboarding and End-User Experience Strategies

[29:30] Optimizing Storage for Data Efficiency

[33:38] Pizza Analogy: User Preferences

[35:18] Token-Based GPU Cloud Monetization

[39:01] Empowering Citizen Scientists with AI

[42:31] Innovative CFO Chatbot Solutions

[47:09] Cloud Services Need Spectrum