Azure Log Analytics Costs Are Out of Control - Here's How We Cut Them by 60%

Azure Log Analytics costs often take up 20% or more of a cloud bill, even though most teams only check logs when something breaks.

Azure's default analytics logs are powerful, but they're also expensive and often unnecessary for day-to-day log inspection. Switching application logs to Basic Logs can reduce Log Analytics costs by up to 60%.

In this episode, Niels walks us through a real customer case where logging costs dropped by thousands per year. They explain the difference between Analytics, Basic, and Auxiliary logs, show when Basic Logs are sufficient, and discuss practical setups using Azure Container Insights and FluentBit. This includes building a custom FluentBit plugin in Go as well as real-world gotchas like missing short-lived pods and why dynamic credentials matter.

Creators & Guests

• Jonny Daenen - Host
• Niels Claeys - Guest

Resources:

• Custom FluentBit plugin: https://github.com/nclaeys/fluent-bit-go-azure
• Click here to watch a video of this episode.
  
• Full playlist: https://www.youtube.com/playlist?list=PLJ_da7qdfL80rA7byzC_CmyrfJWjcCTnb

Chapters:

• (00:00) - Intro: why optimize Azure log costs?
• (03:10) - What kind of logs are we dealing with?
• (06:13) - Plan types & the cost difference
• (09:59) - FluentBit vs Azure Container Insights
• (13:33) - How FluentBit works in K8S
• (16:41) - Can you lose log data?
• (17:36) - A custom plugin for Azure Workload Identity
• (21:05) - Why not use Azure Container Insights?
• (22:35) - Do all clients benefit?
• (23:41) - Summary & takeaways

Data & AI: Technology Explorations is a biweekly show from Dataminded. Each episode a Dataminded engineer demos a tool or technique worth knowing about -- working code, honest takes, no hype.

Music by Aleksandr Karabanov from Pixabay