You Need a Test Automation Strategy Before AI - Michaël Pilaeten

When AI removes your asserts and the pipeline stays green

What happens when AI ships poor quality faster, and nobody notices because the pipeline stays green? With Michaël Pilaeten I talk about why test automation without a clear strategy is a problem that AI only accelerates, not solves. We get into the question of what separates a tester from a test automation engineer, why those two roles need each other but are not interchangeable, and how chasing 100% coverage on millions of lines of legacy code can be a waste everyone agrees to quietly.

"An automated test rarely finds defects." - Michaël Pilaeten

Michaël Pilaeten is Head of Quality Engineering at SOFICO. He is an accredited trainer for various ISTQB and IREB courses, as well as an author, international keynote speaker and workshop facilitator.

Breaking the system, helping to rebuild it and giving advice on how to avoid problems in the first place: that sums up his work. With more than 20 years of experience in software consultancy across a wide range of environments, he has seen the best (and the worst) of software development. Today he guides consultants, partners and customers on their personal and professional path towards excellence.

Highlights:

• Testers and test automation engineers are two distinct roles: testers explore and break products critically, while automation engineers translate requirements into automated cases, and conflating them weakens both.
• AI-generated tests default to quantity over correctness, producing plausible-looking test cases with invented scenarios just to hit coverage targets, not because the scenarios are valid.
• Automated tests rarely find defects; when a pipeline fails, teams blame the test rather than the code, which means real bugs hide behind green dashboards.
• Replacing junior testers and developers with AI removes the learning path that produces senior engineers, creating a skills gap with no pipeline to fill it.
• Convincing management to invest in test quality requires framing the argument around production incidents and customer risk, not coverage metrics they do not understand.

📘 Tool migration ahead? Teamscale playbook: migrate only 20% of tests, keep nearly all bug detection. Download for free