EP39 From False Positives to Karl Popper: Rationalizing Cloud Threat Detection

Guest:

• Jared Atkinson, Adversary Detection Technical Director at SpecterOps

Topics:

• What are bad/good/great detections? Is this all about the Bianco's pyramid? Is high good and low bad?
• How should we judge the quality of detections? Can there be a quality framework? Is that judgment going to be site specific?
• What should we do to build more good directions? Is this all about reducing false positives?
• Can we really measure false negatives? How can we approach this?
• How can we test for detection goodness in the real world? What are the methods that work? It can't be just about paper ATT&CK coverage, right?
• What are your top 3 tips for improving the detection practice at an organization?

Resources:

• "The Pyramid of Pain" post by David Bianco
• "On Threat Detection Uncertainty"
• "Detection Coverage and Detection-in-Depth"
• "Detection in Depth" by SpecterOps
• "Philosophy of Science: Rationality Without Foundations" by Karl Popper (yes, really)
• Red Canary "2021 Threat Detection Report"
• "The Black Swan: The Impact of the Highly Improbable" by Nassim Nicholas Taleb
• John Piaget's theory of cognitive development