When Property-Based Testing Pays Off, and When It Doesn't - Nikhil Barthwal

Unlocking the Secrets of Property-Based Testing

In this episode, I chat with Nikhil Barthwal about property-based testing. We go into how property-based testing can uncover the hidden bugs that often slip past human testers. With its capacity to automatically generate a multitude of test cases, this method helps us see beyond typical limitations. Nikhil also shares when property-based testing may not be ideal, like when it incurs high resource costs. He emphasizes that this approach serves as an assistant to testers rather than a replacement, enhancing productivity and reliability.

"When the system gets too complicated it is not possible for humans to write every test cases." - Nikhil Barthwal

Nikhil Barthwal is passionate about developing decentralized systems. He has several years of professional experience in large corporations as well as in smaller start-ups and acts as a mentor for various start-ups. He is also a guest speaker at various international conferences and gives talks on topics related to decentralized systems and software quality.

Highlights:

• Property-based testing generates thousands of test combinations automatically from a formal description of system behavior, reaching failure modes no human-authored test suite can anticipate.
• In distributed systems with many services, the number of possible failure combinations grows rapidly, making manual test case authorship impractical at scale.
• The shrinker component is what makes property-based testing usable: it reduces a long generated failure sequence to the precise step where the fault occurred, so engineers can act on it.
• Property-based testing becomes costly where each test execution consumes billable resources, such as cloud storage writes or live network calls, because automated test volume can spiral expenses out of control.
• Introducing property-based testing in a large organization is primarily a cultural challenge: winning one willing team first, letting them advocate to peers, is more effective than a top-down rollout.

📘 Tool migration ahead? Teamscale playbook: migrate only 20% of tests, keep nearly all bug detection. Download for free