Avoiding Test-Case Permutation Blowout
www.stevenhicks.me
Sometimes you want to write tests for a business rule that's based on multiple variables. Covering every possible permutation of the variables quickly becomes unsustainable. I've found myself using a different approach — one test for the positive case, and one test for each variable's negative case.
  • RonSijm
    2·
    3 months ago

    In C# I’m generally using Verify for these happyflow tests - So instead of explitly testing every individual property, you just do Verify(state); and compare the entire state against a json saved state.

    A little bit for the same reason of “testing fatigue” - having to manually rewrite assertions of a lot of tests is getting annoying. With that approach you just do a merge compare between results, accept them, and you’re done

    • lysdexicOPEnglish
      3·
      3 months ago

      In C# I’m generally using Verify for these happyflow tests (…)

      I don’t think this is related to this topic. The problem domain cover the exact opposite of happy flow tests: it’s about maximizing edge case coverage by minimizing the amount of tests required. This has nothing to do with what invariants you’re tracking, but how many tests you are using to cover the paths you’re covering and how to tell which tests you can dump while keeping the same coverage.

  • nomad@infosec.pub
    1·
    3 months ago

    Well… This sounds like optimizing tests for coverage. Which is not a novel idea, but certainly not the worst.

    • lysdexicOPEnglish
      4·
      3 months ago

      It really depends on what problem you experience the most. Automated tests do require an awful lot of maintenance, and minimizing the number of tests you need to track all the invariants you care about is something that helps everyone in the long run.

  • tyler
    1·
    3 months ago

    This is a decent idea for certain cases, but you’ll almost always have a better time if you use property based testing, in which case you can handle all negative cases with only a few such tests.