Testing complex flag conditions and ensuring complete case coverage

Recently I stumbled upon an interesting test case where I needed to cover many possibilities and make sure everything continues to work as intended in the future. I had 7 boolean flags that needed to be used to toggle an app functionality.

It's a bit tricky to cover this with tests and to make sure that the function triggers correctly for all the proper cases, ensuring it won't break with future changes. To make things even more serious this logic makes decisions for a critical user path during the app onboarding and could potentially break the app for new users.

Therefore tests should cover all the possibilities and lock the correct behaviour to make it future-proof. There are 2^7 = 128 flag combinations. Writing this by hand would be extremely time-consuming and error-prone. I decided to generate all the possible flag combinations in a 2D array matrix, iterate over rows and use row values to initialize flags. The test simply needs to go over all 128 rows and test for correct output based on all the input combinations.

First I needed to create a function that generates a 2D matrix with all the possible boolean combinations:

  function generateFlagCombinations() {
    const flags = [];
    const numFlags = 7;

    // Generate all possible combinations of boolean values for the flags
    // [[false, false, false, false, false, false, false], [true, false, false, false, false, false, false], ...
    for (let i = 0; i < Math.pow(2, numFlags); i++) {
      const flagCombination = [];
      for (let j = 0; j < numFlags; j++) {
        flagCombination.push(Boolean(i & (1 << j)));
      }
      flags.push(flagCombination);
    }

    return flags;
  }

For the actual test to simplify naming and focus on what's important, I'll just call the boolean flags A – G:

it('should activate in correct cases', () => {
  const flagCombinations = generateFlagCombinations();

  for (const [A,B,C,D,E,F,G] of flagCombinations) {
    const result = shouldShowProceedButton({A,B,D,E,C,F,G});

    if (E) {
      expect(result).toBe(A && !F);
    } else if (!G) {
      expect(result).toBe(false);
    } else {
      expect(result).toBe(A && (!C || (B && D && C)) && !F);
    }
  }
});

And that would be it. We've created:
- a small function that generates all the possible flag combinations — 128 rows representing 128 possible flag permutations
- a simple test that uses the generated combinations. It iterates over all the rows of a 2D matrix and uses the values to initialize the flags to test for every single possible combination

This way we can be sure that we've covered all the possibilities and that we know what to expect from a given function for every combination. This locks the function's behaviour in a way and makes it harder to accidentally change the behaviour.

By the way, I know that using so many flags to make a decision might not be the greatest strategy but sometimes you don't have other choice than to use what's already there 😶‍🌫️

Testing complex flag conditions and ensuring complete case coverage

Comments

More from this blog

Moved to a new domain

My first thoughts on Next.js 13 and the new app router

JS Date constructor timezone oddity

Minimal example of file upload using Next.js app router

Command Palette

Comments

More from this blog