There’s a framework about knowledge called the Rumsfeld matrix, and it gives us a pretty smart way to objectively evaluate our understanding.
The matrix details four distinct “types” of knowledge:
Known knowns. These are variables that we know exist, and that we have a good present understanding of. These could be things like the weather at this exact moment at our current location—we know the weather is a variable, and we know precisely what the temperature and cloud cover is right now.
Known unknowns. These are variables that we know exist, but we don’t have a good understanding of or a solid answer for yet. An example would be the number of ribs that will be required to strengthen a wing: we know that we’ll need ribs for the wing to be functional, but until someone does the work we don’t know how many.
Unknown knowns. This is a bit of a tricky one. It’s variables that we don’t consciously know exist, but we do have the answer for. These are pieces of information we might have seen and dismissed previously.
To my understanding after researching this framework, this could be things like institutional knowledge that only one or two people know: the team as a collective doesn’t realize it’s even a factor, but once that piece of knowledge is rooted out and shared it can actually be utilized or addressed.
Maybe the bolts specified for an assembly are actually too short, so the maintainers have switched to using longer bolts…but that problem has never been voiced, so the group as a whole is unaware of both the problem and its solution.
Unknown unknowns. You can probably guess that this is the most hazardous, uncertain quadrant. These are variables that we don’t have information for, but we also don’t even know they exist.
A tame example of this could be a component, like a servo or electronic speed controller, with an internal setting that causes a certain behavior. If you don’t know the setting exists, or what it’s set to, then this unk-unk will pop up when your component starts doing things you don’t expect.
So how do we apply this matrix to a project or test? Say it with me now—communication.
Use the matrix to take stock of the current state of affairs with your team. Note your known-knowns and your known-unknowns: the facts you already have up front, and the gaps and risks you know need to address.
As best you can, work with the team to flush out those unknown-knowns. They may not be helpful for an individual effort, but surfacing the secret handshakes and silent workarounds can help future ones.
And finally, the unknown-unknowns. You’ll never predict them all, but you can still prepare. Create contingency plans for how to address significant problems, either in a project or during a test. Implement monitors and feedback systems so you at least have good visibility into what’s happening. Foster a healthy risk management culture, where people aren’t afraid to call out even the smallest concerns.
It’s a bunch of up-front work, sure. But the improved clarity may just better set you up for success.