How a humiliating military loss proves that so much of our so-called "expertise" is fake, and the case against specialization and intellectual diversity.
The lesson of Tetlock (and the Afghanistan War), is that while you certainly shouldn’t be getting all your information from your uncle’s Facebook Wall, there is no reason to start with a strong prior that people with medical degrees know more than any intelligent person who honestly looks at the available data.
You just handed off a major redesign. Three months of research, twenty-seven major revisions, and hundreds cups of coffee have all culminated in this pinnacle of glory. It’s finally done!
Except it’s not.
It’s not, even after you have answered every single question the developers have about your red-line.
It’s not, even after you have addressed all the technical constraints developers encountered during the implementation.
It’s not, even after you meticulously documented all the patterns and styles into a library for reference and reuse.
It’s not, because neither you nor the developers have talked to a real user. At the bottom of your heart, you are secretly wishing:
My design looks great on paper, so let’s keep it on paper.
Post-occupancy evaluation (POE) is a practice in the building industry where an architect would visit the building after its occupancy and interview its residents. It sounds like a great opportunity for collecting feedback and learning from mistakes, but it’s rarely practiced. Why?
Many awe-inspiring, prize-winning architectures are half building, half sculpture. Often made of specially molded concrete and steel, they are extremely expensive to alter, let alone any alteration would also attack the architect’s prestige and pride. So whatever usability issues the POE identifies will remain as issues, unless the architect wants to accept the public criticism and shame that comes with the remodeling.
In fear of criticism, an architect would turn down the opportunity for POE, and continue to design the same roof that would leak water in future projects.
In fear of criticism, a developer would use customer service representatives as a shield against user complaints, while focusing on the “technical” aspect of things.
In fear of criticism, a designer would close the contract as soon as the client accepts the design, even though none of the real users are represented by the client.