A/B testing is an effective approach to use science to design and deliver deeply-frustrating user experiences.
A/B testing without upfront research is just random monkeys testing random designs to see which of those designs do “best” against random criteria.
If drug testing was actually implemented like most A/B tests, you’d give 2 drugs to 2 groups of people and pick the “winner” by whichever group had fewer deaths.
Here I describe an approach for defining new information architectures for large organizational websites managed by many stakeholder groups.
Broadly speaking, there are four general phases to the approach:
Auditing. Begin by immersing yourself in existing content and encourage stakeholders to adopt a critical, audience-minded perspective of their content.
Diagramming. Work with stakeholders to develop new conceptual categories that better serve audiences and organizational direction.
Elaborating. Think through content in detail and test new categories against specific instances and edge cases.
Producing. Prepare content teams for production using a shared database of new sitemap pages and editorial considerations that you’ve developed incrementally.
At least half of the work of design is not design, because design isn’t just "making things"—it’s making things with other people, many of whom usually aren’t designers. This is true any time you’re working with others from a domain outside of your own. Communicating ideas, marshaling stakeholder consensus, soliciting and incorporating feedback, and redefining problems that weren’t fully known at the start are all the non-design work of design, what we might generally call "facilitation."