Once you see that an answer is not serving its question properly anymore, it should be tossed away. It's just their natural life cycle.
They usually kick and scream, raising one hell of a ruckus when we ask them to leave. Especially when they have been with us for a long time.
You see, too many actions have been based on those answers. Too much work and energy invested on them. They feel so important, so full of themselves. They will answer to no one. Not even to their initial question!
The hardest thing about customer interviews is knowing where to dig. An effective interview is more like a friendly interrogation. We don’t want to learn what customers think about the product, or what they like or dislike — we want to know what happened and how they chose... To get those answers we can’t just ask surface questions, we have to keep digging back behind the answers to find out what really happened.
I observed something fairly early on at Apple, which I didn’t know how to explain then, but I’ve thought a lot about it since. Most things in life have a dynamic range in which [the ratio of] “average” to “best” is at most 2:1.
For example, if you go to New York City and get an average taxi cab driver, versus the best taxi cab driver, you’ll probably get to your destination with the best taxi driver 30% faster. And an automobile; what’s the difference between the average car and the best? Maybe 20%? The best CD player versus the average CD player? Maybe 20%? So 2:1 is a big dynamic range for most things in life.
Now, in software, and it used to be the case in hardware, the difference between the average software developer and the best is 50:1; maybe even 100:1. Very few things in life are like this, but what I was lucky enough to spend my life doing, which is software, is like this.
So I’ve built a lot of my success on finding these truly gifted people, and not settling for “B” and “C” players, but really going for the “A” players. And I found something… I found that when you get enough “A” players together, when you go through the incredible work to find these “A” players, they really like working with each other. Because most have never had the chance to do that before. And they don’t work with “B” and “C” players, so it’s self-policing. They only want to hire “A” players. So you build these pockets of “A” players and it just propagates.