Observe data collection at the moment of measurement See, observe, learn how data are collected at moment and place of measurement. "You never learn more about a process than when you directly observe how data are actually measured," said Cuthbert Daniel, a superb applied statistician. See with fresh eyes. Walk around what you want to learn about. Talk to those who do measurements. See how numbers came to be. Do those measuring know the desired answer? Are those measuring skilled, alert, honest, biased, incompetent, sloppy, tired and emotional?...Artifacts and errors in measurements measured? How are outliers adjudicated? Edward Tufte, Seeing With Fresh Eyes uxmeasurement
Heuristics That Almost Always Work An Article by Scott Alexander astralcodexten.substack.com Sometimes there’s a Heuristic That Almost Always Works, like “this technology won’t change everything” or “there won’t be a hurricane tomorrow”. And sometimes the rare exceptions are so important to spot that we charge experts with the task. But the heuristics are so hard to beat that the experts themselves might be tempted to secretly rely on them, while publicly pretending to use more subtle forms of expertise. …Maybe this is because the experts are stupid and lazy. Or maybe it’s social pressure: failure because you didn’t follow a well-known heuristic that even a rock can get right is more humiliating than failure because you didn’t predict a subtle phenomenon that nobody else predicted either. Or maybe it’s because false positives are more common (albeit less important) than false negatives, and so over any “reasonable” timescale the people who never give false positives look more accurate and get selected for. expertiseheuristicsprediction