Why Most Published Research Findings Are False A Research Paper by John P.A. Ioannidis journals.plos.org There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. researchsciencetruth
Monkeys testing random designs A Tweet by Jared Spool twitter.com A/B testing is an effective approach to use science to design and deliver deeply-frustrating user experiences. A/B testing without upfront research is just random monkeys testing random designs to see which of those designs do “best” against random criteria. If drug testing was actually implemented like most A/B tests, you’d give 2 drugs to 2 groups of people and pick the “winner” by whichever group had fewer deaths. uxresearch