Flawed Understanding

When Science Is a Siren Song is an article appearing at The Washington Post. Quoting:

The problem with this fascinating study? It appears to be wrong. An analysis led by Stan Young of the National Institute for Statistical Sciences found that the original conclusion was based on poor statistics and is probably the result of chance.

So far, Young's rebuttal, published in January, has received little notice. That it is ignored by many of the media outlets that lavished attention on the original report isn't surprising; in fact, the most remarkable thing is how ordinary that lack of attention may be. A lot of science, it turns out, can't withstand serious scrutiny. Thoughtful analysis by John Ioannidis suggests that more than half of published scientific research findings can't be replicated by other researchers.

Part of the problem is that we've been conditioned to trust university research. It is based, after all, on the presumably lofty motives of its practitioners. What's not to like about science carried out by academics who have nobly dedicated their lives to understanding the unknown, furthering knowledge and serving humanity?

There are too many studies about which false conclusions are drawn. Sometimes this is attributable to research data. More often it is a function of misunderstanding on the part of writers or the general public. More from the liked article:

Does all this mean the system is broken? Surprisingly, no. Ultimately, science tends to be self-correcting, and flawed ideas are eventually recognized and disregarded. There really does seem to be a marketplace of ideas, and many good ideas eventually gain traction and persist, while many attractive but incorrect hypotheses eventually fall under the weight of compelling evidence. The system is far from perfect -- especially with regard to the exploitation of the most junior (and most vulnerable) researchers, who support much of this ecosystem -- but like capitalism, it may represent the best available option.

Self-correction is reliable in the long term. It is the short run which occassions misunderstandings. The problem with the short run is that many readers, who are not professionals, are likely to retain inaccurate information indefinitely.



HT: Clare