Remember this the next time someone tells you about study results proving X, Y, or Z

The next time someone spouts off about this or that study proving this or that 'fact', and that we should all start hopping on one foot because 'they said so', you really should consider the sorry state of medical and other science these days. It turns out that most of what 'they' tell you ain't really so, or at least that's how it's shaping up:

Jonah Lehrer, writing on December 13, 2010 on The New Yorker web site, tells the sad tale of increasingly irreproducible results

The Truth Wears Off
Is there something wrong with the scientific method?
by Jonah Lehrer December 13, 2010

~snip~

But now all sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. It’s as if our facts were losing their truth: claims that have been enshrined in textbooks are suddenly unprovable.

(Emphasis mine.)
Source: http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer
(H/T: WUWT)

And, for a double-whammy that will bend your mind, read this little ditty by David H. Freedman in The Atlantic Magazine:

Lies, Damned Lies, and Medical Science

Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science.

~snip~

This array suggested a bigger, underlying dysfunction, and Ioannidis thought he knew what it was. “The studies were biased,” he says. “Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there.” Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them. (Emphasis mine).

And here's the money quote:

But these studies often sharply conflict with one another. Studies have gone back and forth on the cancer-preventing powers of vitamins A, D, and E; on the heart-health benefits of eating fat and carbs; and even on the question of whether being overweight is more likely to extend or shorten your life. How should we choose among these dueling, high-profile nutritional findings? Ioannidis suggests a simple approach: ignore them all. (Emphasis mine.)
Source: http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and...

Edit: Adding this essay:

Why Most Published Research Findings Are False, an essay by John P. A. Ioannidis

Summary

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.