Yeah, I believe it. A bunch of economists feel the same way about half the theory papers that are published.

Peer review works well when people do it. And replicating experiments and so forth.

Yeah whatever. His claim wasn't even peer reviewed.

Science is hard...

And the pressure to publish results in unrealistic studies being put forth as new discoveries. People let quality suffer because their employer only has quantitive measures of success.

We do the same in economics, and social policy, preferencing the qualities we can count over those we can't count. Leaving number of patients cared for, or number of dollars made more valued than quality of care or how the dollars were made.

The question is, is the peer review getting worse (letting bad papers through) or better (finding more bad papers that are already there)?

I would guess that if you take the papers based on medical trials out, you'll find that a vast majority of the rest of the published research is at least honest, if not correct. The biggest problem isn't "untrue" findings - its the fact that drug trials that don't come out as the Pharma companies wish are hidden. If the results of every drug trial were published, we'd have a lot more information on the actually efficacy and affects of new drugs.

Research is often "guided" by the results required. That's not true science, and it's a widespread enough problem that everyone needs to be wary of published results in general. We are now starting to see money determine what teachers teach in other fields as well. It started in K-12 and is now creeping into higher education. Literature is out; argument is in. In some colleges and universities, the English Department isn't even allowed to teach freshman composition. That goes to a "Rhetoric and Writing" Department instead (or something similar).

I've heard suggestions (on NPR, for example) that peer review is getting worse, that there is a flood of new journals and unqualified people are being asked to be reviewers, people that may expert in one field, but not the relevant one. I've experienced the use of quantitative metrics in evaluating performance. When I was a programmer with IBM our performance review included a count lines of code produced. I wrote elegant code and was dinged for that.

"When I was a programmer with IBM our performance review included a count lines of code produced."

As a performance engineer, I can confirm this is the best way to measure quality code.

@abge - I so truly hope I missed the sarcasm in that remark.