Comments on: Does ‘Peer Review’ Mean Anything to You?

By: Jeremy (PsyBlog author)

Jeremy (PsyBlog author) — Tue, 28 Aug 2007 18:37:00 +0000

Isabella, yes I think you're right.

Here's a radical thought though. Perhaps part of the problem is that the peer review system has no money in it. Academics peer review each other for nothing. If they were paid perhaps it would be tighter. Money attracts competition and this may push up standards.

Who should pay and how that would work is a whole other world of problems though...

By: isabella mori

isabella mori — Sun, 26 Aug 2007 06:01:00 +0000

peer reviews are ok - they suggest that it's likely that the researcher in question used some commonly accepted guidelines. and yes, they also function as gatekeepers.

but what's the alternative? nothing that i know of.

it would be nice if there was an independent, well-educated group non-experts who would look at research as well. however, most people (and unfortunately that includes some researchers) lack even the most rudimentary understanding of science - social science, "hard" science or otherwise.

By: Jeremy (PsyBlog author)

Jeremy (PsyBlog author) — Sat, 18 Aug 2007 16:02:00 +0000

Grue, so you'd suggest a way of showing the tier from which journal comes? It's an interesting idea although probably not something I'd do here as I'd guess that it doesn't mean much for most people. I was only originally thinking about whether or not it's worth pointing out that research is peer reviewed or not.

The feeling I'm getting from the comments so far is that the people who know what peer review means aren't that impressed with it - although it's better than nothing.

By: Grue

Grue — Sat, 18 Aug 2007 04:41:00 +0000

I was an academic but not in psychology. At least in my field, "peer review" didn't necessarily mean all that much in itself, it really depended on the journal. (In some journals, peer review is a very low bar.)

I'm not sure how to communicate this to the public (like me). Informally, the journals in my field could have been divided into tiers with rough agreement. If you report that the research was published in a tier 2 journal, that might eventually mean something to most people.

By: Anonymous

Anonymous — Sat, 18 Aug 2007 00:00:00 +0000

Ie hit the nail on the head. In my experience (not hard science, like physics) peer reviewers act like gatekeepers of the status quo and fresh ideas get short shrift.

WuffenCuckoo

By: emilythestrange

emilythestrange — Fri, 17 Aug 2007 14:09:00 +0000

I actually read this blog, because most of the research you mention are from peer-reviewed papers. Of course this system has its own weaknesses, but it is still more than no control at all.

I also don't thing psychiatry is a "social science".

By: Jeremy (PsyBlog author)

Jeremy (PsyBlog author) — Fri, 17 Aug 2007 08:50:00 +0000

Alone, I would agree there are problems with peer review, some of which you point out, but it's too harsh to say it's a bit of a joke. It's a human system so it's vulnerable to the usual human frailties: jealousy, envy, pride, greed and so on.

That's certainly a dramatic post by the Last Psychiatrist though...

By: letiantu

letiantu — Fri, 17 Aug 2007 02:55:00 +0000

After coming through so many peer review researchs ,sometimes we're in danger of forgetting the original reseach,which should have been worth more attention.A website will be useful to help distinguish both of them,over which we can get out from the reaserch directly what the themes and variations are.

It is true,we are keen to keep a skeptical attitude to the primal reaserch,in fact, this was way these form come to transfer to the peer review .Obviously,we shoult not take product in front of the process.

The truth hardly would reprove our indolence of respecting the original reserch,and consulting to all of the relative peer reviews.

By: alone

alone — Thu, 16 Aug 2007 20:53:00 +0000

Peer review is somewhat of a joke.

In the hard sciences-- physics, etc-- I can see how it matters. But in something like psychiatry-- or any social science-- you run into the very real problem of reviewer prejudice: all reviewers are, essentially, from a homogeneous group: academic psychiatrists.

Not just that, but the same people review, over and over again. And, you may be surprised to learn that many hournals let you "suggest" your reviewers.

If a study funded by Wyeth showed Effexor was the best, you wouldn't be surprised. So why would you be surprised that an academic who has based his career on a certin concept (e.g. antiepileptics for bipolar) wouldn't "review" accordingly?

So peer review, while raising the standards of the actual research that is published, also funnel research based on a prejudice.

You don't believe me. I know:

Farewell, Depression

Tell me I'm wrong.

By: Zachary Tong

Zachary Tong — Thu, 16 Aug 2007 15:58:00 +0000

The term "peer-reviewed" refers to the original research, not the blog post. The idea of the system is to flag posts directly reporting on quality, peer-reviewed research (ie any creditable journal of science) instead of news blurbs or press releases.

Essentially, it is a way to mark posts that have read the original report and are writing about it, instead of posts that have read a secondary source (like BBC, Discover, NewScientist, etc) and are receiving a watered down version of the original study.