Does ‘Peer Review’ Mean Anything to You?
A new website, bpr3.org, has been set up to highlight peer reviewed research - the process by which academics check each other's work.
A discussion about 'peer review' is ongoing over at Cognitive Daily suggesting science bloggers should use an icon to indicate when they are discussing peer reviewed research. Peer review is simply the system academics use for checking each other's work. Before research is published in peer reviewed journals it gets sent to other experts in the field to be checked. Peer review is seen as the academic gold standard.
Whether you realise or not, almost all the research I cover here on PsyBlog is peer reviewed. That's simply because most academic journals I use are peer reviewed. I rely on the reference section at the end of each post to highlight the fact that I'm reporting reputable scientific research. For example, you can see a study has been published in Psychological Science so you know it has been properly checked - or at least it should have been(!).
Dave Munger at Cognitive Daily (and others) argue that it would be useful for readers to see when bloggers are discussing peer-reviewed research. To this end they have set up a website aimed at producing a logo for science bloggers to display. More than that, they aim to aggregate peer-reviewed research so that it can be accessed on one site.
Dave argues this site and logo will help to distinguish peer-reviewed research from 'scholarship by press release' - the process by which many journalists (and bloggers) simply regurgitate press releases without looking at the original research.
Cabiblog, meanwhile, points out that some fantastic, rigorous scientific research is published outside peer reviewed journals.
Post your views
I'd be very interested in your views on this. I know many of you are academics and students and so will be familiar with, or at least aware of, the peer review process. But what about those who aren't familiar with this - does a reference section convince you the research is kosher? Does the phrase 'peer review' mean anything to you?
Join 22301 readers
Sounds like an excellent idea. I wonder how many papers get knocked back or is it more of a rubber stamp job ?
Hi Anon, it's often far from a rubber-stamping. Some journals reject 75% or more of the papers submitted to them.
I don't have a scientific background, but if a peer review only entails looking at the paper(as opposed to recreating the experiment) then I don't know if it really adds much credibility. I imagine when an academic reads the actual paper they would be skeptical of the results (after all isn't that what science is all about?) and would point out any discrepancies. I guess there would be an advantage in adding the reviewers name, because after all if it is found out that the study has holes his/her name will be on the line as well, so they may pay more attention. But on the whole, assuming I'm not missing anything, I don't see the peer review as being all that powerful of a concept.
The term "peer-reviewed" refers to the original research, not the blog post. The idea of the system is to flag posts directly reporting on quality, peer-reviewed research (ie any creditable journal of science) instead of news blurbs or press releases.
Essentially, it is a way to mark posts that have read the original report and are writing about it, instead of posts that have read a secondary source (like BBC, Discover, NewScientist, etc) and are receiving a watered down version of the original study.
Peer review is somewhat of a joke.
In the hard sciences-- physics, etc-- I can see how it matters. But in something like psychiatry-- or any social science-- you run into the very real problem of reviewer prejudice: all reviewers are, essentially, from a homogeneous group: academic psychiatrists.
Not just that, but the same people review, over and over again. And, you may be surprised to learn that many hournals let you "suggest" your reviewers.
If a study funded by Wyeth showed Effexor was the best, you wouldn't be surprised. So why would you be surprised that an academic who has based his career on a certin concept (e.g. antiepileptics for bipolar) wouldn't "review" accordingly?
So peer review, while raising the standards of the actual research that is published, also funnel research based on a prejudice.
You don't believe me. I know:
Farewell, Depression
Tell me I'm wrong.
After coming through so many peer review researchs ,sometimes we're in danger of forgetting the original reseach,which should have been worth more attention.A website will be useful to help distinguish both of them,over which we can get out from the reaserch directly what the themes and variations are.
It is true,we are keen to keep a skeptical attitude to the primal reaserch,in fact, this was way these form come to transfer to the peer review .Obviously,we shoult not take product in front of the process.
The truth hardly would reprove our indolence of respecting the original reserch,and consulting to all of the relative peer reviews.
Alone, I would agree there are problems with peer review, some of which you point out, but it's too harsh to say it's a bit of a joke. It's a human system so it's vulnerable to the usual human frailties: jealousy, envy, pride, greed and so on.
That's certainly a dramatic post by the Last Psychiatrist though...
I actually read this blog, because most of the research you mention are from peer-reviewed papers. Of course this system has its own weaknesses, but it is still more than no control at all.
I also don't thing psychiatry is a "social science".
Ie hit the nail on the head. In my experience (not hard science, like physics) peer reviewers act like gatekeepers of the status quo and fresh ideas get short shrift.
WuffenCuckoo
I was an academic but not in psychology. At least in my field, "peer review" didn't necessarily mean all that much in itself, it really depended on the journal. (In some journals, peer review is a very low bar.)
I'm not sure how to communicate this to the public (like me). Informally, the journals in my field could have been divided into tiers with rough agreement. If you report that the research was published in a tier 2 journal, that might eventually mean something to most people.
Grue, so you'd suggest a way of showing the tier from which journal comes? It's an interesting idea although probably not something I'd do here as I'd guess that it doesn't mean much for most people. I was only originally thinking about whether or not it's worth pointing out that research is peer reviewed or not.
The feeling I'm getting from the comments so far is that the people who know what peer review means aren't that impressed with it - although it's better than nothing.
peer reviews are ok - they suggest that it's likely that the researcher in question used some commonly accepted guidelines. and yes, they also function as gatekeepers.
but what's the alternative? nothing that i know of.
it would be nice if there was an independent, well-educated group non-experts who would look at research as well. however, most people (and unfortunately that includes some researchers) lack even the most rudimentary understanding of science - social science, "hard" science or otherwise.
Isabella, yes I think you're right.
Here's a radical thought though. Perhaps part of the problem is that the peer review system has no money in it. Academics peer review each other for nothing. If they were paid perhaps it would be tighter. Money attracts competition and this may push up standards.
Who should pay and how that would work is a whole other world of problems though...