Tuesday, 22 April 2014

The ins and outs of peer review

When a peer review request comes in, it's very exciting as an early career researcher, as it shows there is someone out there who respects your views. But, it can be quite daunting when you sit down to do it. Having done a few reviews myself, and recently been to a workshop on peer review, I thought I'd gather up my knowledge into a how-to post for those who are new to it. Please comment any other thoughts or questions below.

Should I accept the request to review?

You should have a good understanding of research in the field if you agree to review the paper. However, as an early career researcher it's tempting to think that you're not expert enough to review it. The chances are that you've been asked because of your knowledge in that field. If you feel like you would have useful things to say, and know some of the literature, then I think it's safe to accept. Remember that you are one of two (perhaps more) reviewers, and the overall responsibility lies with the editor. You're not expected to be the number one expert on that topic, and your opinion will be one of a few.

Another thing to bear in mind is that you should not have conflicting interests (in an extreme example, a colleague of mine was asked to review her own paper!). You need to be impartial, so if you know it's written by your best friend or by someone you hate, then you should decline. Finally, if you're too busy to do it any time soon then definitely decline - imagine you're the author; nobody wants to be sat waiting for months.

What kinds of things should I write?

When I did my first couple of reviews, I wrote corrections to every minor error including spelling or punctuation mistakes. I've since been told that this is not necessary! If there are lots of small errors like these, it's worth just writing a note to the editor that there are some small mistakes throughout and the manuscript needs further editing. As a reviewer, your job is more about the academic quality rather than spelling and punctuation.

It's important to keep suggestions positive if possible.

Have the authors included the relevant literature? If there seems to be an important area of the literature missing, suggest some papers that they could use rather than just pointing out a hole. Is there evidence for each claim made?

Have the authors used the appropriate methods and analysis? If yes, then say so. If not, you could ask for a rationale behind their choices, or if they have considered doing it differently. If you find there is an area of the methods or analyses that you're not familiar with and don't feel you can comment on, that is fine. Again, a note to the editor explaining that you don't know about that particular area is helpful.

Do the authors make appropriate conclusions based on the data? I think this is a really important one. At journal clubs this is the kind of question we ask ourselves, and too often the answer is no. If the authors need to be more cautious in their conclusions then say so.

Does the manuscript have a clear message throughout? Sometimes in first drafts it's difficult to figure out what the overall message is. If you're not really sure what the paper is trying to say, then suggest that it needs to be clearer.

How long should I spend on it?

I couldn't track down the reference for this, but apparently reviews reach their peak of quality when completed in 3 hours. This is probably a helpful guideline - enough time to read the paper a couple of times to begin with, making notes, then to go into detail for the completed review. Of course the first few may take a little longer while you get the hang of it. I found at first that I'd read a paper once and think I didn't have much to say, then when I read through it a second and third time I found that I had a lot of comments.

The review system usually asks you to complete the review within a month. Again this is a good guideline. Nothing terrible happens if you go over the time limit, but it's good to get it in on time if possible. At my recent peer review workshop, we were told that if you are someone who spends a long time doing reviews, your papers are likely to purposefully be sent out to other reviewers who also take a long time! Pretty shocking, but apparently true!

Some final notes

Ethically speaking, you shouldn't show the paper you're reviewing to anyone else.

Keep in mind how you would feel if you were the author. You'd want useful, constructive feedback. A further word of caution is that if you review a paper anonymously (most likely), the author may be able to find out who you are in the future, if they then work as an editor for that journal. Don't be needlessly rude as you may suffer in the future!

Finally, remember that you're there to help improve the paper. It is very satisfying to get a paper back to re-review it and see the changes you suggested put into action. You're doing your bit for science as a reviewer!

No comments:

Post a comment