So you’ve been sent a paper to review. Before you even start thinking about what to do, and before you start thinking about reading beyond the abstract, it’s a good idea to check the stance you are about to take.
As an editor I see a lot of reviews. Many of them are productive and helpful. Some are on the blunt side and could do with being toned down a bit. A significant minority however are very tough, and some are just downright nasty. I’ve come to see that there are basically two kinds of reviewers out there.
First of all there’s Dr. Jekyll, the collegial respondent:
Dr. Jekyll is mindful that they need to read and review the paper They’ve been sent in the way that they would like their own work to be treated.
Reviewer stance: The writer is doing their best. They have spent time on this paper and they want to be published.
Primary questions: What does the paper add to what we know about this topic? What is sound about the argument and evidence? What is its potential contribution?
Action: I will focus first on the paper’s strengths. I will summarise how I understand the paper so that the writer can see if I have read it in the way they intended. I will then look for the key ways in which the paper might be improved. I will single out only the most important things that need attention and assume that as the writer is fixing these they will find the other things that follow on. I will maybe provide some helpful additional reading that might add to the texts the writer already knows.
Dr Jekyll may sometimes say what we don’t want to hear – after all, who really wants to do more work on a paper that’s already taken months – but the care they’ve taken with the response compels us to think about it again.
Then there’s Dr. Hyde, the ruthless critic: On getting the paper to review, your collegial peer goes through a sudden transformation and becomes Dr Hyde. Dr Hyde has a very big dose of ‘the will to critique’ and is determined to exercise it, gagging for it you might say.
Reviewer stance: There are bound to be serious weaknesses in this paper that need attention.
Primary questions: Where are every single one of the gaps in this paper? What hasn’t been cited that could have been? How would I have written this paper (much better obviously)?
Action: I will list every single thing that is wrong with the paper, in no particular order of priority. I will not explain or evidence my opinions. I will refer to other literatures the writer ought to know, but not give references – if the writer is any sort of scholar they will be able to find all of them. If the paper is using theory in a way that I disagree with, I will suggest that the writer doesn’t know what they are talking about. Critique is after all about pointing out what’s wrong.
I’m sure that many of us have been on the receiving end of Dr Hyde! Dr Hyde’s responses don’t encourage revising a paper and are more likely to have the effect of making the writer give up. Is that what you want to do?
So now you have the paper, it’s time to think about what kind of reviewer you want to be.
See also patter posts on reading,reviewing and giving feedback on a journal article here, here and here.
Reblogged this on Political Animals and commented:
Sage advice before setting out to review an article
But do reviewers really review the book well? I’m reminded of C.S. Lewis:
“Until you come to be reviewed yourself you would never believe how little of an ordinary review is taken up by criticism in the strict sense; by evaluation, praise, or censure, of the book actually written. Most of it is taken up with imaginary histories of the process by which you wrote it. The very terms which the reviewers use in praising or dispraising often imply such a history. They praise a passage as ‘spontaneous’ and censure another as ‘labored’; that is, they think they know that you wrote the one currenete calamo and the other invita Minerva.”
I think everyone starts off being Dr. J, then the paper under review begins to wear down the reviewer. Mr H. then has to look for a few things that are not too bad to make the review less harsh.
Thing One: Peer review ought to be taught/experienced in doctoral education. Skills with peer review fail to make it into doctoral education save for the doctoral writer being at the receiving end of an opaque process subject to all the vagaries of the Jekyl/Hyde syndrome which is called the viva or oral defense. Give an A+ to a doctoral program that encourages students to engage in the publication of a student journal.
Thing Two: An Australian academic, Royce Sadler wrote some papers on assessment and self-assessment. His argument to make assessment criteria more explicit could help to soften the Jeykl/Hyde syndrome. To implement Sadler’s approach, journal editors would need to ‘train’ reviewers and provide them with guidance in the form of exemplars, discussion and perhaps a rubric. The goal would be to create a consensual understanding of the kind of critique the journal wants. Assessment of articles then could comply with principles of reliability and validity and inter-rater reliability.
Pingback: self-citation by proxy | patter
Pingback: Self-citation by proxy |