So you’ve just got an article to review and you’re not sure how to go about it.

Before even beginning to read, the first thing to get clear about is the STANCE you have to take as a reviewer.

Once you’ve clicked ‘agree to review’ and you have the article in your inbox, you now have to put aside all of those debates about whether blind peer refereeing is a good or a bad thing, or whether it’s here to stay or on the way out. You’ve got the article and you need to do a good job. The author(s) has spent a piece of their life writing it, they have put their faith in the reviewing system – that’s you – and there is probably a lot riding on whether it gets published.

The job of reviewing is about deciding whether the paper is of sufficient quality to be published, not whether it ‘s the most ground-breaking piece of research you’ve ever come across. And you have to read the text, not as if it’s the paper you would have written if you’d done this bit of research, but rather as the research and writing that has been done.

This is reviewing as an appreciative critical stance, rather than one which is dominated by the will to offer a killer critique delivered in the most assertive and acerbic prose possible.

The second thing is to MAKE SURE THAT THERE IS NO CONFLICT OF INTEREST, that is, the article is not obviously written by your best friend or your worst enemy or is about a project that you’ve worked on. If any of these are the case you need to send it back. We do eventually end up reviewing things by people we think we know and this is tricky – we have to try to hold that kind knowledge entirely at bay. We have to treat all papers as if they were written by complete strangers. If we can’t do this, we have to send them back.

The third thing to do is to READ THE ABSTRACT AND THE ARTICLE RIGHT THROUGH ONCE. The goal here is to see what the author(s) is trying to say and to grasp the paper as a whole text.

Next READ THE ABSTRACT AND THE ARTICLE AGAIN asking yourself these TWELVE QUESTIONS – and I’m sure you can think of some others:

(1) Does the paper fit in the journal? Does it address an issue /problem /report on a piece of research which the readers of this journal will find relevant and/or of interest? The answer here is likely to be yes, since Editors usually weed out articles that don’t fit before they send them out, but it’s useful to ask the question.

(2) What is it about the paper that will be of interest to readers of this journal? What existing debates, spaces in the literature, problems or issues does it address? Is it connected to existing and ongoing conversations in the journal and if so how? Does it explicitly refer to other articles in the journal about the same topic? Is this paper of sufficient interest – that is, is it something that is significant enough to warrant publication? What does it offer that is new? ( Of course it might be highly innovative or field changing, but remember it just needs to be new enough and important enough to this readership, not world shattering – and yes, that’s a value judgement and all of this refereeing process is, which is what makes it contentious.) Can you justify your judgment?

(3) Does it establish a clear warrant for its topic within current policy/practice or the field, and if not, is the lack of warrant a problem in this journal – that is, are all articles expected to be explicitly situated in the field of readers’ interest in some way? (The vast majority of journals expect this.)

(4) Does it have a point to make? Does it have one, or at most two, ideas – or is it hard to work out what the point is? Could the point be made clearer, and if so where and how – check the introduction for a statement of intent and the conclusion. Can you summarise the point the article is trying to make in a sentence or two? If you can’t do this, then there is a problem with the article.

(5) Does it refer economically to the key literatures and/or theoretical resources it needs in order to make its case? Or does it offer an inappropriate peacock’s display of reading?

(6) If it is an empirical piece of work, do you know enough about how the research was conducted to trust it? Do you understand the basis on which the writer says they will make claims? Or is it an over-detailed methods treatise?

(7) If it is a theoretical piece, is there sufficient detail about the theory to allow you to follow the way it is used? Is it a set of quotes strung together or is it clear that the writer knows and understands the theory they are using? Is it really just a self-indulgent theor-orgasm?

(8) If it is an empirical piece of work, is this reported in a way that is comprehensible and defensible? Does it go beyond the merely descriptive to offer some kind of interesting analysis? Is there enough evidence to show how the analysis has been made? Does this seem robust and rigorous? If quantitative, are the calculations accurate and sound? If qualitative, do the interpretations seem well justified?

(9) Does the conclusion address the so what question – here is this piece of research I’ve written about – so what? Who cares? What difference does it make to whom and why? Or is the conclusion just a restatement of the article? Or does it introduce new information? (Neither of these last two options cut the mustard as a conclusion.)

(10) Is the abstract a fair representation of the article that you’ve read? Does it tell the reader what is to come? Does the title aptly sum up the essence of the piece? Does it move beyond an advertisement for the article and offer a taster of what is to come? If you found this title online would you want to click on it to go further?

(11) Is the article well written? Is the prose too dense or too naïve? Is it well balanced, that is, it’s not top- or back-heavy /light? Are there enough headings or too many? Are the headings informative – could you understand the argument in the article just from reading these headings? Are the sections in a logical order? Do they flow from one to another or does the reader get lost? Is there some clear signposting for readers to follow, particularly if the argument is complex?

(12) Does the article meet the journal conventions in titling, headings, referencing and word length? Does the English expression need attention? Has the article been carefully proof read?

Now CHECK YOUR BIAS. Do you disagree violently with the article? If so, on what basis? Could you justify publication on the grounds that the article is well written, well-argued and defensibly produced but nevertheless contentious. Consider whether this journal is one which welcomes internal debate. Would this article contribute to that? One of the critiques of peer reviewing is that it supports the status quo – do you think that there is any chance that your judgments do that, and if so, is this OK or could you live with making a more risky judgment?

You now have sufficient information to make a decision about whether the article is publishable, needs revisions and if so to what extent, or should be rejected – and that’s another task and the next blog.

But finally, remember that the reviewing process is CONFIDENTIAL and as much as you might want to, you can’t discuss it with others.