So you’ve sent the paper into the journal and now the referee comments are in your in-box. You finally pluck up the courage to open the email and what do you find? Contradictory comments. Not helpful. Not at all.
We all know that reviewers don’t always agree. Sometimes their disagreements are profound, diametrically opposed even. Usually the journal editor will suggest which of the reviewer comments need to be taken on board.
But this doesn’t always happen – and the two reviews are just sent out as is. This can leave you wondering what on earth to do. Which things should you take up? If one set of recommendations are followed then they lead in precisely the opposite direction to the other. You can’t do both. So you choose one, and then when the paper goes back to the reviewers, which it usually does, one will be happy and the other probably even more unhappy.
I’m in this situation right now. A few months ago I sent off an article which set out to challenge a concept usually taken-as-a-given by writers in a particular journal. I also used a somewhat unorthodox narrative structure to make my argument. I knew that either or both of the referees might have an issue with either or both the argument or the structure. While of course I wanted uncritical approval for my contribution, I was prepared for a somewhat puzzled, perhaps even hostile response. That’s what you expect if you offer a bit of a challenge.
As it happened the two reviews were about as different as they could be. The editor sent both to me, simply asking what I made of the vastly polarized responses.
One of the reviewers clearly ‘got’ what I was trying to do. They described the paper as ‘nearly perfect’ (very happy smile) but suggested it needed a stronger introduction to justify why I was challenging the taken-for-granted idea and a more extended conclusion which re-argued the importance of reconsidering the issue. In other words, they wanted me to write more about the So What and Now What questions. Fair enough, I thought.
The other reviewer didn’t get the paper at all. They suggested it needed major revisions. They wanted more of everything – more methods, more literature, more analysis. They really wanted me make the paper into an empirical report using the IMRAD (introduction, methods, report and discuss) structure. This was exactly the kind of response I had been expecting, and indeed it epitomized an aspect of the thinking I wanted to challenge.
It won’t come as a surprise that I am going to follow the first reviewer. However, the second reviewer’s comments are still useful. While I’m not doing any of the things that they suggest, I am using the points that they find problematic as a set of prompts for thinking about whether there is any way I can clarify any further why I am writing as I am about this particular topic using a very particular structure.
I’ll put all this in my response to the editor of course. If she/he decides to send the revised paper out to the same reviewers they will also have access to this letter and my reasoning. I’ll be saying something like … I have taken reviewer 1’s comments in entirety, and have added to the introduction and conclusion. I have not followed reviewer 2’s suggestions to the letter but have instead: 1. (reviewer comment followed by my response), 2.( reviewer comment followed by my response) … and so on.
So are there some general points to be taken from my current experience?
Well, here’s my view of the process:
(1) When you have conflicting reviews, you get to make the decision about what to follow and what not to. In fact you always get to make this decision with any reviews. You don’t have to slavishly do everything that’s been suggested. If the editor gives you a steer, it’s probably sensible to follow that, but even then, you can still argue your case.
(2) You need to write a letter to the editor saying what it is that you’ve done to the paper in response to the reviewer’s comments. You have to address each of the reviewer’s points, saying what you’ve done or explaining why you haven’t responded. It’s easy to do this as a table, but sometimes the digitised systems don’t allow for attachments, they just give you a text box. If this is the case, present a list with recommendations and then the changes or not- making your reasoning clear if you’ve not done what’s been suggested.
(3) It’s also a good idea to resubmit the paper with the revisions clearly marked as track changes, or highlighted. This allows the editor, and then the reviewers, to see what you’ve actually done to the text.
patter 
This is a rare insight into how expert readers and expert writers respond to each other in the review process, thank you. I am reading this as an EAP instructor (so as someone who has to help novice non-native academic writers fit into the British academic system) and I can see parallels between the review process you describe and the scaffolding needed in the writing process of inexperienced writers (especially in your points 2 and 3). What strikes me most is how, even at the expert writer level, there remains the need to ‘talk’ about the written text, to weave a meta-narrative around what the writer means and how they are making this meaning. This re-affirms the dialogic and socially-situated nature of writing, and how it cannot take place in an individual vacuum for it to then be judged in an equally sterile environment.
The articles below report on L1 and L2 reader-writer interaction specifically, and having now read this post on how an expert writer engages with conflicting feedback, I see even more the need to facilitate reader-writer interaction at the novice stage through a pedagogy that supports conversations around text construction.
– Pomerantz, A. and Kearney, E. (2012) “Beyond ‘write-talk-revise-(repeat)’: Using narrative to understand one multilingual student’s interactions around writing” in Journal of Second Language Writing 21 (3): 221-238 available here: http://www.sciencedirect.com/science/article/pii/S1060374312000483
– Thompson, G. (2001) “Interaction in academic writing: learning to argue with the reader” in Applied Linguistics 22 (1): 58-78 available here: http://applij.oxfordjournals.org/content/22/1/58.short
LikeLike
Pingback: A case of conflicting reviewer comments | Life ...
A good guide, particularly for those new to the process. Having been on both sides of this process I completely concur that the letter explaining is key. It needs thought and consideration, including page numbers and paragraph indicators. I would stress that in making the comments about why one did not follow a particular reviewers set of suggestions there needs to be an argued response that is respectful (even if you as the author feel the person is an idiot and missed the point). If they missed the point then other people will too, which in itself is helpful. I’m going to save this and circulate to my grad students!
LikeLike
Great post. Wish I’d seen this earlier – I was recently in this situation with one of the reviewers actually asking why did you not research (a completely different research question) rather than what you are writing about? Too late I find your advice very useful. Ah well, there’s always next time!
LikeLike
Pingback: The Saga of Academic Publication | Nic Combe