writing a highly cited paper – a sceptical view

I’ve been somewhat irritated recently, as I’m sure most academics have been, by the increasing interest that our universities have in citations. Citations count in league tables.  We – academics – are increasingly told that we must focus more on how to ramp up our own citations. It’s recently been explained to me for example that citations can be increased by writing review articles and by writing with colleagues from particular US and Australian institutions.

This kind of advice is invariably based on interpretations of citation metrics. I strongly suspect that these explanations of how-to-write-a-highly-cited- paper are a case of people  (who ought to know better) mistaking correlations for causes. Apart from the times when academics cite papers that are bad, or because they are written by someone already famous, my sense is that it’s more generally the case that the papers that are cited a lot also happen to be significant in some way. That’s not to say that all significant papers get cited a lot, but rather that if a paper gets a lot of citations it’s because colleagues are referring to what it says for a reason. Well, that’s my hypothesis anyway.

So how would I test this out? Well, I think it might actually be helpful to have some further research and analysis. Maybe we could actually look at some highly cited papers to try to understand better why they have become important in their field. It might also be interesting to talk to their authors and some of their readers, those who do the citing.

Let me give you an example to show why I think that such an endeavour might be of use.

A little while ago, and in my own field of education, an editor of the journal  Discourse asked an author – Stephen Ball – to look back at, and talk about, what is now a very highly cited paper. It was something that he’d written twenty one years previous. Ball recorded his reflections on What is policy: texts, trajectories and toolboxes (passworded) in a Youtube clip.

Ball pointed to the paper as a transition point in his own thinking. Ball suggests that the paper was intended to provoke peers and to create a new critical basis for policy analysis – to provide “more theoretical heft”. The paper not only marked a profound shift in his own research and provided the basis for his next two decades of subsequent inquiry, but also laid out a new agenda for the wider field. Ball had an ambition for the paper, beyond his own thinking and work, which went to the state of scholarship at the time. He wasn’t thinking about getting citations, but about purpose and influence. He was concerned about getting read but not about measurement.

This is a particular case, of course, but what’s most interesting to me is the way in which Ball is able to talk about his own research trajectory at the same time as making clear his ambition to make a very significant contribution to the field in which he was researching. He thought the field was limited, and he wanted to change it.

Even this one example sheds a little light on one way that important contributions to the discipline and to the field can be made through an individual  intervention. This single example illustrates a way in which making a significant contribution might be imagined and accomplished. Personal research and change in the field can happen together. Getting a highly cited paper in the field is a by-product, not the point, of the publication exercise.

My hunch is that something like Ball’s example may well be the case with a lot of highly cited papers. I can’t really believe that the papers that everyone refers to have been developed by people saying to themselves – Now I need to get a highly cited paper, I’m going to co-write a review paper with a colleague in a university above mine in the league tables. That’ll make sure I get a big hit in the indices. NOT. I suspect it’s much more likely to be the case that the highly cited paper comes from a happy meeting of luck, the zeitgeist and a researcher thinking seriously about their own scholarship in the context of the current state of the field.

In other words, I strongly suspect that the bulk of high citations are connected to scholarship, and are not a highly calculated and self-serving act.

I’m pretty convinced that we need something in addition to metric analysis to actually understand citations. Don’t we all teach research methods courses which say that numbers can do some things and not others? …. But of course we need many more than this one example I’ve given. N = one doesn’t fly either, even if it is interesting. It might be enough however to suggest the merit in looking beyond numbers when we think about influencing the field and getting readers.

But, in the meantime and in the absence of research more thorough than that available from indices, I’m going to take those metric-based causal analyses about how I should write, about what and with whom, with a very large grain of salt.

About pat thomson

Pat Thomson is Professor of Education in the School of Education, The University of Nottingham, UK
This entry was posted in citation, metrics, Stephen Ball and tagged , , , , , . Bookmark the permalink.

5 Responses to writing a highly cited paper – a sceptical view

  1. Haitham Al-Sheeshany says:

    “It is all about numbers” after all then? 😦

    Thanks for this Pat. Let’s hope for more (much more) papers of: (happy meeting of luck, the zeitgeist and a researcher thinking seriously about scholarship) elements in them

    Like

  2. Brian says:

    Pat, when doing my masters, I was guided to ‘preferred authors’ by instructional staff and my supervisor. Mostly becuase they were famous (to use your phrase). I also recall submitting a paper to a journal – based on my lit review – and having it sent back with guidance/instruction to inlcude the works of a particular author; eventhough they had no real relevance to my area of research, being police instructors. I decided not to resubmit.

    In my current doctoral program I was again advised [by my ex-supervisor] to focus on two or three prominent authors. Once again, they had little to do with my domain of focus, the emergency services.

    Like

  3. SheriO says:

    Ball wanted to influence other researchers with his insights and reorg of policy research with his seminal paper and that is well and good. He was already established as a voice “at the dinner party,” so others at the dinner party were inclined to listen to and be persuaded of his arguments.
    Not every paper is pivotal or seminal, but still contributes to the literature and draws upon it. Academic authors need to publish papers regardless of the metrics and add to the conversations as little and big voices contribute to the discussion.

    Like

  4. Pingback: What’s your most overcited paper? | Scientist Sees Squirrel

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s