This post is from Inger, Thesis Whisperer, about the process of researching academic blogs. Here she discusses making decisions about method, and provides a glimpse, via a link to her google doc, of actual data analysis happening in real time.

I used to be a designer. Doing research and making buildings are creative processes that are frequently messy and uncertain. Of course, the mess in both cases is inevitably tidied away at the end, leaving only gleaming lines and surfaces. In my opinion the research community does not talk about mess often enough, creating the perverse impression that research is effortless if only one has enough experience and talent.

Not true.

For that reason Pat and I are sharing our ‘warts and all’ experience of writing a paper together. Our paper is attempting to make a provisional taxonomy of blogging practices amongst academics. We don’t want to know everything about academic blogging; this analysis is a way for us to start asking some interesting research questions about it.

Pat has been sharing her literature review techniques with you in a series of very interesting and useful posts. I’m sharing my experience of the analysis work for the paper. Just as Pat is showing you one way of doing a literature review, suited to this particular project, I am showing you one way of doing an analysis – but as a springboard for talking about managing the self during this difficult part of the research process.

My first challenge was to carve out a sample of blogs to study. As I wrote in my last post, the only way forward was to accept that a representative sample was not possible and work instead for a ‘good enough’ sample. Using the snowball technique I talked about in that post I now have around 160 blogs on my list. While I will keep adding blogs as I find them, my attention and effort has now turned to analysing them.

Choosing the tools to help you carry out an analysis will affect what you are able to find out. I could, for instance, have taken ‘field notes’ on the blogs I found and interviewed their authors. This approach allows interpretation and analysis to occur in tandem with collecting the data. This kind of ‘ground up’ approach, which is usually referred to as ‘grounded theory’ (Strauss and Corbin, 1998) because themes and categories are meant to emerge from the data.

Many researchers think grounded theory is a more ‘honest’ approach to qualitative data because the researcher does not impose their preconceptions on what they are studying. Others argue that the researcher is never removed and that to try and pretend otherwise is dishonest. It can be hard to move forward when you get caught up in these epistemological debates. What position is right?

I have my own views, but my decision about how to proceed with the analysis was not informed by my epistemological world view; it was informed by how much time I had on my hands. Time routinely shapes how research is done, but it’s a bit of a dirty secret. Not many people will be honest about the effects of time, because it’s seen as a bit of a cop out. But I promised warts and all…

It takes longer – far longer – to discover patterns in the data by massaging it carefully in the way grounded theory practitioners prefer. When I work in this mode with interview data I use a 1/6 ratio: for every hour of interview data there are at least 6 hours (or even more) of analysis and interpretation. If I followed the grounded theory approach I would be able to study, in detail, only a small number of blogs and bloggers – even if I had a year or more to conduct the study.

As a professional academic I just don’t have this kind of time, but I struggled with this realization. The researcher inside me yearns for aesthetically satisfying truth claims; the professional academic yearns for publishing runs on the board (and the chance to work with and learn from most excellent and talented Prof Pat Thomson!). I don’t want to compromise the quality of my work (Pat might think I was a crap researcher!), so I needed to find a compromise between time and quality that I was comfortable with.

After some careful thought I realised the kind of knowledge we want to create about blogging is similar to some of the banal work I do for the School of Graduate Research. When we want to introduce a new policy or procedure we look at what other universities are doing about the same problems; we call this a ‘bench mark’ study.

I am good at bench-mark studies because I have developed technological ‘hacks’.

One of these hacks is the humble spreadsheet – a tool of great analytical power. As I remarked to Pat, a good spreadsheet can make you feel like a God with some all-seeing eye. Over the years the technology to collaborate with spreadsheets has become quite sophisticated. It took me about 2 minutes to make a Google spreadsheet with the research questions along the top and the list of blogs down the side.

I’ve learned to be careful with spreadsheets as they can flatten out important and subtle distinctions. I found myself tempted to delete blogs that didn’t easily fit the categories I had created. I started to create more and more categories and columns in an effort to preserve the complexity of the data.

The tradeoff was that the spreadsheet became painfully messy. I found myself moving back over the spreadsheet trying to create consistency. I know that consistency will speed up the process of compiling my results at the end using the various sorting tools I have available to me. But this iterative activity, while it creates order, also acts to smooth out the ‘noise’ in the data. It’s not really noise I’m losing – it’s fidelity, accuracy, diversity – all the good stuff my researcher self yearns for.

I’m painfully aware of that loss. Awareness of loss creates discomfort, which I soothe by reminding myself of the rhetorical purpose of this paper. It’s work in progress. One of the side benefits of a work in progress paper is the creation of a data set and a mark in the sand that can be used to inform future research.

Research is full of these ambiguities and trade offs. I’m still learning how to dampen the little voice in my head that yearns for aesthetically satisfying and complete pictures of the world: that’s a rabbit hole I don’t have time to fall down.

As the explorations in style blog noted this week, it’s dangerous to compare our half finished work with the finished work of others. In writing this series I have realised that one of the most important abilities I carry with me from my decade or so in architecture offices is a high tolerance for mess and ambiguity. I hope by letting you into my research mess you can take comfort, even joy, in your own.

Postscript: See more on research mess here, here, here, here and here!! Clearly a topic to love coming clean about!