REFs, RAEs and h’s….what about the students?

A student perspective on the role of research assessments, influence indexes and impacts factor measurements.

by Chris A.J. Brown


As a geographer studying the UK labour market, measurements are clearly an important part of my research – levels of employment, self-employment, unemployment, jobs. Comparing these measurements against each other and across space is another key part in examining change and possible repercussions. However, academia in the UK as a whole has been developing a growing obsession with different type of measurement…of itself. Universities and staff are bracing, or indeed energising themselves, for the next round of measurements that asses the quality of research being produced by higher education institutions (HEI). The Research Excellence Framework (REF) due to be completed in 2014 is the new incarnation of the Research Assessment Exercise (RAE) that was last completed in 2008. There is much discussion about the relevance, purpose and benefit of such quality assessment but it is part of a wider inception of measurements beginning to encroach upon the academic world, not least that of impact and influence. The London School of Economics has its own research project dedicated to the impact of academic research.  However, these assessments and measurements of research and academics is set at a somewhat bureaucratic level and focus and has left me asking the question: what is the purpose of these assessments and measurements from a student’s point of view?

A recent discussion seminar within our department focussed on a 2010 paper by Andrew Bodman which broadly looked at the ‘influentialness’ of geographers within academia using the ‘h’ index (I will encourage you to look up its definition yourself rather than attempt that here!).  The discussion among the group largely focussed around the purpose of such quantification– that of the measurement and impact of academic research and the challenges that come with it.  The forthcoming REF will determine which universities within the UK are producing quality research which in turn will be used to determine resource allocation and funding. Indirectly such measurement will also create another form of ‘league table’ with regards to which institutions are producing the ‘most’ or ‘best’ high quality research and to a certain extent the individuals associated therein. The Bodman paper was however specifically relating to the role of individuals and their level of influence on wider academic research, predominantly in relation to citations and references that come from publications.

As a postgraduate researcher I was left with a couple of questions. Given the likelihood and growing  impetus within academia to introduce measurement such as the REF or even individuals academic influence based on the ‘h’ index, at what stage should postgraduates or even undergraduates concern themselves with this measuring? Further to this, why is the level of influence or impact that academic research is having upon students not being considered? As is often the case, it is the definition that confuses me. In the case of Bodman’s paper it is the definition of influence, or lack of,  that struck me in this respect- ambiguous at best. To me measuring an individual’s impact or influence based on citations among peers, colleagues or even policy is a very lateral practice.  If such measurement and assessments are to become commonplace among HEI’s, then surely it is somewhat ignorant not to consider the influence or impact this research is having on current and future cohorts of undergraduate students, a number of whom will remain for postgraduate research and academic positions. After all a large proportion of the HEI population are students.  As undergraduates we are predominantly directed towards the key and leading thinkers in certain fields of research. Effectively these people or their work may subsequently influence dissertation topics and further to that postgraduate or PhD research. Ultimately leading to the production of papers included in future REFs and impact factor ‘h’ indexes. This in my mind is a measurement of influentialness that is being overlooked.

A somewhat intriguing analogy used within the Bodman paper was the idea that some of these measurements and indexing can be considered in a manner of a forest – some of the trees are taller and stand out from the crowd but are not necessarily stronger or broader than others. Furthermore, the collective whole – or forest – must play a certain amount in contextualising and supporting directly or indirectly the growth and development of others, but receive little recognition if measured by simply by height.  To put more ‘real worldy’, there are many reason why measuring the influence of individual academics and their work is flawed. Apparent success, influence and impact measured in this way may be down to improvements in technology allowing for better journal and citation access, field of study, working on a ‘hot topic’, or purely publishing in the right journal at the right time. And to risk taking the HEI forest analogy even further, surely the role of tree saplings [under/post-graduates] has just as pertinent a role.

While these measurements are clearly important at a certain level for resource management, funding, support, academic rigour and even individual confirmation (or gratification), for the students who make up the bulk of the HEI population it remains to be a somewhat abstract practice that is easier to overlook than concerns yourself with – although the figures may be used to try and influence a students decision on where to study. Should students be concerned or engaging with this? Clearly the purpose of such measurements and assessments is a more structural and institution purpose. Perhaps a student  ignorance of sorts is a good thing and allows us to focus solely on our own research and measurements of our own data.  But I do feel there is perhaps a missing link in this quantification of academic research – student academics.

P.S. The irony of the Bodman paper is that it has itself received a very good ‘impact factor’, which I assume is largely due to the relevant topic it was covering. I also realise therefore the irony of writing this piece as a personal blog, as undoubtedly I will be checking the figures that measure how many people read it! Have I caught the bug!?!


Bodman, AR. (2010). Measuring the influentialness of economic geographers during the ‘great half century’: an approach using the h index. Journal of Economic Geography. 10: 141-156.