Friday, 20 February 2009

Bibliometrics and the REF

The Research Excellence Framework [REF] will use citation analysis or bibliometrics as one part of research assessment. The extent to which bibliometrics is used compared to qualitative methods will depend on the subject. Subjects that favour publication in journals and conference papers are better suited to bibliometric analysis than subjects which use the full range of publication and other forms of non print outputs - performance, animation or artefact's.

Citation counts for individual researchers do not tell us much about the impact of an article or conference paper, is 24 citations good or bad? Set against the average of citations in a particular subject it is possible to assess whether a paper has above or below average citations. So if the average number of citations for papers in subject X is 16 then 24 (24/16) is 1.5 times the norm for that subject.

The REF is likely to profile the output from one subject within an institution against the norms for citations in that subject assigning outputs to categories on a scale of excellence, below average citations through to well above average (see link to leaflet on Bibliometrics and the REF below).

Perhaps more of a challenge is how to define subjects in terms of collections of journal titles from which citation subject norms can be derived. The REF may follow a similar path to that of the Excellence in Research for Australia [ERA], creating lists of journals through a process of expert review and assigning journal lists to Fields of Research [FoR] [2 digits] further broken down into sub fields [4 digits].

Links

Options for Defining Normalisation Fields - A report to HEFCE by RAND Europe