Posted by gazjjohnson on 28 April, 2009
Those of you interested in the preparations for the REF (which with my LRA hat on I’ve been somewhat involved in here at Leicester) will probably be interested in the following publications from HEFCE website
The second of these makes for an easier read, though those of you very interested in the REF will find a lot of information in the ProVCs workshop papers. I must confess most of that paper covers areas that I’ve got very little interaction with, interesting though the insights into the thought processes at that level might be.
David Sweeney’s presentation though gives a rather thorough examination of why we need the REF (or something like it) stemming from the lessons from the RAE 2008, from the funders’ point of view. And if the funders think it is important, then those of us in HE need to think about it a lot as well.
From what I’ve seen so far – it’s either going to be very, very complex in gathering the data, esteem factors etc and take a lot of staff time to support. Or it’s going to end up being quite easy for most people, but a fair amount of work for a very few administrators. Role on the June workshop at Kings to tell us more!
Posted in Research Support | Tagged: hefce, infornation, lra, news, ref, research excellence framework, workshops | Leave a Comment »
Posted by gazjjohnson on 19 March, 2009
As anyone who’s following me on Twitter knows, the last week or so has been rather dominated by my work on bibliometrics. Let me state up front here, I’m not a bibliometrician (sounds worryingly close to mathamagician to me) nor statistician, rather I’m a former scientist who spent a lot of time working with stats in another life. I sat in on a meeting about statistical teaching last week which served to rather poitnly remind me of all the things I used to know how to do (linear regression, chi squared, two tailed T-tests etc).
On the other hand I’ve always quite enjoyed working with data collection and simple anaysis; when I was a library researcher at Univ Warwick I spent quite a bit of time doing just this. So this does mean that any outputs that I produce aren’t going to be stunningly complex, but they should help people to get a picture based on fact. This, and my role as LRA personage involved in the Research Excellence Framework (REF) preparations, are doubtless why I was tapped by Louise to run a bibliometric profile of the Chemistry dept.
Bibliometrics, in case you didn’t know, is the analysis applied to texts or information. In this case I was asked by the Dept. to run a sample profile of their publication outputs; in an attempt to establish where they stand in relation to the rest of the academic world. In practise this meant taking a sizable sample (half the departmental academics) and looking at which journals they’ve published in over the last 9 years (2001-date). This is a key range for a number of reasons – firstly due to the suggestion that the REF will take account of publications back to this date. It’s also due to the fact that Journal Citation Reports (JCR) only goes back to 2000 online, so it’d be harder work to analyse publications beyond this point.
Now whilst the results are naturally confidential at this point I can tell you about what I sample in brief
- Article outputs- Number of articles produced and indexed within the time frame.
- Citation counts - Number of references to articles produced.
- H Index- The Hirsh Index quantifies both the actual scientific productivity and the apparent scientific impact of a scientist. It is increasingly viewed as a major indicator of academic esteem. Anything over 100-120 and you’re into Nobel laureate territory.
- Journal Impact Factors – A measure of how often all articles in a specific journal have been cited. Usually the most common “How important is this journal?” value.
- Cited Half Life – measures the number of years, going back from the current year, that account for half the total citations received by the cited journal in the current year.
From these I’ve been able to profile the departmental research publications as a whole, as well as getting an idea about the impact of the individual contributions to it. Quite looking forward to discussing the results with Chemistry in the near future.
The biggest challenge (data collection aside, which currently is very long winded) is knowing when to stop. I’m still very new to bibliometrics, and my inner scientist kept suggestion other ways to contrast the data or to analyse it. Essentially I could have been at this for weeks. And since we’re still not quite sure what metrics the REF will be using there didn’t seem much point in going to far with the first attempt.
There’s also the question of benchmarking. Raw stats on our depts are all well and good – but where do they stand contrasted with the rest of the world? That’s something that I might need to follow up on, but would likely be a far more time consuming operation than a week’s work. For now the Chemists might just need to trade notes on H Factors held by comparator academics, in contrast with their own.
Posted in Research Support | Tagged: analysis, bibliometrics, chemistry, cited half life, h factor, impact factors, profiling, ref, research excellence framework | Leave a Comment »