As This talk by Anthony van Raan from the University of Leiden (who was a consultant to HEFCE on the use of bibliometrics I think)

Anthony is at the Centre for Science and Technology Studies (CWTS), who have been working on bibliometrics for many years. CWTS have a licence agreement with Thomson for the use of Web of Science raw data, but has also created its own bibliometric data system using algorithms.

Apparently citation isn’t as complicated as quantum mechanics! That’s a relief…

What do citations measure:

• There is a positive correlation between citation based studies and qualitaty judgements by peers

Anthony is going through this at quite a pace, so these notes may be sketchy…

There is a set of total publications, and various systems list subsets of this ‘universe’ – Web of Science is a big subset, but also LNCS, Medline, arXiv. Now Scopus is another big subset, and CWTS are now comparing Web of Science to Scopus.

Anthony is now going of key indicators and what they help measure – however, he is going through it so fast I can’t keep up – it might not be quantum mechanics, but as this speed I can’t capture it all.

As Web of Science covers (as you might expect) mainly science – and it is the ‘science’ based disciplines that use journal publication as the major route for scholarly communication. In Humanities and Arts, books are used more, and CWTS is experimenting with a new kind of ranking – which measures if ‘your’ book is available in the top 250 university libraries (he doesn’t say how top 250 university libraries are measured) – if this gets adopted, will we start to see an increased demand for libraries to buy specific books to increase ranking?

Anthony demonstrating how Chemistry researchers publish across journals from many areas (e.g. Rheumatology, Cell Biology, Applied Physics) – so even an established discipline like Chemistry turns out to be very interdisciplinary – at least when it comes to publication.

Anthony indicating that bibliometric indicators can be manipulated and/or they discriminate certain persons/types of research – but establishing evidence for some of this needs work – for example CWTS have found that the assertion that bibliometrics discriminate against younger academics is not supported by the evidence. However, they have found that journal editors behave in such a way that maniuplates impact factor (CWTS published an article on this in the Journal of Documentation)