LLLotW 2023.39

Leiden rankings to add open-source version in 2024

This story came out in September and I thought it was interesting, though not for the reasons Leiden University might have hoped.

The Centre for Science and Technology Studies at Leiden University in the Netherlands, which publishes university rankings, plans to start a new ranking based entirely on open data and open algorithms in 2024.

The open-source CWTS ranking will sit alongside listings produced, as in previous years, based on bibliographic data from the Web of Science database of Clarivate*.

I might write something up about this at some point, but in short there are three problems here:

  1. The new index is simply in addition to the “normal” one;
  2. Web of Science is pretty problematic (see below);
  3. Rankings are extremely bad of teaching, research, and universities generally, regardless of what data they are based on.

Recalibrating the scope of scholarly publishing: A modest step in a vast decolonization process

There seems to be a bit of slightly dodgy geo-location data in this paper but apart from that, it's pretty interesting. This is by some people associated with Open Journal Systems and this paper is essentially a study of the reach of the OJS network (as best they can tell) and the extent to which it is represented in major systems purporting to encompass “global world's scholarly output” (not much, but Web of Science is the worst).

Situating Search

The last in our reverse-chronological list is a paper from Chirag Shah and Emily Bender (of “stochastic parrots” fame) published in March 2022. It's actually more interesting to me how this holds up perhaps even better now than it did before the explosion on LLMs and “AI” search tools that it is about. This paper is really coming at it from an information science perspective rather than a computer science perspective, and has given me a bunch of references I now need to read!

In short, from the introduction:

removing or reducing interactions in an effort to retrieve presumably more relevant information can be detrimental to many fundamental aspects of search, including information verification, information literacy, and serendipity.

But there is way more to it than that, outlined in the paper. Really interesting stuff for anyone who needs to assess search tools for a living (hint: it's me).

Libraries and Learning Links of the Week is published every week by Hugh Rundle.

Subscribe by following @fedi@lllotw.hugh.run on the fediverse or sign up for email below.