LLLotW 2023.03
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
In 2021 Timnit Gebru, the head of Google's “Ethical AI” research team, was abruptly fired. The reason? She refused to remove her name from this conference paper. I've read a lot of articles about chatGPT specifically and generative AI generally in the last two weeks, but this article really explains best the fundamental problems with the very large language models behind all of the tools gaining so much attention. No wonder Google didn't want to hear it:
...human-human communication is a jointly constructed activity... Even when we don't know the person who generated the language we are interpreting, we build a partial model of who they are adn what common ground we think they share with us and use this in interpreting their words.
The problem is, if one side of the communication does not have meaning, then the comprehension of the implicit meaning is an illusion arising from our singular human understanding of language...[Language Models] is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot.
Lib-Static is a provocation to rethink how we do digital infrastructure in libraries to recenter our technology choices around sustainable, pragmatic, and minimal approaches.
I keep thinking about this. Probably because I'm the sort of person who always prefers to fire up a text editor and write in Markdown rather than firing up a word processor or, heaven forbid, a WYSIWYG edit-in-browser setup.
ChatGPT Is a Blurry JPEG of the Web
You might have already seen this one, and apologies for all the AI/GPT links recently but, well, that's been my main focus at work the last few weeks. I really like this article because it manages to explain why the technical implementation of large language models will always make them “hallucinate”, in a way that normal people without PhDs in mathematical modelling can actually understand.
Enjoy!
Libraries and Learning Links of the Week is published every week by Hugh Rundle.
Subscribe by following @fedi@lllotw.hugh.run on the fediverse or sign up for email below.