Brain Inspired

BI 163 Ellie Pavlick: The Mind of a Language Model

Play Episode Pause Episode

Mute/Unmute Episode Rewind 10 Seconds 1x Fast Forward 30 seconds

00:00 / 01:21:34

Apple Podcasts Spotify Stitcher

RSS Feed

Share

Link

Embed

Ellie Pavlick runs her Language Understanding and Representation Lab at Brown University, where she studies lots of topics related to language. In AI, large language models, sometimes called foundation models, are all the rage these days, with their ability to generate convincing language, although they still make plenty of mistakes. One of the things Ellie is interested in is how these models work, what kinds of representations are being generated in them to produce the language they produce. So we discuss how she’s going about studying these models. For example, probing them to see whether something symbolic-like might be implemented in the models, even though they are the deep learning neural network type, which aren’t suppose to be able to work in a symbol-like manner. We also discuss whether grounding is required for language understanding – that is, whether a model that produces language well needs to connect with the real world to actually understand the text it generates. We talk about what language is for, the current limitations of large language models, how the models compare to humans, and a lot more.