A podcast where neuroscience and AI converge.


Get full episodes and join the BI Discord community

Donate Bitcoin


Donate via Paypal


Listen to Stitcher

Latest Episode

BI 163 Ellie Pavlick: The Mind of a Language Model

Brain Inspired
Brain Inspired
BI 163 Ellie Pavlick: The Mind of a Language Model

Ellie Pavlick runs her Language Understanding and Representation Lab at Brown University, where she studies lots of topics related to language. In AI, large language models, sometimes called foundation models, are all the rage these days, with their ability to generate convincing language, although they still make plenty of mistakes. One of the things Ellie is interested in is how these models work, what kinds of representations are being generated in them to produce the language they produce. So we discuss how she’s going about studying these models. For example, probing them to see whether something symbolic-like might be implemented in the models, even though they are the deep learning neural network type, which aren’t suppose to be able to work in a symbol-like manner. We also discuss whether grounding is required for language understanding – that is, whether a model that produces language well needs to connect with the real world to actually understand the text it generates. We talk about what language is for, the current limitations of large language models, how the models compare to humans, and a lot more.

Want a Brain Inspired Mug?

Music by: