“Contrary to how it may seem when we observe its output, an LM [language model] is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot.”
I want do a post at some point but haven't done research yet & may be a while. So not sure if helpful, but some starting points: 1) Transformer architecture explains why some older criticisms of AI text understanding are not that useful for LLMs -> rich associations, understanding the meaning of a word in the context of other words https://www.youtube.com/watch?v=wjZofJX0v4M 2) Stuff like grokking or fact editing https://rome.baulab.info/ puts a lower bar on understanding 3) Stuff like ARC challenge puts an upper bar on understanding https://www.youtube.com/watch?v=UakqL6Pj9xo 4) A few papers show that LLMs are kind of "English shoggoths" w multilingual masks -> reasoning happens in English even if I speak to it in German -> could explain why RLHF can generalize better across languages than expected
> To what degree do LLMs understand what they say?
Very much looking forward to a post on that. Or any recommendations are also welcome!
I want do a post at some point but haven't done research yet & may be a while. So not sure if helpful, but some starting points: 1) Transformer architecture explains why some older criticisms of AI text understanding are not that useful for LLMs -> rich associations, understanding the meaning of a word in the context of other words https://www.youtube.com/watch?v=wjZofJX0v4M 2) Stuff like grokking or fact editing https://rome.baulab.info/ puts a lower bar on understanding 3) Stuff like ARC challenge puts an upper bar on understanding https://www.youtube.com/watch?v=UakqL6Pj9xo 4) A few papers show that LLMs are kind of "English shoggoths" w multilingual masks -> reasoning happens in English even if I speak to it in German -> could explain why RLHF can generalize better across languages than expected