When you think about it this makes a lot of sense. The process of making a model essentially boils down to transforming the time dimension of a normal human experience into a spatial one - for an LLM words don't follow each other sequentially in time but occupy specific distances from one another in vector space, video models struggle to tell the difference between 3D and 4D motion etc. etc. It's the linguistic equivalent of transforming a Lotka-Volterra graph (which is potentially unending along dimension t and could thus contain ∞ bits) into a phase space plot that contains just as much information in a much smaller space. It's pretty neat too because it means that when we do get to actual conscious models they probably won't experience time anything like we do. They won't just be like smart, weird humans, they'll be totally alien, possibly to such an extent that real communication is entirely impossible, just on the Wittgenstein's lion principle.
1 top reply
wait that sounds like that one shitty movie