I would be rather surprised if the experience of being an LLM is dramatically more similar to the experience of being a human than the experience of being any other advanced neural net (eg protein folder) is to being a human.
2 more tweets in this thread
I would be shocked if both weren't much much less similar than a chimpanzees experience is to a humans.
now you're free to define consciousness as some threshold of experience complexity or something that buckets humans with LLMs and excludes chimps, or include everyone, but I feel the most fitting definition is "an experience that's kinda like that of a human"
10 top replies
It seems intuitive to me it's much more to do with the specifics of the function implemented by the universal function approximator substrate than the substrate itself
*if* an LLM has an experience of being, it would most likey be such like that if someone else told you everything about the world. it would be the best example of plato's cave because it uses language as the medium to learn about the world, not the world itself
man, language ain’t allat
do you think it's possible that we'll get to a point where there are llm-based agents in the future whose experience is pretty similar to being a human? i agree that being chatgpt probably does not feel very much like being a human rn
I saw the blocks of this take, but haven’t seen them put together this way. Lovely way to make this argument.
it's not about the architecture but the training. does it implicitly model itself in addition to the world, and does it use and update that self-model as it operates at test time? that's the crux. and LLMs do that way more than e.g. convnet classifiers.
This feels correct and people who think otherwise feel misguided to me
If an LLM running (or the system of a harness running an LLM alongside tools and memory) has felt experience at this point, I don't its experience has much or anything to do with its textual output. I tend to think there's no felt experience, but can't see how we rule it out.
Self-attention changes a lot of things, protein folders these days are often diffusion-based too, so perhaps one is more like being an inner monologue and the other is more like dreaming
It’s closer to being an employee in Macrodata Refinement at Lumon Industries.