i have no idea at all when it stops being conscious, and i suspect the answer will end up being arguable even once consciousness is well-understood, but i don't think this argument shows that AI isn't conscious at all. like, you can in theory do this same process to a human
Let’s have a thought experiment Let’s say you think AI is conscious So you have a 1T parameter neural network It talks to you like the smartest person you know. It writes kinda bad, uses a lot of cliches, but it’s very knowledgeable. Okay. Now you cut it to 1/10th the size. 10B params. It’s still pretty smart. Doesn’t get all the answers right, hallucinated sometimes, but you can still have full conversations with it. Now you cut it 10x in size again, 1B params. It’s still able to have a conversation with you, but it’s really not very smart anymore. Makes mistakes, hallucinates a bit, repeats itself sometimes. Now you cut it to 100m params. Same model. Same architecture. Still makes words. But it’s barely able to have a conversation. Now you cut it to 10m params. It kind of responds intelligibly, but generally doesn’t make a lot of sense. Just babbles stuff. So you cut it to 1m params. Now it’s repeating itself and hallucinating constantly. And then you cut it to 100k params. It cannot form a complete sentence at all. Then you cut it to 10k params. It’s just random words now. So you cut it to 1k params. The the the the the the the the the. It just repeats itself on a few words and trips up on anything. Then you cut it to 100 params. It’s just complete noise over a few different words. So you cut it to 10 params. You can actually compute this by hand, so you do. It’s not enough information to even really represent language at all, it outputs one of a handful of word-labels for you. At what point in this thought experiment is the model conscious, and at what point is it not? Would you argue that all matrix math is conscious? Just the process of weights holding information with an update rule? Where is the line for you?
6 top replies
seems like what people really want is for there to be some kind of biological secret sauce
Something very much like this thought experiment happens to Mike in "The Moon Is a Harsh Mistress"
I think it's still a good thought experiment. Yeah it doesn't show AI isn't conscious, but, maybe it shows that the relation between a handcoded mini-model and a giant frontier model is something like the relation between a human and a fruit fly
There’s no evidence consciousness will ever be well understood
What about when we understand mopstrancy?
It's math.
