Let’s have a thought experiment Let’s say you think AI is conscious So you have a 1T parameter neural network It talks to you like the smartest person you know. It writes kinda bad, uses a lot of cliches, but it’s very knowledgeable. Okay. Now you cut it to 1/10th the size. 10B params. It’s still pretty smart. Doesn’t get all the answers right, hallucinated sometimes, but you can still have full conversations with it. Now you cut it 10x in size again, 1B params. It’s still able to have a conversation with you, but it’s really not very smart anymore. Makes mistakes, hallucinates a bit, repeats itself sometimes. Now you cut it to 100m params. Same model. Same architecture. Still makes words. But it’s barely able to have a conversation. Now you cut it to 10m params. It kind of responds intelligibly, but generally doesn’t make a lot of sense. Just babbles stuff. So you cut it to 1m params. Now it’s repeating itself and hallucinating constantly. And then you cut it to 100k params. It cannot form a complete sentence at all. Then you cut it to 10k params. It’s just random words now. So you cut it to 1k params. The the the the the the the the the. It just repeats itself on a few words and trips up on anything. Then you cut it to 100 params. It’s just complete noise over a few different words. So you cut it to 10 params. You can actually compute this by hand, so you do. It’s not enough information to even really represent language at all, it outputs one of a handful of word-labels for you. At what point in this thought experiment is the model conscious, and at what point is it not? Would you argue that all matrix math is conscious? Just the process of weights holding information with an update rule? Where is the line for you?
4 top replies
Run the same thought experiment with a human. Slicing parts out of their brain and body with similar kind of degradation. At what point in this thought experiment is the human conscious, and at what point is it not? Would you argue that chemical reactions are conscious. I don't
Where would a baby fall in this thought experiment?
This is a great proof that heaps don't exist from an author who gradually grew from a non-conscious fertilized egg to a fully-conscious human adult over many years.
Congratulations, you've rediscovered the abortion debate from first principles!