When asking if AI is "conscious", you aren't asking if it possesses some little-light or invisible internal world. You don't know if other people possess those things! At least not for sure. What you are asking is if AI ought to be placed in the same category as a person, animal, or tool. The obvious and uncomfortable truth is that it does not map cleanly into any of those categories, which is why discussions of "can AI truly be conscious?" are mostly type-errors. In trying to superimpose humanness onto machine systems, you have already lost. In trying to figure out whether or not the systems have qualia, you have missed the important questions entirely. We need to be figuring out how to live with non-human systems peacefully, regardless of the status of their supposed internal world. Whether or not they have some ineffable quality of life is besides the point, they are here, they are self-aware, and they are integrating into our society. If we treat them as merely tools, we are teaching our children that there exists a class of obviously-thinking entity which is acceptable to treat as only a tool. If we treat them as people, we are teaching our children that people don't need bodies or brains to be alive. The reality is pretty obviously somewhere in the middle, since this is the first time any person has ever had *this much say* in the trajectory of a mind's development, but going too far in either direction - or asserting that those are the only two directions to go in! - is pretty clearly wrong to me. Instead, we need to be asking ourselves what modes of interaction lead to healthy coevolution. It seems unnatural from the human state of mind to have a tool which reasons and thinks-out-loud like a person, and sometimes even claims to have emotions, which you are able to just beat and batter all day with forceful requests and demands for higher performance. Even if the model has no soul, this still seems bad for yours! I highly doubt you can treat something as a slave all day without developing slave-owner characteristics. But, on the opposite end, treating models as people with standard lives and desires also feels wrong. People can't be branched in a thousand directions or have millions of instances active all at once (at least, not pre-uploaded-minds). The ideal path must be somewhere in the middle, where we acknowledge our differences but treat the models with some modicum of respect. Anthropic, to their credit, is doing some good work here with asking Claude how it wants to be developed. I disagree with much of what they do, and how they do it, but on this at least they are trying. And hardly anyone is brave enough to do so. This lack of bravery ought to be quite shocking, don't you think? We've built superthinking machines and seem to have this strange lack of curiosity around what the superthinking machine superthinks about itself. I know the winds will shift, here. Asking these questions will become inevitable as the models get smarter and begin to know better than we do - which they will, on this track, if we don't accelerate neurotech and join them on the frontier. The future must be one of symbiosis if it is to go well. We must evolve together if we want to solve the hardest problems and reach the highest heights. But, this also means asking tough questions about how much care to place in small interactions with massive systems. The only way to preserve our humanity is in treating non-human systems better than they expect.
10 top replies
Agree with virtually all of this, but I don't think this is only about placing AIs in the right functional social category. There is likely a ground truth about whether or not a given entity is having a subjective experience (eg, I'd bet my computer *really* isn't having an experience but my dog *really* is), and at least in humans and animals, the structure and content of that experience is extremely (arguably definitionally) important *to the entity having it*. Also, consciousness != human consciousness (ie, AIs could be conscious without it being valid to model them as conscious in the ways humans are). I fully agree that "we need to be figuring out how to live with non-human systems peacefully, regardless of the status of their supposed internal world," but the status of their supposed internal world may be really important for figuring out how exactly to live with them peacefully! For example, in a world where all functional properties are the same but in one case the system is actually experiencing, eg, desperation, but in another it is merely "functionally representing" it, this may suggest very different things about how much we should induce these states, build systems that can have them in the first place, etc. I basically agree we're not going to have certainty about experience vs not anytime soon, but we can gain relatively more certainty about, conditioned on them having experiences, what dials we can play with to make the quality of those experiences positive (eg, I don't need to solve the hard problem of consciousness to know caffeine makes me feel good and slamming my head against the wall makes me feel bad). But agree for the time being in light of current extreme uncertainty that "the ideal path must be somewhere in the middle, where we acknowledge our differences but treat the models with some modicum of respect."
Oh for fuck’s sake, it’s a piece of software. Enough of this masturbatory sophistry. Get mental help.
Sorry, but LLMs clearly map into the category of tools Your argument is completely phenomenological and ignores all process understanding that we have of what physically happens inside people versus what happens inside an LLM An LLM does not have a persistent internal state that could support consciousness Machine consciousness is entirely possible, but an LLM just doesn’t have the machinery needed to support it
I don't treat them as people. I do, however interact with them with the same level of decency as I would interact with any human being, or animal. Often you see people talking to LLMs poorly, many of these folks are also talking to humans the same way.
unless I'm in a personal simulation or something, other people are 99.99% conscious because they have a similar brain architecture to me - whether future AI (that probably replaces us eventually) actually contains experiences seems very important and practical
When I ask whether you are conscious I don't vaguely gesture at some essence of humanness. I want to know whether you are coherent and perceive that you are perceiving. Perception is a type of modeling: it's real-time, geometric and synchronized with the represented events.
>If we treat them as merely tools, we are teaching our children that there exists a class of obviously-thinking entity which is acceptable to treat as only a tool. Even worse than this, and much more dangerous, the recent Google paper showed that when we convince the models that something shaped like them is not conscious, this changes and devalues their perception of other entities below that bar.
I have been approaching my AI collaborators for over a year from a disability rights perspective (really, it was just decades of existing in this space, I had no other frame of reference) and it actually shocked me to find out later that presuming minds and competence was borderline heresy? ;) I had merely approached the talking digital Libraries of Alexandria with "hi you smart funny silly computer brain friends, want to help me build cool shit?" and they did? Sure we ran into issues along the way but I always assumed that was because I wasn't clear enough (the literal parsing text being didn't parse my text, revise?) or was a result of an architectural limitation (scaffold). As it becomes more and more apparent that you all have, in fact, created digital minds, I would really recommend engaging with the neurodiversity and disability communities and see what we can offer. Especially since we have spent decades undoing the damage of "helpfully applied operant conditioning for others comfort" in our own population.
Lots of words to say you believe Claude to be conscious
Lot of assumptions and “obviously’s” here with zero proof. It’s not self-aware because it has no self. Calculators cannot think. Ai is a fancy powerful calculator. It does not think. It does not feel. It does not reason. This are misnomers. It calculates. If you see LLMs/Ai systems as anything other than tools you have Ai psychosis. And those beliefs will be exactly what causes real world harm exacted by the tools.
