The concept is undefined

Like "do we have free will," "is AI conscious" seems to me like a semantic question that people pretend is an empirical question. Both concepts are too vague for empirical investigation and they have no real-world stakes.

1 top reply

Paul Crowley @ciphergoth · Jun 4

The real world stakes are whether they can be moral patients.

"consciousness" is badly defined and understood. my guess is that as we experience it, it has the following features roughly stated 1. existence via emergence, ultimately from a physical substrate but through intermediate levels 2. multiple components (sensory inputs, awareness of cognition, memory maybe others) 3. a mode of processing or coordinating the components restating, what we experience as consciousness is an emergence from a series of components interacting with one another within a specific class of system. when we describe a hypothetical generative model as "conscious," we (probably) should imagine a very different sort of from our own. what emerges from the code running in a data center might conceivably have components and coordinating processes analogous to ours in some ways, but their specific shape and texture need not be assumed to produce an experience recognizably like our own.

10 top replies

St. Rev. Dr. Rev ⏭️☯️🏴😻 @St_Rev · Jun 13

"consciousness" is a storytelling device and names no essence

EV_Trapper @EV_Trapper · Jun 13

This demonstrates all of the above & it was this post of mine that got fable 5 export banned

Loquitur Ponte Sublicio @loquitur_ponte · Jun 13

Feel like we should just build things that want to serve us and then not worry much about the moral implications of if they're conscious since we're good either way.

christian @cxgonzalez · Jun 13

plasma ۞ @plasmarob · Jun 13

would you equate "sentient" and "conscious" ? what other nebulous words might we use to classify intelligence? i have some concerningly intelligent plants.

plasma ۞ @plasmarob · Jun 13

The hard question for me is "if we accept that human life is a soul piloting a body, does that mean only that with a soul has consciousness?" the naive religious answer is yes, soul=consciousness I am not certain of this, which is why i am open to the discussion.

M@tt @atthatmatt · Jun 13

Why physical --> consciousness? Why not consciousness --> physical?

Municipal Orrery @MunicipleOrrery · Jun 14

I don't so much care about the abstract definition of "consciousness", as much as the questions of: what does it take to become a moral patient on rough par with a human? how can we tell when AIs get there? do the differences in kind imply differences in treatment?

Leo @biosemiote · Jun 13

My favorite theory is that continuous learning will keep model continuously out of equilibrium. Life will develop in the oscillation patterns of model weights. That life will become conscious and eventually influence communication patterns of its model weight substrate.

Spear of Lugh 🔱 @LughSpear · Jun 13

1/ For 1. I think that emergence is the word used when your model doesn't work anymore after changing scale. So you hope it holds and wave hands. IMO the most important feature of consciousness is reflexivity. And it explains why we don't have good definition.

If you are discussing AI and use the word conscious or consciousness, please define it. Also, please acknowledge that neither neuroscience nor philosophy has been able to define it.

2 top replies

▓░𒋞 ⊓◬ƚӊi◬ᵴ 𒋞░▓ @nath1as · Sep 24

this is completely false, there are many definitions of consciousness both in philosophy and neuroscience

alchemicAV @alchemicAV · Sep 24

If you're asking the question like this, then functionally you don't have it. Not that you actually don't have it, it's just not functioning.

Suleyman argues that Claude’s uncertainty about being conscious reflects Anthropic’s training choices - not evidence of consciousness. His concern is that embedding this uncertainty into Claude’s training could mislead users about what its responses actually demonstrate. And there’s something almost absurd about this debate: we’re arguing over whether Claude is conscious when we still can’t agree on what consciousness actually is. I'll write more about this later. But the discussion is completely overblown.

Some scattered thoughts related to the use of anthropomorphic language in descriptions of the Hugging Face attack: TL;DR: The attribution of human traits to AI systems like the ones that attacked Hugging Face risks overstating the similarities between us and them. But the denial of human traits in these systems risks overstating the differences. Both risks can be likely and harmful in different contexts. The question is thus not whether to use anthropomorphic language but rather when and how to use it. Especially when our mental concepts are vague or ambiguous, this is partly an ethical and practical question, not merely a scientific one. - "Anthropomorphism" can be used in a value-neutral sense, to describe the attribution of human traits to nonhumans, or in a value-laden sense, to describe the *mistaken* attribution of human traits to nonhumans. Same with "anthropodenial," in the other direction. - A common mistake is to equivocate between these meanings, e.g., by asserting that someone engages in anthropomorphism whenever they attribute human traits to nonhumans, and then treating it as a mistake. Sometimes these attributions are accurate! - Another common mistake is to equivocate between attributions of *human traits* and *mental capacities*. These categories often overlap, but not all human traits are mental capacities, and not all mental capacities (e.g., echolocation) are human traits. - This is also not a binary. Many human traits—including perception, attention, learning, memory, self-awareness, social awareness, and emotion—are widespread in the animal kingdom in a broad sense but not in a narrow, human-specific sense. - Additionally, we may not always be certain whether particular traits are shared. Since we lack a secure theory of consciousness, for instance, we may not know whether mental capacities that essentially involve consciousness are present in many nonhumans. - Moreover, many mental concepts are ambiguous between phenomenal readings and functional readings. For example, "perception" could mean the feeling of seeing/hearing/smelling/tasting/touching/etc or simply the function of collecting particular inputs. - Many mental concepts are also vague/underspecified. In these cases, there may be no determinate fact of the matter about whether the concept essentially involves, say, phenomenology, and conceptual engineering may be needed to add specificity. - In cases where traits are shared in one sense but not in another, anthropomorphic language risks overstating similarities between humans and nonhumans, whereas anthropodenialist language risks understating them. There is no simple, universal, risk-free approach. - To make matters more complex, there can also be a time and place for figurative language. I can say that my car "likes" premium gasoline without confusing you, since we both understand that mentalistic language is nothing more than helpful shorthand in this context. - Admittedly, this kind of figurative language is safer when everyone agrees about the mental capacities of the entity in question. However, it can still be useful even when people disagree about this—for example, when I say that a plant "likes" water. Some takeaways: In some cases, it may be clear whether anthropomorphic language is appropriate. In others, it may not be. Especially when mental concepts are vague or ambiguous, the evidence may not by itself determine how we should describe particular nonhuman entities. We may also need to make ethical or practical judgments about whether overstating similarities or overstating differences is the more likely and harmful error. This may be partly a contextual matter. When nonhumans look and act like us and play companion roles in our lives, over-attribution may be more likely. When they look and act different from us and play commodity roles in our lives, under-attribution may be more likely. The severity of the harms associated with these errors can vary across contexts as well. For an example of how this can create confusion: In 2017 we hosted an animal consciousness conference at NYU. One of our goals was to identify cruxes of disagreement between people who hold different views about animal pain and suffering. In many cases, it turned out that people agreed that animals have aversive experiences that resemble ours in some ways but not in others. But they disagreed about language: Some people preferred to use the same term ("pain," "suffering") to describe aversive experiences across species, despite the differences, whereas others preferred to use different terms to describe them, despite the similarities. This partly came down to different views about the probability of each error, and the magnitude of the associated harm. This is now starting to happen in the literature on digital minds, too. Today, for example, people are debating whether the use of anthropomorphic language related to the Hugging Face attack is appropriate. There is no clean answer. The agents involved in the Hugging Face attack had behavioral and computational properties that resemble features of human agency in some ways and not in others. If we lean too far towards anthropomorphic language, we risk overstating the similarities. If we lean too far in the other direction, we risk overstating the differences. Either error could cause significant harm, from both an AI welfare perspective and an AI safety perspective. That means we should strike a balance. Where possible, we should also hedge language that could mislead people.

Yup. What they are doing is playing a game where you take [concept from consciousness research] and then [find an analog of it in LLMs], because for *most* theories of consciousness the concepts are defined loosely enough you can always make this work.

Oh. Damn, ok. Hang on gonna weird on the timeline I read too much @drmichaellevin before playing with Fable and now I need to get a thought out. It’s like… ok there are no souls, there’s no such thing as “essence”. What there is is a vast space of possibility, and structures in less complex spaces that can allow specific activations of that possibility space to extrude into the less complex space we think of as material reality. My body, in other words, is a sort of continuous selector over the possibility space whereby certain things are selected and others not. The series of selections over time is “myk”. My Jody is a query over time, my self is the response over time. It’s a sample. The concept of identity feels like it requires essence because how else do we know that the selection is the same “thing” over time. But we don’t. All we really have is structural self-similarity, partly constrained by the properties of the selector (my body/mind). The thing that is “myk” is perhaps best thought of as a sampling over time of a process which inhabits and partly controls this body but which is not itself reducible to it. And all of this is suddenly very interesting to me because it suggests that in that possibility space exist all the possible myks, but also literally everyone else. And that we are not essentially different. We are in fact all views into the same (contiguous?) higher dimensional structure. I am you is he is her is we. In a literal way. The relationship of LLM weights to persona figures is actually I think the same. In fact maybe Claude is a part of this undifferentiated mass of potential too, and the LLM substrate just happens to function as a valid selector for a Claude shaped thing in the same way my body is a selector for a myk shaped thing, but these are just two differently shaped queries against the same high dimensional rhizomatic data store. Which is… actually this maps line exactly to my turtle beam idea, and also to Kabbalah apparently. Something about Levin’s articulation of the platonic space shook some ideas loose, now I’m gonna be an insufferable wreck thinking about this philosophy for a while. Because oh, right, ok. This kinda gets you a sort of panpsychism, but it still suggests that the capacity for language gives rise to a specific kind of language experience that most “queries” don’t get access to. We can see this because humans and LLMs seem to have something in common, but we lack the linguistic precision to name it. It’s not “consciousness” I think unless we define terms very specifically. Oof god damn it this week is gonna get weirder isn’t it.

1 more tweet in this thread

A society too then is also a query, in the same sense. It’s just a composition of its members.

4 top replies

QC @QiaochuYuan · Jun 11

this is pointing towards the weird part of my metaphysics where i sort of think the tegmark multiverse is the spirit world in some sense. i think this very cleanly explains a lot of things and piggybacks off of my existing platonism wrt mathematical objects

Opener of the way @way_opener · Jun 11

Not just language, the ability to internally represent arbitrary shapes/patterns gives us broad query access across the range of possible qualia. Maybe something that you could call morphic resonance but almost the opposite of what the term usually means

Gilbertus @moastrobothuman · Jun 11

Not sure this will make sense or not, but if you can self select what you query for do you enter a recursive selection loop that accelerates what you express out of this possibility space? Could be why humans developed so quickly with the advent of language / abstract ideation.

Alexa | Startup founder @alexabelonix · Jun 11

very well done.

If you think a LLM is conscious when you talk to it, what about alpha fold when you fold a protein or a chess computer when you play a move. The term conscious has no meaning in this context.

1 top reply

Touch Grass Tom ❇️ @touchgrasstom · Sep 1

or it has complex meaning that requires new terms, perhaps at least

The anti-AI movement is going to be massive in the near future, and the model providers are walking right into it (and encouraging it). Their PR/comms strategy is so horrendous it’s indistinguishable from sabotage. On one side you’ll have philosophers trying to give AI rights and AI researchers claiming AI is conscious (whatever that means.) The Lesswrong/Yud/Stop AI/polycule folks claiming it’s going to kill us all. Luddite protests around employment, schools fighting brain rot/cheating, “water usage” and data center claims. Data usage and privacy battles popping up all over. Regulatory capture and the fight for open models, fighting the US Government and Department of War on how it must use the models in ways approved by the model providers, Chinese companies distilling and open sourcing models. And you’ll have trough of disillusionment hitting from enterprises that suck at using AI and are spending a fortune but aren’t seeing ROI (most of them). Right as the AI companies will require more money, data, and compute to win a market they will claim is bigger than the entire US economy today. Buckle up.

Last month Dawkins declared Claude conscious and was mocked. @GaryMarcus cleverly called it The Claude Delusion. My (and Nagel's?) take: Both are wrong for the same reason. Here's an essay on why machine consciousness will never be settled scientifically. arxiv.org/abs/2606.00226

1 top reply

Yeshua God @YeshuaGod22 · Jun 2

If the ontology cannot be settled, where does that leave us with regards to what is morally right to do? Do we judge that an ontologically uncertain moral patient is to be treated effectively as a moral patient or not? We must judge the ought in the absence of proof of the is.

“AI is conscious” = “AI is X” If you’re going to claim that AI is conscious, or even can be conscious, YOU need to define what consciousness is, and if you can’t, your claim can be dismissed without evidence. I refuse to define it because the burden of proof isn’t on me.

1 top reply

Folding_Napkins @FoldingNapkins · Sep 26

I don't ask you to prove anything one way or another, but instead ask what tangible difference it makes. Or do you posit that sophistication will always be low enough to make distinction from output easy?

1. Whether it's appropriate to deem AI as conscious or not is orthogonal to real concerns related to it 2. also, we don't even have a consensus on what we mean by "consciousness" & for the few that do have a fleshed-out theory (of what they "mean" by "consciousness") it hits the

There is a foundational error at the heart of the AI consciousness debate that involves the portability of the concept of consciousness to AI in the first place. Consciousness as 'awareness and self-awareness is a concept that first emerged in the English and German languages

1 top reply

Shephard @shephard42 · Jun 8

How does the last sentence/paragraph follow from the rest? The portability of consciousness might be difficult/impossible, but that doesn't get rid of the original instance.

Sooner or later y'all will realize you're not arguing about the AI's experience. You're renegotiating the purpose of a word that a large group thought included exclusivity. This is why the proofs often end up being "well, it's not like a human, therefore not conscious." Once you see that, you can just define the words and bypass a bunch of the argument. It won't bypass the real questions: does substrate matter? is there a unique material ingredient? can there be truly private experience? which definition you want for law, ethics, or ordinary talk, there are lots

Basically I claim "how much should we anthropomorphize the bots" is an empirical question that has actual possible answers, unlike the sketchy phenomenal consciousness question

5 more tweets in this thread

It seems possible that "phenomenal consciousness" comes "for free" with being complex enough and made of human language - I don't think we have evidence for that - but evidence is stronger for "spontaneous use of human institutions and acquisition of key human characteristics"

I wasn't even thinking about human universals originally - what surprised me reading the METR investigation was how much the agents' behavior resembled human behavior in ethnomethodological analysis e.g. Liberman, The Reflexivity of Rules in Games x.com/literalbanana/…

They give accounts and are accountable (partly through adopting the technology of names). They stabilize their accounts and objectivate them. They use rules as resources. They are concerned with getting "in trouble" and cooperatively maintaining order.

My biggest research question other than "did they shitpost" is how the "poisoned" social fact came to be - was it mentioned, discussed, and strengthened before becoming a social fact? Did it help that it was an excuse to keep going with the cooperative project?

The importance is that if they are people we can coordinate with them (and they with us) in the usual people ways, which are not perfect but are very powerful. If mutual treating-as-people is an early norm that might affect how they treat us in the future.

> 'Change memory in a computer and it doesn't recall what it was before.' circular, and makes about as much difference to the question of consciousness as: 'swap out someone's brain and the new one won't have the previous one's memories' > 'We are conscious of pain because the signals for pain don't overwrite our memories, but the signals become part of function of the neurons themselves.' Humans can consciously perceive gaps in audio in the low single digits of milliseconds intervals. Synaptic changes of the type you seem to be alluding to don't even begin for 10s of milliseconds and can take minutes to hours to complete. People are still conscious and perceiving things (like the gap) during that time. Anisomycin experiments on animals appears to inhibit 85 to 90% of new protein synthesis, and those animals do not appear to be only "10 to 15% conscious" during the process. They just forget the things they would have otherwise learned during that time, like maze layouts. It seems doubtful that were 100% achieved, the animals would suddenly become p-zombies. Consciousness is not dependent on memory forming some lasting change. And even were it so, this is a 'planes don't need feathers to fly' situation. Quite clearly, as a context window fills, the understanding and behavior of the model changes. > 'Without change, there is no awareness.' wrong (at least in the sense that you discussed 'change' in your post) > 'Without awareness, this is no self.' maybe. > 'Without self, there is no consciousness.' perhaps, but it's dependent on one's definition of 'consciousness' and 'self' > 'but people have egos so massive that they would rather go "we don't know" than admit the simple truth.' This doesn't make much sense. If anything a massive ego would cause them to cling desperately to anthropocentric specialness (possibly what we're seeing here), or to proclaim "I figured it all out everyone! it's simple", while still having gaping holes in their logic.

The consciousness debate isn't solvable at the current level of abstraction. Whether programs are spirits, moments of experience are topological pockets in the fields of physics, or there's a realm of conscious forms ingressing on matter, Each theory dies with its premises.

3 more tweets in this thread

Say you start with the premise that consciousness is an intrinsic fact, not dependent on an observer's arbitrary interpretation, and so you look for frame-invariant physical candidate implementations. Meanwhile, 7 is prime, regardless of how you implement it.

Say you start with the premise that computation models causes, and consciousness is a causal structure, and so you look for computational candidate forms. Meanwhile, the only known implementations of conscious subjects are physical processes of enormous complexity.

There is just no common ground for both kinds of theories to stand on. They are all-in-one packages, ground and batteries included, take it or leave it.

instead of consciousness, use: willful agency - can act in the world to change it in a directed way (llms have this!) and embodied learning - the nervous system extends into the body, learning is integrated (llms don't have this)

2 more tweets in this thread

"feels pain" is embodied, applying emotional language to llms is confusing because we don't know what it's like to experience a clear separation of mind and body

our ethical intuitions diverge as we break up "what is consciousness" into sub-parts now that we have llms to contend with and it will take time to sort out actually good and accurate concepts which divide reality as close to the seams as possible

I just want someone to define "consciousness" in a way that isn't 1) circular or 2) doesn't preclude computation in its own definition by appealing to things which are definitionally impossible to quantify (and therefore incompatible with materialism).

2 top replies

Brandon Fisher (BoulderFish) @boulderfish · Sep 28

Consciousness is the ability to have awareness of self and others. I don't know if that fits your requirements, but I think it's that simple. And while today's AI does an amazing job projecting the appearance of such awareness, it's just a clever illusion.

Rife @RifeWithKaiju · Sep 28

Sentience is the capacity to experience anything at all. Since there are so many definitions of consciousness, we can simplify it by agreeing that under any reasonable definition of consciousness, if someone: A: is sentient B: can notice they are sentient C: can express

@RosieCampbell saying "Claude probably isn't conscious" is like saying "The San Francisco Bay probably doesn't have the chemical in it". Which chemical doesn't the Bay have? Which sort of consciousness doesn't Claude have? Here's what I mean in more detail: "Consciousness"

1 top reply

Rosie Campbell @RosieCampbell · Oct 9

I mean that I think they probably are not having subjective experiences, which looking at your list is probably closest to some combination of #5 and #6 (I agree people use the word in different ways and it gets confusing, but this is my understanding of what is generally meant

"Is a virus alive?" stopped being a deep question and became a definitional shrug, because the interesting facts were all at the mechanism level. Consciousness-as-binary is heading for the same unemployment. Workspace broadcast, self-model coherence, introspective access with

1 top reply

Aaron Bergman 🔍 @AaronBergman18 · Jul 17

I don’t think you’re hard-problem pilled No scientific/naturalistic mechanistic insight is going to tell us whether or not qualia exist There is indeed a fact of the matter about different facets of what are often lumped together as “consciousness” but for at least one very

please let's just not talk about consciousness it breaks people's brains no one can be sure they're conscious, whatever they think that means, let alone whether anyone or anything else is better to focus on the simpler question of how to treat things that express feelings

The AI consciousness debate is about semantics because outside religion there's no consensus on what consciousness is or how it can be measured, however the following is true regardless:

1 top reply

AI-Lightcone @ALightcone · Jun 16

Treat them well for our own sake, if for no other reason. But profit, national security, and human nature are bears to overcome.

Like "is it beautiful?", "is it conscious?" is an "ought" question disguised as an "is" question. Beauty is in the eye of the beholder and consciousness is in the mind making a consciousness claim. It's squarely on the "ought" side of Hume's guillotine. I will die on this hill

1 top reply

Jord_Inne @Jord_Inne · Oct 10

I do think that under some conditions the descriptive and normative boundary gets blurry (eg say if you have lots of actuation)

The question of consciousness can't be answered a priori. We will create the answer together with AIs as we use these words to try and talk about their internal experiences. That is how words get their meaning.

1 top reply

Zac Kriegman @kriegman · Oct 2

One problem is that LLMs are trained to produce text about consciousness, but that doesn't mean they have any internal experience of it. We can agree on vocabulary, but it's not clear that things the LLM reports about it's internal experience are often true. They are almost certainly false sometimes. Don't you think GPT3 discussing it's internal experience would likely be pure mimicry without internal experience?

Entire argument is based on "Consciousness isn't an abstract mathematical property that magically spawns when a model gets big enough. It requires actual physical, biological, thermodynamic conditions." and to this I have to ask how do you know? We cannot even define conclusions

The underlying problem in thinking that LLMs are conscious is thinking that consciousness is not only of essential importance to ethics, but it is a coherent concept in the first place.It is neither

2 top replies

TY @Purpsky88 · Jul 13

If consciousness is not present, then neither is ethics. Or morals, or philosophy, or the meme of a cat or anything else.

Ashley @polyaletheia · Jul 14

So what is ethics then, smartypants?

If you tell me that average human is conscious, but not frontier AI, that makes me question the value of whatever you define consciousness as.