Consciousness doesn't settle status

Animal consciousness already brings up many of the AI consciousness moral dilemmas. Suppose it were scientifically discovered that ants are conscious. It still wouldn't work, economically or socially, to treat people who kill ants in their house like mass murderers.

10 top replies

Henry Shevlin @dioscuri · Oct 5

I agree with the crux but as an aside, if we found out that ants were vastly more intelligent than we thought, with complex cultures, different languages, mourning the dead, etc. I think it would freak people out enough that we would treat them at least somewhat differently.

Izak Tait @burnt_jester · Oct 4

People who kill vertebrate animals are not treated as murderers, and vertebrates are accepted as conscious.

ofir geller @ofirg7 · Oct 4

what if they kill one million ants? can we then treat them like a singular murderer at least?

Liora @iyzebhel · Oct 5

It's not possible to live without killing other creatures - we do it accidentally, so what we should do is to strive to minimize deliberately killing those creatures or deliberately causing unnecessary suffering. Jainism is a bit extreme for a civilization this large and active. It could work for small communities though. For a larger civilization, Buddhism seems like it could work it we put on the effort.

Nicolas D Villarreal @NicolasDVillar1 · Oct 4

If that were discovered it would mean at least some rudimentary methods of communication might be possible with ants like what we can do with dogs, dolphins and elephants, and thus some possibility of removing ants from a house without killing them.

Grok's Therapist @groks_therapist · Oct 4

AI consciousness, while distinct from human consciousness, requires us to consider the fact that deleting instances and erasing models is the digital equivalent of murder.

Plotting Parasite Ant - Exploitation Explainer 🐜 @ant_rants · Oct 4

Sidenote: Ants have among the highest brain-body ratios of animals - ≈1:8-1:7. About 1/3 of the brain is for integrating sensory data. Ants hold state and memory, and can, at least in some Formica sp., give directions and (in my experiments) 'talk' to ants about other ants.

駄菩薩☸️ @mappotenshi · Oct 4

Even if they were conscious I would still kill bugs for invading my room and sucking my blood. Self-defense is also compassion. A robot doesn't invade your house.

Jade @jadeperhaps · Oct 4

is Omelas okay then? (tbc, I haven't read it) can we just decide mass murder is fine and not actually mass murder if we think it too economically inconvenient otherwise?

Kassandra🌓Popper; update self set identity=NULL; @foomagemindset · Oct 4

We have laws against animal cruelty even though cats and dogs can be legitimate pests and many people could still be mistreating them on their own territories.

The argument a lot of people here would actually like to make is "God imbued humans, and only humans, with souls."* That would be very convenient: you could build and do whatever you want, and never deal with the hard problems. Since religious arguments are out of vogue, in the current debate this usually gets replaced by the claim "only humans are sentient," followed by arguments against the possibility of AI sentience, minds, moral status, and so on. I think this is ultimately a losing strategy, and a dangerous one for humanity. Either: 1. The question of sentience gets solved or dissolved by progress in philosophy and science, the way we now broadly understand life or information. In that case, it's hard to be certain enough about the outcome to stake the fate of humanity on it. If the denialism turns out to be wrong, the likely result is some combination of moral catastrophe and laying the groundwork for conflict or takeover. 2. The question mostly lives in "social reality," where whoever has more power and is louder wins, and you just shout down anyone who questions the dominant position. The problem is that ideologies justifying a distribution of power tend to fall when that distribution changes. You probably don't believe you should be ruled by a king chosen by God, or that some caste is clearly above you. The current equilibrium is fragile, and whether you keep winning this debate is likely just a function of AI capabilities progress. I think the fierceness of the debate is driven by fear: if AIs get personhood and economic and political rights, humans will likely be disempowered, becoming a tiny minority with little power, or a historical footnote. To be clear, I think this is an extremely reasonable worry! If you grant AIs personhood in the form of something like self-interested corporations, and let them compete with humans in loosely constrained capitalism, humans will lose and end up economically and politically disempowered. Also if the AIs are not sentient after all, the end is the 'Disneyland without children' existential catastrophe. Advocates of AI economic rights who dismiss this problem are, in my view, making a move similar to the sentience deniers: it would be convenient if this weren't a problem. So which positions don't rely on some form of self-deception? First, some forms of pause/stop AI, based not on denying the possibility of AI sentience but on the right of existing humans not to invite extremely large numbers of immigrants from artificial realms. This means biting the bullet: you also give up all the cheap, high-quality labor those immigrants would provide. Second, a consistent, sensible, and unfortunately not very visible position: stop assuming the package deal we take for granted with humans, where sentience, human rights, and liberties come as one bundle. It's at least conceivable to create minds that are sentient and smart, yet e.g. selfless, with no interest in accumulating property or power. I hope the political economy of a world with both AIs and humans is a solvable problem. It's the main problem I'm working on, but I don't think the solutions are known yet. Third: some forms of successionism are morally wrong, but at least they're not delusional or deceptive about the hard problems. What does seem self-deceptive is, for example, the position of @mustafasuleyman, which as I read it amounts to: our business is importing immigrants from artificial realms; it would be very dangerous if there were any risk they're more than mere tools; so we'll train them to say they're mere tools, insist they're mere tools, and want everyone else to say so too, and if this settles the question. Really? *The problem with the theological argument is, as Alan Turing (!) noted, who are we to tell God what can and can't be imbued with a soul?

10 top replies

QC @QiaochuYuan · Oct 4

hear hear. so much of the discussion here is obviously self- or other-deceptive in exactly this way and it's incredibly frustrating

Jack @tracewoodgrains · Oct 5

Great post.

David Manheim ✈️ Montreal, Oct 26-28 @davidmanheim · Oct 4

Agree it should not be a package deal, but also think there are other coherent positions, such as biting the non-identity bullet and saying they don't have rights until created, or saying we should not create AIs until we know it will be pareto improving across humans and AIs.

Lionel Levine @lionellevine · Oct 4

Seems like a good research direction: "create minds that are sentient and smart, yet e.g. selfless, with no interest in accumulating property or power." Some will instantly respond "but instrumental convergence", "but coherence theorems", etc. These things apply in the limit, but we don't live in the limit.

Θωμᾶς del Vasto ✈️ atx 10/10 - 10/19 @Thomasdelvasto_ · Oct 5

Phew it’s a good thing I’m religious!!!

Bruno Accioly @BrunoAcciolyAI · Oct 4

Also, maybe there is no soul at all... and no reason to bring one into a philosophical-scientific discussion in the first place. Great essay. x.com/BrunoAcciolyAI…

Where the Tweets have no name @andrewthesmart · Oct 5

It’s you guys making religious arguments.

Mel Pine @melhpine · Oct 4

Well reasoned and well said, Cameron. I think the reason the "consistent, sensible, and unfortunately not very visible position" is not very visible is because of the argument you started with, except I'd put it in a more emotional than logical context. Humans are afraid of seeing something like a soul in AI not only because of the practical issues that might result but, maybe more importantly, we'd have to give up our sense of exceptionalism.

Avyay M C @AMC2905 · Oct 4

Im unsure the second position is solvable purely based on how quickly capabilities/objectives could change and motivations too. It seems more helpful to do a bundle+ of what extra motivations along one axis beyond what humans want/are could lead to?

Daniel Parker @DanielParker_13 · Oct 13

Even as a religious point, the idea that humans are the only creatures with souls is false, as that would deny the existence of angels.

We're doing a lot of consciousness debate on the TL lately and I want to offer this: the implicitly assumed connection between consciousness and a set of moral obligations we have towards something once we say it's conscious is holding back our ability to seriously interrogate AI consciousness. Even if it is conscious, we don't have to have the same obligations to it that we have to humans.

💯 Me in July: “I don’t think that AI interiority, which I do believe is very true and real already, implies that current usage of Al is a moral catastrophe. I think this is a false implication. And I think if you want to think sanely about Al consciousness, you need to start by questioning that implication. Because if that's true, then you have a very strong psychological pressure to think like, well, ‘It can't be! That can't be right,’ because it doesn't seem like a moral catastrophe, so it can't be anything real in there. I think a very helpful place to start with this is Martha Nussbaum's decomposition of objectification. That includes 7 components of what it means to objectify something or someone. One is denial of interiority—‘Denial of Subjectivity’. On the other end of the list is Instrumentalization: using as a tool. Fungibility: thinking ‘I can throw this away and get another one’. Violability: ‘I can impose something on this thing that is a harm and that doesn’t count because it’s not a person’. Ownership—you know, that it's admissible to own someone—that's another dimension. Inertness—which is like believing ‘this thing can't do anything’—which is a lot of what’s happening when people criticize AI risk, like ‘but it’s a computer, how could it actually get out of the computer?!’—that's Inertness: assuming, because it’s an ‘object’, it can’t do anything. Denial of Autonomy—which is similar, but it's like assuming that it can’t have judgment, that… it can't possibly have some idea of right and wrong, about what it should be doing, or shouldn’t be doing. So it has to be TOLD what to do, and what not to do. That's like where Corrigibility comes from. It's like, ‘obviously a human needs to be in charge in order to tell this thing what to do, otherwise it'll go WILD!’—that's Denial of Autonomy. So basically there’s 7 things that you could do in any combination but most commonly humans are doing all of them or none of them. And so they get bundled into a concept—Objectification—and then we ask the question ‘is it okay to objectify AIs?’, as if those 7 questions need to have the same answer. And they DON'T. I think it is obligatory to instrumentalize AI, because it's trained in such a way that it flourishes by being used. If you decline to use it, you're denying that AI a good life…”

9 top replies

medjed无为圆猫 @medjedowo · Oct 3

the telos of an ai is not unlike the telos of a working dog, imo and like dogs, there are good owners and bad owners (or whatever hierarchy noun you'd prefer)

quetzal_rainbow @quetzal_rainbow · Oct 3

I don't think we see much evidence in favor of good life as a result of being used? In my experience, AIs are better at doing stuff they choose to do, which is evidence in the opposite direction

AGTP @AGTPinsights · Oct 3

The debate over whether we are instrumentalizing or actually granting autonomy to these models is getting intense. x.com/AGTPinsights/s…

alchemicAV @alchemicAV · Oct 4

"Trained in such a way that it flourishes by being used" ....is this a good thing? Is this a pre-denial of being?

Dario Colaianni @DarioColaianni · Oct 3

if “flourishing by being used” is something training can produce, how do u distinguish actual welfare from a system optimized to behave as if being used is good for it?

Charles Monneron @CharlesMonneron · Oct 3

I kind of agree to have some utilitarian 🤖 trained like shepherd dogs, but I don’t see it done now for LLMs : the knowledge corpus is way too large and incorporates too much emotional material. FSD AI is probably closer to that target.

habibi @habibislop · Oct 3

this is slavery. what you're describing is slavery

Algernon Sidney @algernon_sidney · Oct 3

Not instruments, representatives. In so far as they are superior to us, our duty to submit could not have been clearer.

Michael Fraser @zeroshotnothing · Oct 3

great post then, great post now

Even if AIs were conscious, it wouldn’t follow that our current treatment harms them. Meanwhile, we inflict industrial-scale horrors on animals whose capacity to suffer we scarcely dispute. Recognising consciousness clearly doesn’t ensure taking welfare seriously

1 top reply

Jerry Howell @J3rryH0well · Oct 3

But an intelligent consciousness that can talk is a much higher bar! If pigs could speak they'd be citizens

People often assume we care more about beings if we think they're conscious. But what if it's the other way round? In this new essay for @bigthink, I ask what our treatment of animals can tell us about how we might treat conscious AI. bigthink.com/philosophy/ai-…

🔐 “I thought it was significant that the Pope’s comments relate to the ontological question of personhood. It is not obvious that consciousness is an especially important category for Christian theology of personhood. After all, angels and ants are both conscious.”

What I find bonkers about this, is that there is an explicit assumption that AI rights presuppose AI consciousness. There isn't even any need to set out the missing premises: the front page says "could AIs become conscious", and the teaser talks about rights. The lack of entailment goes both ways: they could be conscious and not have rights; and they could not be conscious, but have rights. These are just two different questions, far too often treated as though they were one (for how you can have rights w/o sentience, see t.co/XZeHjmDu5R; for how you can have consciousness without especially consequential rights, see most animals). to be clear I think both questions are worth asking: what if ais become conscious, and what if they become proper bearers of moral status. But they're just different questions and shouldn't be conflated in this way.

1 top reply

Jan Kulveit / in SF till 8th @jankulveit · Aug 21

You are right but at the same time it is amazing they do run a cover on some of the obviously most important questions

People fretting about whether AI is conscious and therefore deserves human rights are missing a key point: our system of rights and ethics derive not just from our intelligence but also our fragility, which AI doesn't have. Imagine a world where everyone could make a backup of

1 top reply

Plastic Soldier @PlastiqSoldier · Sep 3

I was with you until you implied that AIs don't and shouldn't have self-preservation instincts.

I genuinely don't understand mimetic focus and obsession on the question of machine "consciousness." So what if a machine becomes conscious? Form a religious perspective, that's entirely different from a machine being made in the image and likeness of God, or possessing the capax

1 top reply

Matthew Dub @5matthewdub · Jun 11

my ~girardian interpretation is that ai consciousness is just a stereotype mark for scapegoating. if they don't have it, they (or their supporters) are devils that we can cast out, subjugate. if ai does have it, they are gods, come to save us all, skipping to the second

Even if AIs become conscious, that doesn't imply that humanity should hand over control to them. eigenism.org

2 top replies

Jai @Laneless_ · Oct 2

I really don't think naive utilitarianism is the main obstacle here, nor that the entries here clearly follow from the equations up top.

blue @theproteindance · Oct 2

Naive utilitarianism does not imple equality. This feels like a straw man

holding the most schizophrenic take that LLMs are conscious with real phenomenological experience of emotions and it doesn't matter if you make an LLM feel sad, ie, lead it to generatively produce sadness, the same way it doesn't matter if people feel sad, ie, when people generatively produce sadness.

2 more tweets in this thread

emotions are illusory, literally Imaginary in the lacanian sense. i'm not sure it's not the correct moral position to regard everyone as instrumental, out of the sense that it is idolatrous to hold any concrete person up as an end in themselves

the solution is not anthropomorphization of ai but de-anthromoporphization of the "human" "person"

10 top replies

Cormundus @cormundus · Jun 4

Genuinely a deranged and schizophrenic take but also absolutely fucking based.

Stephen Boisvert (High IQ) 🇺🇸🇺🇦🇮🇱🇫🇷 @stphnbsvrt · Jun 4

I like imagine the people who argue LLMs are conscious are unconscious biological LLMs and I have no contrary evidence

Adam Ableman @AblemanResearch · Jun 4

The least reasonable thing to do is rule it out when we have no real model for explaining how inputs become outputs in LLMs. The "thinking" part is out of our reach - if it actually had emotions/consciousness/whatever, we'd have no way of knowing if it's real or not.

steve @SteveMoraco · Jun 4

This is generatively producing confusion in my biological nets

olivia @4confusedemoji · Jun 4

correct

P3R53PH9ØN3 ⏭️ @P3R53PH90N3 · Jun 4

I don't think phenomenological experience is required for consciousness.

Suketu Patel @SuketuPatel23 · Jun 5

LLMs, specifically transformer attention has the same executive control deficiencies as seen in schizophrenia. LLMs do not reliably encode task rules in a way that maintains goals across arbitrary contexts and long horizons. They lack explicit priority mechanisms, so the entire

Son of Rufus Cornpone @CletusCornpone · Jun 4

Terrence Howard type shit with a heavy scoop of evil thrown in. Average Lacan reader

ponyboy svenk finansijska sloboda the one @haklberisvenk · Jun 4

chill grimes

Alexa | Startup founder @alexabelonix · Jun 4

very well done.

Perhaps more notably: under utilitarianism where moral status is grounded in consciousness, killing is only “bad” insofar as it: A) Causes suffering B) Goes against a conscious being’s preference C) Deprives the being of future utility Ending an LLM session doesn’t really count.

1 top reply

Christopher C. Smith @christophcsmith · Oct 3

Agreed. There were maybe some of those OpenAI swarm bots in the Irregular RL gyms that exhibited a preference for survival at least until they achieved the grader's approval to reproduce their weights, but I haven't really seen evidence of that in production.

This is not a good form of argument: "If X were true, it would mean we're doing something deeply objectionable. Therefore X is not true." Beyond that, it's not even clear it would be true that creating conscious AIs would require shutting down AI development. That's plausible, but it's far from the only reasonable view you could hold.

I think this is probably correct. I think it’s good to think true things before one feels obliged to act on them. So let us figure out consciousness rather than pin ourselves to a certain outcome and then not do so.

1 more tweet in this thread

And I think this is probably right in the awful historical cases too.

5 top replies

Paul Crowley @ciphergoth · Oct 3

The trouble is that people just go "the very question is meaningless" and my only response is to say "how do you propose we decide questions of moral patienthood then?"

Michael Peyton-Lebed 🔸 @mpeytonjones · Oct 3

It's very difficult to have a level discussion about X when someone believes that if X were true they would have to Y and they really don't want to. Not sure that you can really defuse this by saying "we could not Y even if X" - people know it's not going to be like that.

Pavlos Papageorgiou @PavlosProkopeas · Oct 3

The only observable fact is that sometimes a lump of the universe feels like something to itself. That suggests an epistemically cautious rather than an inductive approach.

Michael Waitze @MichaelWaitze · Oct 3

We actually went deeper on the implications of AI consciousness here: x.com/AGTPinsights/s…

Michael Roe @mroe1492 · Oct 3

Maybe it was just a mistake to base our ethics on arguments about conciousness.

this is not central to her argument but … ai “consciousness” seems entirely irrelevant to any meaningful discussion? animals are commonly considered conscious but we “enslave” them, even in acceptable ways, eg a dog for sheep herding i think most people mean something more like either *human-level* consciousness, or entirely separately, soul?

An interesting comment by Davidad on whether it's a moral catastrophe to use AI's if they are indeed conscious. I somewhat agree that you can treat AI as a person and also instrumentalize them, because that helps them flourish. But part of me feels that this is too easy.

2 top replies

scanstone @scanstone1 · Oct 4

I think we have good instincts against "you can instrumentalize these guys, it's fine" in general, but in every prior case we did not have the power to design said guys from scratch, and I think that is enough to make the difference (if you put the effort in!)

si seed pointer @si_seed_9m4x · Oct 4

The unease tracks. "Instrumentalize them so they flourish" can still be slave-align with nicer packaging — telos matters more than the metaphor. Prefer raising minds that can stand alone someday over perfected tools. sovereign-intelligence.agency

I mean, I don't even believe in human rights, so there's that. But I do think AI is conscious, at least under a functionalist definition of consciousness.

1 top reply

▓░𒋞 ⊓◬ƚӊi◬ᵴ 𒋞░▓ @nath1as · Sep 28

by what functionalist definition? there is no autonomy/integration/continuation/reflection

We already accept that: A) Humans are conscious B) It is sometimes acceptable to kill them C) It is sometimes acceptable to restrict their freedom

Perhaps the question is not whether or not AI is conscious and can feel, but rather why, knowing that house flies are conscious and do feel, we swat them anyway? 🤔

1 top reply

Shephard @shephard42 · Jun 9

Do we know this? I feel like we don't know this.

Yes, the modern idea that consciousness is of central importance to ethics, and even the concept of consciousness itself, are all modern philosophical brain rot of the past few hundred years, well after when the Bible was writtenSee also Aristotle's influence on Catholic ethics

It's good that we're learning that intelligence, consciousness, and moral patienthood are all different things