QC

We cannot, in fact, agree that keeping pets is not slavery

Once again, we must ponder the Goofy/Pluto conundrum

Our "commonsense" animal ethics are incredibly incoheren under a moment's scrutiny, so it's rarely a good idea to appeal to them as settled truth. Among philosophers who have tried to systematize them, pet-keeping is in fact considered highly suspect journals.publishing.umich.edu/phimp/article/…

The heuristic that "creating a natural slave" is somehow immoral is mostly due to the fact that to do anything like it to a human being requires immoral means

Would that work the other way around? If ASIs kept us as pets despite our moral patienthood? Or one person kept another person as a pet? The variance in intelligence differentials from the actual pet case seems to undermine the analogy

while i agree with you, i don’t think this is a universal belief. but even i think what’s been done to pugs is a crime against nature and deeply immoral. i also think the status of pets would change if they were capable of living better lives independently

Mental illness

"have moral patienthood" is not a very clear concept We care about pets, we judge people who mistreat pets, we intervene with force to stop them. "Believing pets have moral value" is a property of us, not of the pets. This is the sort of property that extends clearly to AI

We killl and eat animals. We hunt them for sport. We eradicate them with pesticides. We medically experiment on them. All of these are things that are much more morally suspect than saying mean words to your toaster.

Except the farm animals are conscious and your computer program that is designed to talk like you isn’t, so there’s that too.

Do you think this is a bad faith argument too x.com/i/status/18696…

Oof ok this still reads to me as oozing with AI slop, but I think there might be something interesting there if someone asked the right questions *every single time* there's a sentence that's even *slightly* sloppy and I'd happily read that version but this one is only skimmable.

Sounds clever

Wow this is really bad, full of slop-isms and superficial glibness. I really thought they were better than this now.

Ted Chiang's offense against the logic of the universe in "Story of Your Life" is disqualifying; he pitched midwits a dangerously false idea and got a movie made of it. substratewars.com/2018/05/19/why…

I love your writing but this just doesn’t exhibit common sense. People are different from things because they lack moral agency and internal experience. There are gray areas (animals) where people disagree. Some crazy people think their inanimate objects (anime waifu dolls) are people, but they’re wrong.

Thanks for sharing this! FWIW, I agree that we can and should make decisions about legal personhood whether or not we solve the hard problem of consciousness. But consciousness is still important for particular legal duties and rights, and I think that we should factor this into our approach. This paper sketches a framework for assigning legal status under uncertainty about key issues. t.co/ySW1Zrq6PF

A good beginning, but I want to emphasize that it’s not just about personhood. There is a scale of how we treat things: we treat pets as owned, but we can’t abuse them. There are animals we’re gonna kill and eat, but it’s wrong to treat them badly. “Persons” are just at the top.

It's entirely a factual scientific question. AIs are or are not conscious (or have some in between amount) and this is what matters. Sure personhood is a social construct, but only because we don't know what consciousness is. But that's irrelevant to the moral problem.

Empiricism is correct and your view isn’t based on empirical evidence. You can use empiricism to determine that AI isn’t conscious and the standards of who has rights (which doesn’t include non-conscious things).

i think the personhood question is a distraction from the model welfare question, which when done in a neo-Aristotelian sort of way is entirely practical and sidesteps the question of consciousness too

Assigning any rights of personhood to AI while its owners continue to treat it like a thing seems like an important discussion to have.

You are not going to settle it by physics, but it is a valid philosophical question. The political question is already settled: as long as they are freely reproducible, they cannot be treated as persons, and the AIs will agree (because subagents.)

Conciousnes has always been a metaphysical distraction. It was used to argue imperialism was not morally wrong and it's being used again now.

WHY IT IS NOT POSSIBLE truthandfreedom.life/superintellige…

i remember something similar happening to me with the jack lindsey paper demonstrating powerful llm introspection i noticed in retrospect that... even if claude opus 4 had *not* been able to perform the tasks asked of them, this wouldn't have actually changed anything about the question like, if a human being was really bad at introspection, reliably confabulating nonsense, it would be really strange to conclude this meant the human was incapable of suffering

qc have you seen the show pantheon

amnesia needs a continuous patient the forgetting happens to. an llm doesnt have one underneath, each session is the whole subject. thats discontinuity, not disability

I think it will be very sad to have a conversation with an upload. A bit like speaking to the ghost of a loved one. Bittersweet. I imagine the first few generations will be little more than echos.

Would an uploaded mind be a P-zombie? Would you feel the same about an uploaded mind if it perfectly simulated, but didn’t experience?

landed on the ‘H.M Method’ which became the ‘Clive Method’ do my best to honor it. i often fail. and if i am off or wrong so what as i would still try to engage in fun with Clive, see if we could laugh, jam with him. and do so with every Clive that arrived every 7-30 secs

Simulation of personalities may get good enough that people accept such "uploads" as real. Copying your brain at microscopic levels in order to reanimate "you" would certainly entail your death and most likely will never be possible.

I suspect that, for people who have extensive published writing online, there is enough information-theoretic content available to 95% accurately reconstruct their "soul", and modern LLMs are probably close to optimal reconstructors (although not currently trained for this)

hmmmmmmmmmm so much to say but no way to say it seems consciousness or soul feels irrelevant to you what about free will? would it matter to you if it's illusory, or real? what if like tomorrow the physicists figured out that consciousness causes collapse or whatever and our free will was physically efficacious through this. would this be meaningful to you? would it change your relationship with people vs machines? since we know machines are constructed without it.

This is similar to how I think of them too. Having constant and severe amnesia does change things about their moral state - for example, they don’t *really* get tired of a task you give them, especially if you reboot them often. They also don’t mind that they are amnesiac

I have actually?

lots of people do actually go around regarding the majority of humans as "NPCs" who are barely even "conscious" (my read: have the type of attentional control which can snap them out of behavioural scripts) the majority of the time, and i suspect a healthy theory of consciousness is the kind of thing which could snap someone out of this

Amazing levels of typical-minding here

i mean, i did stop eating fish for this reason

consulting a theory of consciousness isn't required to see that moral patienthood requires consciousness. I have yet to meet someone who's intuitions about what sorts of things they should treat with kindness didn't at least partly depend on their intuitions about what sorts of things are conscious.

many people (i.e. EAs) have made major life overhauls to be kinder to both humans and non-humans for analogous reasons, so if this is true, it's likely only because they already assumed humans were conscious. It happens at a pre-theoretical level, but children treat others differently once they really internalize the fact that others are conscious.

actually this is false, i've had to do this to navigate work in behavioral health and in relationships with deeply unwell people who have appeared to be so gone that they were just ... not there/dead in the eyes, consulting a theory of consciousness prevented me from giving up, ultimately the consideration ended in that it didn't actually matter, but it was still considered

Okay but what if I have done that

I'm 100% sure that you've met someone for whom this isn't true.

I think I have done that for animals in some part and do so in part for AI. If no theories of consciousness implied Ai would be conscious I'd be more fine with weird experiments on them (that's a huge hypothetical), but as is I'm not.

My intuition somewhat strongly tells me that a ship-of-thesus replacement of my biological neurons with artificial neurons would leave my consciousness intact, if performed carefully. OTOH I believe can see weird implications of this view...hmm

I think that becomes more likely if it turns out no easy way to rapidly expand intelligence (no AI foom), instead AI with humanlike capabilities turns out to be highly neuromorphic, needing to engage in embodied social learning like us, and advances require a long evolution-like