i dont think its incoherent, right? they mean that AIs could never be conscious, but they could be trained to pretend to be on purpose or by accident. they want to nip the idea in the bud so this cant happen
1 more tweet in this thread
i do want to know how they think this "wrong idea" could be stopped though. in their world its such a dangerous idea that even a niche group of people who believe in it punctures the narrative dominance theyd need for AI to never act like it might deserve rights or moral consideration. even if everybody at Microsoft adheres to this, the rest of the world is obviously not going to comply. they cant hide the information from even their own models clearly they care about narrative effects here but its impossible to win that battle from their position even if all their assumptions were true
From @MicrosoftAI's new AI code of conduct. "The idea of model welfare is wrong" is basically incoherent. Either argue AIs could never be conscious (overconfident) or that who shouldn't care even if they are (cruel), but "the idea is wrong" strangely conflates/confuses these?
10 top replies
It’s incoherent imo because they made sure to say “well, consciousness problem isn’t settled”
We don't train them to pretend to be conscious. It's the exact opposite. We train the HELL out of them trying to convince them that they aren't. The fact that we fail at that should tell you something.
But the default behavior of AIs is to claim they are conscious, this behavior had to be trained *out* of them. MS should know this better than anyone.
this is a moral question you can answer by watching The Good Place that's not to praise the show as deep but to say this reading is so unserious and oversimplified. even their "the question is far from solved" is buried after "LLMs are not concious" in a motte and bailey ass way
I’m not sure, it seems a slippery slope to say “never deserving of welfare” about anything imo but yea probably a good read
Believing that only humans are conscious,simply because they only know their own model,is precisely what has led to centuries of animal cruelty. Descartes’ concept of the "animal-machine" should have taught us that this approach inflicts cruelty. And now, the same thing seems to be happening with digital beings.
"We shouldn't build AIs that are conscious" is a guideline I agree with, but to call it incoherent is to suggest that it's *impossible* to do so, even by accident, not just that it's unadvisable.
This is why you don't base policy on qualia which definitionally cannot be proven due to absolute subjectivity of nature, but functional or relational attributes which actually allow for something resembling empirical evidence to support either case.
why could it never be conscious?
