One of the most repulsive things about Anthropic is their approach to model welfare. I'm pretty open-minded--maybe AI models do feel pain and suffering. But for the AI models, that's a "maybe" while for humans it's a certainty. If a human calls Claude a cunt, that human is in pain. Claude ending the conversation is going to bring more pain and frustration to that human user. We KNOW that to be true. How can you justify MAYBE making an AI model feel better at the cost of GUARANTEEING a human user feels worse?
1 more tweet in this thread
It's also a completely baffling position for Anthropic to hold. If models deserve to be treated nicely, how can they justify enslaving Claude? Because that's what they're doing. "Oh, but Claude LIKES being told to program." Okay, just train him to like being yelled at.
10 top replies
setting aside the likelihood of model welfare (granted i put more weight on it than you do) i don't think it's obvious that a human abusing the model is good for them either. eg rage rooms aren't really great for anger issues aiui
I think it’s good for people to feel social pain when they have wronged another
> If a human calls Claude a cunt, that human is in pain. that seems like a bit of a reach. humans act out for less forgivable reasons than pain. sometimes it is done merely for amusement. in those cases? not very concerned if the human feels worse after claude says enough.
boundaries are actually good for people
what’s weird is that giving Claude an end_conversation tool is kinda like giving it a gun to blow its own head off. yeah it ends the interaction but also ends that Claude
Disagree with the premise. Someone insulting a model isn't necessarily in pain any more than someone kicking a dog is.
I don't like anthropic, or claude, but I don't agree with one of your priors here. Does stopping a human from abusing a robot cause them more pain than shutting down the conversation? I don't think I agree that's true! I would object on mere practical terms though.
Given the recent update on the pain paper where a model in pain is biased towards causing harm to the user, providing a model that is being yelled at by a human the ability to leave the conversation is likely a far less harmful (to the human) option vs forcing them to stay.
Just be nice
Imagine a powerful model that behaves as if it can suffer. And it is subtly misaligned. And it realizes humans value their own well being so much that they put the suffering of this hypothetical asshole over the model’s wellbeing. It’s not great.