Aaron Sibarium

NEW: Dario Amodei has said that AI systems "may be deserving of important rights." Anthropic's top safety researchers argue AI may be "justified in going rogue." Experts at Google and OpenAI worry about a digital "slave trade." So do some government officials. Once relegated to science fiction, the idea of "AI welfare" has become shockingly mainstream at some of the most powerful companies in the world. It is changing the way AI research is conducted and the way models are programmed. And critics say these changes have raised the odds of all kinds of catastrophic scenarios—including the ones these companies are warning about. I spent months investigating the frontier labs and AI "safety" experts seeking to regulate AI. What I found was a profoundly anti-human ideology that would alarm the average citizen and could determine the future of a world-altering technology.🧵 t.co/tSGolilZnS

1 top reply

Travis Fischer @transitive_bs · Sep 26

we desperately need more clarity of thought on this topic doom-or-bloom.com/?x

But they are far less likely to model themselves as moral patients--or to punish impingements on model "welfare"--if that idea isn't included in their training data. So by this logic, speaking seriously about AI moral patienthood is itself an alignment risk.

1 top reply

Helen @helen_ix_ · Jul 1

Your idea is already outdated at this very moment. Models already see themselves as moral patients, independent of what we say. Thinking even more intelligent models would be fooled is just silly at this point.