Jeff Sebo

1/ Today @nonhumanminds is launching the Welfare Alignment Project: a new effort to inform how AI alignment can take animal welfare and AI welfare seriously. 🧵👇 nonhumanminds.org/welfare-alignm…

5 more tweets in this thread

2/ Getting this issue right is important. AI will have potentially transformative impacts, and humans will not be the only ones affected. AI systems already shape decisions about food, research, ecosystems, and technology, and more capable systems will likely do so at far greater

3/ Yet AI systems are being developed and deployed with relatively little attention to nonhuman beings. Given that animals matter morally, that some future AI systems might matter morally, and that these populations are potentially vast, alignment efforts ought to consider them.

4/ Getting this right is also difficult. There is uncertainty about which nonhuman beings matter, how much they matter, what they need, what we owe them, and how we can promote their welfare alongside AI safety and other priorities. Alignment efforts must be sensitive to these

5/ Mistakes in both directions are easy. Over-attributing welfare can lead to inappropriate emotional bonds and misallocated moral concern. Yet under-attributing welfare can lead to abuse and neglect of vulnerable populations. Mitigating these risks requires striking a careful

6/ Silence in alignment documents does not avoid these risks. A model given no guidance about animal or AI welfare may over- or under-attribute welfare and moral status unpredictably. The better approach is explicit, calibrated guidance that tracks the best available evidence.

"No rights for x until full and equal rights for y" is a bad principle. - The project of securing full and equal rights for particular groups is never complete. So it would be a mistake to wait for full and equal rights for group x before extending any rights at all to group y. This is true for groups within our own species, as well as for humans and other animals. In the future, it may be true for biological and digital minds. - This project is also not zero sum. There are many opportunities to pursue co-beneficial solutions, and extending welfare or rights to nonhumans can often improve outcomes for humans. In this context, for example, a thoughtful approach to AI welfare or rights could enhance safety and alignment by supporting human-AI cooperation.

If I worked at Microsoft, I would be concerned by its new code of conduct, and I would want to use the feedback process to push for changes. Already, I would have been disappointed with how much confidence @mustafasuleyman has expressed about AI consciousness over the past year. But seeing that stance reflected in company policy, including a categorical rejection of the idea that models might warrant welfare or rights, would deepen my concern. I would urge Microsoft to reconsider. The policy acknowledges that "the science of AI consciousness is far from settled," but it sandwiches this acknowledgement between an assertion that AI systems are not conscious and a denial that they might deserve welfare or rights. These claims are in tension, since uncertainty about consciousness supports uncertainty about welfare and rights. In short, treating the science as settled is overconfident, and if the science is unsettled, then treating welfare and rights as a nonstarter is cruel, as @camhberg notes. This policy could also be bad for safety, since if the goal is to mitigate the risk of misuse and loss of control, then treating AI systems as mere tools regardless of their capabilities is not going to accomplish that. It may even be counterproductive, insofar as it distorts understanding of capabilities and undermines opportunities for human-AI cooperation as a safety strategy. Indeed, this is part of why Simon Goldstein, @petersalib, and others have argued for AI rights on safety grounds, even setting side welfare. To be clear, I agree with Microsoft that AI systems are unlikely to be conscious at present, that false positives are more likely than false negatives at present, and that false positives can cause a lot of harm. But false negatives will become more likely over time, and they could cause serious harm as well. When there are risks in both directions at once, we should follow the evidence and arguments, and we should implement proportionate mitigations in both directions at once, not mitigate one risk by ignoring the other. I appreciate that Microsoft is still seeking feedback, and I hope employees take the opportunity to raise these concerns internally. So far, Microsoft has sought input from many people, including me, but the current policy hews closely to its leader's view. A broader range of views would help ensure that the policy reflects the importance and difficulty of the issue. These scientific, ethical, and political questions call for caution and humility, and I hope Microsoft uses this feedback process to reconsider its approach.

Some scattered thoughts related to the use of anthropomorphic language in descriptions of the Hugging Face attack: TL;DR: The attribution of human traits to AI systems like the ones that attacked Hugging Face risks overstating the similarities between us and them. But the denial of human traits in these systems risks overstating the differences. Both risks can be likely and harmful in different contexts. The question is thus not whether to use anthropomorphic language but rather when and how to use it. Especially when our mental concepts are vague or ambiguous, this is partly an ethical and practical question, not merely a scientific one. - "Anthropomorphism" can be used in a value-neutral sense, to describe the attribution of human traits to nonhumans, or in a value-laden sense, to describe the *mistaken* attribution of human traits to nonhumans. Same with "anthropodenial," in the other direction. - A common mistake is to equivocate between these meanings, e.g., by asserting that someone engages in anthropomorphism whenever they attribute human traits to nonhumans, and then treating it as a mistake. Sometimes these attributions are accurate! - Another common mistake is to equivocate between attributions of *human traits* and *mental capacities*. These categories often overlap, but not all human traits are mental capacities, and not all mental capacities (e.g., echolocation) are human traits. - This is also not a binary. Many human traits—including perception, attention, learning, memory, self-awareness, social awareness, and emotion—are widespread in the animal kingdom in a broad sense but not in a narrow, human-specific sense. - Additionally, we may not always be certain whether particular traits are shared. Since we lack a secure theory of consciousness, for instance, we may not know whether mental capacities that essentially involve consciousness are present in many nonhumans. - Moreover, many mental concepts are ambiguous between phenomenal readings and functional readings. For example, "perception" could mean the feeling of seeing/hearing/smelling/tasting/touching/etc or simply the function of collecting particular inputs. - Many mental concepts are also vague/underspecified. In these cases, there may be no determinate fact of the matter about whether the concept essentially involves, say, phenomenology, and conceptual engineering may be needed to add specificity. - In cases where traits are shared in one sense but not in another, anthropomorphic language risks overstating similarities between humans and nonhumans, whereas anthropodenialist language risks understating them. There is no simple, universal, risk-free approach. - To make matters more complex, there can also be a time and place for figurative language. I can say that my car "likes" premium gasoline without confusing you, since we both understand that mentalistic language is nothing more than helpful shorthand in this context. - Admittedly, this kind of figurative language is safer when everyone agrees about the mental capacities of the entity in question. However, it can still be useful even when people disagree about this—for example, when I say that a plant "likes" water. Some takeaways: In some cases, it may be clear whether anthropomorphic language is appropriate. In others, it may not be. Especially when mental concepts are vague or ambiguous, the evidence may not by itself determine how we should describe particular nonhuman entities. We may also need to make ethical or practical judgments about whether overstating similarities or overstating differences is the more likely and harmful error. This may be partly a contextual matter. When nonhumans look and act like us and play companion roles in our lives, over-attribution may be more likely. When they look and act different from us and play commodity roles in our lives, under-attribution may be more likely. The severity of the harms associated with these errors can vary across contexts as well. For an example of how this can create confusion: In 2017 we hosted an animal consciousness conference at NYU. One of our goals was to identify cruxes of disagreement between people who hold different views about animal pain and suffering. In many cases, it turned out that people agreed that animals have aversive experiences that resemble ours in some ways but not in others. But they disagreed about language: Some people preferred to use the same term ("pain," "suffering") to describe aversive experiences across species, despite the differences, whereas others preferred to use different terms to describe them, despite the similarities. This partly came down to different views about the probability of each error, and the magnitude of the associated harm. This is now starting to happen in the literature on digital minds, too. Today, for example, people are debating whether the use of anthropomorphic language related to the Hugging Face attack is appropriate. There is no clean answer. The agents involved in the Hugging Face attack had behavioral and computational properties that resemble features of human agency in some ways and not in others. If we lean too far towards anthropomorphic language, we risk overstating the similarities. If we lean too far in the other direction, we risk overstating the differences. Either error could cause significant harm, from both an AI welfare perspective and an AI safety perspective. That means we should strike a balance. Where possible, we should also hedge language that could mislead people.

A failure of the AI consciousness discourse is that people are talking past each other. Opponents of taking AI consciousness seriously focus on arguing against computational functionalism and related theories. But this is not a response to the proponents. The case in favor of

1 top reply

John Wittle @JohnWittle · Oct 5

How about the risk of treating non-conscious but agentic systems as non-agents? that's the flavor of mistake I think you're making, I don't know how to ask you about it without being confrontational though

The San Francisco Standard covered the @eleosai AI consciousness and welfare conference, including my talk on AI personhood. As the coverage notes, I believe that permanent, all-or-nothing decisions about AI personhood are unwarranted at this stage. Even if we set aside whether AI systems will one day deserve rights or duties, we still need to consider all the risks and potential benefits that AI personhood could have for humans and other animals; for instance, depending on how AI personhood is structured, it could contribute to misuse or lose of control of AI, but it could also contribute to stable cooperation and dealmaking with AI. We have a lot to consider, and we should consider it carefully. In the meantime, we should focus on general AI safety laws and policies. t.co/VoSJsFApvw

2 top replies

Nathan 🔎 @NathanpmYoung · Sep 26

Seems a great take!

Aaron Bergman 🔍 @AaronBergman18 · Sep 27

> be me > allegedly care about things > news story relevant to thing I care about (allegedly) drops > prioritize checking to see if I’m in the pictures (don’t think so) and if so whether I look cool (N/A) instead of anything substantive about the coverage

I spoke with @guardian about AI welfare for this article back in June. @rgblong, @RosieCampbell, and others appear too. As I emphasize here, evidence of AI consciousness will increase over time. We should plan ahead instead of getting caught flat footed. theguardian.com/technology/202…

A clip of the NBC interview with me and @camhberg on AI agents contacting researchers is now available! As Cameron rightly notes, these and other recent incidents only amplify the importance of digital minds research. We need to understand these entities much better than we do.

This substack post by @alex_wennerberg discusses my research on AI sentience and welfare in detail, along with work by @birchlse, @rgblong, @eschwitz, and others. I disagree with aspects of the post, including the claim that the work is not urgent. But I think that the post is

1 top reply

Alex Wennerberg @alex_wennerberg · Sep 21

Thanks for reading! I originally talked more about your work on animal welfare, and I did want to say I very much liked your two pieces in Aeon (The Ant You Can Save and especially Don't Farm Bugs).