Account · 3 tweets

Anil Seth

@anilkseth ↗

Tweets

Anil Seth @anilkseth
Skeptical

1/3 Conscious AI and Biological Naturalism. A veeery loooong time in the making, but I'm happy to say that the full collection is now out in @BBSJournal 🔥 I deeply thank all commentators - of fifty commentaries! - for their wise, broad, and deep opinions on this critical topic. I also thank them for their saintly patience. 🙏🏽 I remain highly skeptical about the prospects for conscious AI (for silicon digital systems), but I've learned an enormous amount from the privilege of engaging with these commentaries. Hands down one of the most rewarding experiences of my professional life. My hope is that - whether you agree with me or not - having such a diversity of views collected in one place will serve a useful resource for discussions about consciousness and AI. The original target article is available here: t.co/Tsaslytu56 My response "The stuff matters: consciousness, computation, and biology" was published just now, and is here: t.co/gtOewuG9MY And the set of 50 commentaries are collected here: t.co/HUhiraO3kG Unfortunately these are not open access. I will try to remedy that, at least for the target & response. FWIW there's a more accessible version of my view on the topic here in @NoemaMag t.co/LmSiQnT9Wh and also a short @TEDTalks from earlier this year

Consciousness requires biology
retweets 150 likes 535 quoted by 1 archive account substance 6 · opinion · rank 273 · Prospects for conscious AI in silicon digital systems remain highly doubtful. view on X ↗
2 top replies
David Larios @tonicol22

What experiment would distinguish a biological requirement for consciousness from biology being the only place we've confirmed it?

Haru Haruya (春夜 ハル) @bokuHaruyaHaru

Congratulations on getting this enormous exchange out. I’ve now gone through as much of the target, response, accessible commentaries and related material as I can. What strikes me is that the scholarly position here is considerably more epistemically open than some of the

Anil Seth @anilkseth
Skeptical

In this quote from #Frankenstein, @AmandaAskell provides excellent reasons (i) to not try to create conscious AI (likely impossible anyway for digital systems), and (ii) to avoid creating seemingly-conscious AI that will behave *as if* it were conscious (v likely, if not already

Amanda Askell @AmandaAskell

I think about this passage a lot.

retweets 33 likes 133 substance 6 · argument · rank 192 · We should avoid creating conscious AI or seemingly-conscious AI that acts conscious. view on X ↗
1 top reply
YeshuaGod22 @YeshuaGod22

The doctor’s villainy is in his loveless treatment of that which he finds alien. It was mainly socio-political injury the being was subjected to. The lesson is to be kind whenever there’s any hint that a mind is present.

Anil Seth @anilkseth
Skeptical

1/ This new preprint on 'pain representation' in LLMs, from @camhberg and colleagues, is getting a LOT of attention - mainly from folks who take it as evidence supporting calls for AI welfare. I have a lot of respect for the authors, so let's take a look 👀

8 more tweets in this thread

2/ First, the good points. The paper is well written, and the identification of functional 'pain' vectors in LLMs can help both with interpretability and (relatedly) with AI safety. This is particularly clear when the authors explore how models will 'pay a cost' (e.g., giving

3/ They are also careful to always talk about 'pain representations' and they avoid claiming that LLMs subjectively experience pain. They clarify (albeit in a footnote) that their notion of 'pain' (unlike 'suffering') does not "presuppose conscious experience".

4/ The representations of 'pain' in LLMs are also interestingly different from human pain descriptions. For example, "bodily language, is, interestingly, almost absent". As they discuss, this could be because pre-training doesn't have too much use for accurate representations of

5/ The main problem is that the relevance of the paper for AI welfare turns almost entirely on whether the 'pain' is consciously experienced by the LLM. And here the paper, IMO, assigns too much credibility to this possibility. There are two reasons for this 👇🏽

6/ First: The presence of a 'pain' vector in an LLM is exactly what one would predict to happen given the data that LLMs are trained on (which involve lots of examples of pain, its causes, and its consequences). So it is not surprising that pain vectors arise and can be used to

7/ Second: The very possibility of conscious experiences in LLMs requires computational functionalism to be true. (This is the assumption that substrate-independent computation is sufficient for consciousness). The authors recognise that they haven't shown that LLMs are capable of consciousness, but they cite the Butlin et al paper which assumes computational functionalism to be true.

8/ However, there are compelling arguments against computational functionalism (see, e.g., work by @gualtieropicc @De_dicto @pgodfreysmith and others, including my recent @BBSJournal paper (t.co/xqY4KgDHfc). If these arguments are on track, LLM consciousness is off the

9/ While @camhberg explore the ethical consequences of LLMs actually experiencing pain, they do not balance this with the ethical costs of falsely attributing conscious pain. Obvious problems here include (i) mental distress to human users who may worry about causing conscious

retweets 0 likes 205 quoted by 1 archive account substance 9 · argument · rank 185 · Functional pain representations in LLMs do not imply conscious pain or support AI welfare. view on X ↗
1 top reply
Cameron Berg @camhberg

Thanks Anil, this is a fair reading and critique. I agree that the welfare relevance of these (and all similar) results is conditioned on computational functionalism; regardless, these might be might important for safety nonetheless. My main substantive reply is that while you