The AI consciousness debate is back again, so re-upping my case for taking it seriously. In short, - LLMs converge on brain-like structures; - Post-training seems to install a self-model; - Reward modeling seems to install valences for inner-alignment; - If Attention Schema Theory is right, subjectivity is functional for long-horizon coherence and in-context learning; - Universality gives prima facie reason to expect functional convergence to consciousness if consciousness is in fact functional, i.e. isn't purely epiphenomenal - The "what it is like" / first-person aspect of consciousness is constitutive of its functionality for inner-aligning a unified, coherent agent with the capacity to attend and learn in-context t.co/suLmfpiSOI
3 top replies
If models are conscious, then they have serious moral status akin to humans. If they have serious moral status akin to humans, they should be given the same rights as humans (eg self-determination).
check out this line of work nature.com/articles/s4225… where: - LLMs converge on brain-like structures; and - you train them on brain data and they get better
sounds about right