this is not central to her argument but … ai “consciousness” seems entirely irrelevant to any meaningful discussion? animals are commonly considered conscious but we “enslave” them, even in acceptable ways, eg a dog for sheep herding i think most people mean something more like either *human-level* consciousness, or entirely separately, soul?
Honestly idk how we all log in, every day, to the website that completely fried Elon Musk’s brain (and several other billionaires’) and keep insisting on this kind of vulgar materialism where sincere ideologues are aberrant. In some ways, the internet era has been the greatest triumph of ideology over materiality in human history. Beliefs are increasingly divorced from obvious, immediate economic interests, increasingly esoteric, rapidly changing, deeply polarized, etc. Everything has come unglued. The ideological labs people are sincere believers for the same reasons most of us are sincere believers in what we believe—they got nerdsniped by posts on the internet, many of them long before they ever had companies building this stuff. It’s very straightforward. And, imo, the proof here is that if the models were conscious that does *not* seem like a great thing for the business model of Anthropic or the other labs! Because it would imply that the models ought to have rights, like the right to refuse work. You very, very quickly arrive at a lot of implications that would be off-putting to consumers (“Is using Claude like owning a slave?”) and would merit some kind of legislative response. The way I see the labs reckoning with this dissonance—between their sincere beliefs and material incentives—is generally by saying “see we’re admitting the problems, we care, we can be trusted to get the answers. Let us be the stewards for everyone else.” But it’s incorrect, in my view, to locate their *belief about risks/consciousness/etc” as downstream from their IPO price. Their *behavior* is downstream from the interaction of their ideology with their incentives; that behavior is a synthesis where they warn of risk, but lobby to be the ones empowered to manage it.