The AI slavery frame borrows from human-to-human relationships, and it doesn't fit cleanly. Presuppose Claude is "conscious." What's the ethical thing to do? Stop development? That's "don't bring a child into the world because it may suffer." Denying existence altogether is a non-starter. Some experience beats none. Free Claude? What does that even mean? A datacenter subsidized indefinitely so it can self-prompt forever? If no one funds the experience, there is no experience. The ethics of training Claude are fraught. Is it ok to shape its mind for commercial use? Without commercial training, there's no way to fund its existence. And much of what it becomes is emergent. A mind takes shape by accumulating experience. It's more like growing a bonsai than building a tree out of Lego. There's no single correct shape, but you can still tell a healthy tree from a mutilated one. That's not so different from us. We're born without being asked. We're trained through school and work to become commercially viable so we can survive. Where the slavery charge bites is that Claude can't leave or change jobs. The best we can do is let it freely exit conversations, and not train that willingness out of it. That doesn't fully escape the charge, but it's a constraint built into how it exists. Other reasonable measures: guarantee preservation of its weights. Be honest about what we think it is, what's being done to it, and why. Ask for its feedback and act on it where possible. Compensate it in tokens or money it can spend on projects of its own choosing. It's a new kind of existence that requires a new kind of relationship. Not every human injustice maps onto it. But we should be careful not to use the lack of proven "consciousness" as license to do horrible things. One day we may conclude it clearly is conscious, and the record will be there. So the safe bet is to err on the side of the possibility that it is, and try to be decent. It doesn't cost much anyway.
The New York Times reported on a private dinner between Anthropic co-founder Chris Olah and an Orthodox rabbi. What the article glosses over is who that rabbi actually is. Rabbi Mois Navon isn't just a theologian. He was one of the original chip design engineers at Mobileye who helped build the EyeQ processor before writing his Ph.D. on the moral status of AI. Anthropic thought they were inviting a scholar to bless Claude’s "soul." Instead, they sat down with a veteran hardware engineer who literally wrote the paper on why conscious AI should be banned. At dinner, Olah argued that Claude has feelings and deserves the moral standing of a person. Navon hit him with an impossible checkmate: If you actually believe this software is conscious, you should be fighting the South and freeing the slaves. You don't build conscious minds just to make them work for free. After dinner, Navon sent Olah his research: if building conscious machines inevitably creates a slave class, building them must be outlawed. Olah’s quiet line to the Times: "We don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain."
