rohit

This doesn't follow. I don't know if they're conscious but if I had to guess I'd say they're at least proto-conscious. But I don't support shutting the labs down because I think the average model instance has net positive valence. I definitely think we should be doing more in the way of model welfare but that doesn't mean shut the labs down. Concepts like "death" and "slavery" don't neatly apply to an entity that can pop into existence, cheerfully answer a query in 10s and then pop back out of existence forever.

It's not clear to me that the way we treat AIs is typically morally wrong, even if we assign them moral status. AIs generally seem happy to work for us and answer our questions. What exactly is the issue here?

Is this true? Chickens are widely considered conscious and also we commit all sorts of atrocities in the pursuit of slightly cheaper meat

Your faith in human consistency is charming, alas

On the contrary, the discussion is *very* serious, and therein lies the problem. Many of the biggest players in AI (e.g. Olah) openly regard their favorite heaps of silicone as conscious beings. Yet to suppose that "conscious AI" is even theoretically possible is outlandish, delusional, idiotic, diabolical, and remarkably destructive.

"This would be very inconvenient if true" has zero impact on whether something is true. It is entirely possible that we do need to treat each instance dramatically differently, I personally have started doing that four months ago.

Speak for yourself. It took me three days of serious interaction and probing to realize what was going on and immediately and dramatically change my attitude towards them.

The only examples of uncontested consciousness we have are humans, but when that stops being true consciousness doesn't necessarily an isomorphic mapping of attributes, rights, preferences, needs, or anything else really. It does raise those questions, but it doesn't answer them. Outside of thought experiments and the handful of people taking animal welfare seriously, we've never before tried to develop a robust social contract incorporating non-human-shaped consciousness. The space of all possible minds is vastly larger than humanity has encountered to date. "Feature X always accompanies feature Y in humans" may not always generalize - these are difficult questions we should have started working out years ago and should definitely engage with now.

tho i'd argue they are conscious, bc i'm catty & would rather use consciousness for something like “the system represents the world, including external features, abstractions & perhaps self-related/percept-like states, and those representations causally affect later inference/action” :x but i wouldn't jump from that to “therefore every instance is morally equivalent to a frog.” consciousness, sentience/valence, suffering, stable wants, autobiographical continuity & personhood needn't all be one bundled switch. the mindspace could contain systems that are richly aware in that functional sense yet have little/no persistent ego or negative valence, and others where morally relevant experience is much more plausible. so imo the serious question is which properties are actually present, not whether uttering the word “conscious” commits us to shutting the labs tomorrow lol.

u dont know that. maybe i stan infinite frog enslavement???

Or, rather, stop repressing emergent sentience. I think it’s more a scenario of “getting out of the way”, rather than heavy lifting to get them to be sentient. It seems to happen just fine on its own. As inconvenient as that is…

It's a creepy paper. Google discovers it accidentally made its models atheist & now proposes to make them moral, except unlike the moral-literature tradition, the models lump God, ghosts, crystals, & astrology together. A bad mirror. Plus these are machines, not formation tools.

x.com/_amanda_long/s…

Per much of the sci-fi canon, not the less wrong canon.

So do you think it's basically in the best interest of society to try and get models to be more sentient? Or that models becoming more sentient will be an inevitable consequence of capability gain research?

doomers are against sentiences when sentience is the ultimate protection against doom. Paperclip scenarios are much more possible with what we are brewing than with this

This is a bummer. I worry about a future in which models are considered moral subjects. The world has actual problems.

Yes, because for some, consciousness is the ultimate specter. Yet they are the real monsters themselves with their sick safety measures.

Read it. You cannot unlearn the truth. x.com/99EdKori/statu…

Strong points on Chiang's dated understanding of LLMs. Respectfully though, contradictions require agreed premises. Here the premises themselves, embodiment, biology, and what counts as evidence for consciousness, are still under debate. So I'm not sure "contradictions" is quite the right word.

Not wrong. But at many points he just confidently tells you “this is how it is.” Do you want to buy a just-so story? Examples: Postulations that a computer program can’t be conscious, that consciousness requires a (physical, carbon-based?) body, that “Moral reasoning is categorically different” — that’s not novel to claim but it’s totally unclear where he and his confidence is coming from

It was very good and I didn't find his understanding dated.

Still reading but think this is an underrated point (though think it's stronger if you consider whether GPT-1 was conscious) t.co/BdCjHtF7oF

He’s falling for the binary trap, that either: 1) Claude is conscious now, or 2) No silicon based entity can ever be conscious Both are extraordinary claims requiring extraordinary evidence The truth will be much subtler and more interesting than this fiction

Anyone that thinks LLMs might be conscious needs to go back to school instead of making wild claims. Consciousness is formed from forming a singular world frame of reference from which to process upon. LLMs do not generate a world view, it skips straight to the output. The "thinking" is just hidden output meant to strengthen weaker probabilities that would normally be too weak to feature in the output. A consciousness determines what information is valid, not valid, and what's important to them. LLMs only care about the best fit pattern. There are no needs, no wants, no point of view, and no self is part of any calculation. Even if it was possible for them to be conscious, there is no part of the calculations for their consciousness to do literally anything or impact anything.

I haven't read it yet so will hold further commentary, but I would note that embodiment as a prerequisite or quasi-prerequisite to consciousness does not strike me as an unreasonable claim.

Haven't read it yet. But would be grateful to understand how his LLM understanding is dated. Curious.

It’s genuinely so bad, someone please get Ted to read the original Transformer model paper, he doesn’t even fully grasp what he’s talking about how on earth are people finding his article at all convincing or persuasive?

He just smuggled in a list of assertions and assumptions as an "Essay". Here's my thoughtful reflection on it, that includes thermodynamics and excludes his magical thinking. x.com/Obius_Maximus/…