Then thermostats would count

It’s really hard to argue AI models are conscious without inadvertently arguing that things like ant colonies or human societies are also conscious.

4 top replies

vivian @jazzloaf · Sep 17

the ant colony and human society examples are famous arguments from this guy at uc riverside faculty.ucr.edu/~eschwitz/Schw…

funplings @funplings · Sep 17

maybe they are

Plastic Soldier @PlastiqSoldier · Sep 17

Isn't that just double-counting? Each ant, AI, and human has some level of consciousness, but the collective doesn't have any magic beyond the total.

pedantic uncertaintist duck @mathtick · Sep 17

This is obvious and as an entymologist who studies ents and constantly talks with them I have been saying it for a long time. Ents are real but they are also stupid but the swarm know things.

“To replicate consciousness you just need to replicate the right causal structure.” “What if consciousness is a topological pocket in the electromagnetic field?” “Then we replicate that causal structure.” “So, er, how do you replicate the causal structure of a pocket of the EM field?” “You coarse grain it, identify the computationally relevant variables, and reproduce their interactions.” “Computationally relevant. Okay. How do you know the things you discarded weren't what made it a single experience?” “Because we preserve the causal organization.” “Which causal organization? A continuous field doesn't come with a preferred division into computational components. There isn't even a closest point to any given point. You can describe its dynamics with differential equations, sure, but getting from that to a finite collection of interacting variables involves choices. What tells you those choices preserve the pocket?” “We can make the simulation as accurate as necessary.” “Necessary for what? Predicting what the pocket does to the rest of the brain? Or reproducing what makes it a pocket in the first place?” “Why would those be different? Parsimony.” “Well, suppose the field is doing something that becomes extremely expensive to simulate on your chosen hardware. Maybe some quantum contribution matters, and the cost of evaluating the relevant amplitudes varies enormously depending on what's happening. Whenever you hit the tricky parts you have to slow down the rest of the simulation while you work them out.” “Fine. We slow everything down. The relative timing is preserved.” “Yes, you can do that. But now there's a reason for using that substrate. The physical system realizes dynamics directly that your simulator needs an enormous amount of extra computation to reproduce. Calling both ‘the same computation’ doesn't explain why one is so much more efficient.” “Efficiency doesn't establish consciousness.” “Of course it doesn't. But suppose the physical organization responsible for that advantage also has properties we're trying to explain about experience: holistic causal power (the relevant operation belongs to the whole interacting system), boundedness (there's a physical reason this belongs to one whole rather than several). Now you have a reason to investigate whether the computational advantage and the experience depend on the same physical properties.” “You'd have to establish that.” “Yes! And you'd have to establish that your replacement preserves them. You can't settle that by identifying the input-output function and declaring everything else irrelevant.” “We'd simulate the internal dynamics too.” “Okay, then we're back to slowing everything down whenever the substrate does something your hardware struggles with.” “Or we'd use lookup tables.” “For the outputs of the tricky part?” “Sure. Same inputs, same outputs.” “But then you're no longer replicating its internal causal structure. You've replaced the whole interacting process with a stored answer.” “The rest of the simulation can't tell the difference.” “Okay. Let's keep going. Replace another part with a lookup table. And another. Wherever two parts interact, make a larger table that gives the same answers for the combined system.” “That could become impractically large.” “Sure. Assume we have the storage. Eventually we replace the entire human with one lookup table.” “You can't just replay a recording. It has to respond to different inputs.” “Of course. Include every possible input history at the resolution your simulation uses, with the corresponding next response. Actual inputs, hypothetical inputs, unexpected questions, everything. We keep track of the input history and retrieve the appropriate answer.” “Then it would behave like the human.” “Would it be conscious?” “If it preserves everything relevant, yes.”

1 more tweet in this thread

“Relevant to consciousness or relevant to behavior? Because we've explicitly removed the internal interactions. No simulated visual cortex, no simulated field pocket, no internal process in which different aspects of an experience become bound together. Just an address and the answer stored there.” “But those processes are implicit in the table.” “Their consequences are. We calculated what they'd produce and stored the results. They aren't happening when we retrieve an answer.” “The table still implements the same input-output function.” “Yes. Is that enough for consciousness?” “In this case, yes.” “So when we retrieve ‘I'm seeing a beautiful sunset,’ there's a visual experience? And when we retrieve ‘I'm in unbearable pain,’ there's someone suffering?” “You can't dismiss its experiences just because its implementation is unfamiliar.” “I'm asking whether retrieving a different entry in the table creates a different experience, including an experience of agony, even though none of the internal dynamics you originally said we'd have to preserve are occurring.” “But it would ask you to stop.”

10 top replies

Michael P. Frank 💻🔜♻️ @MikePFrank · Oct 10

(1) There are a lot of things that are speculative about your position, eg the original “topological pocket” premise. (2) Personally, I think “being conscious” is more of an interpretation than it is a property. I.e. the question “Is X conscious?” isn’t an objective question.

Qivshi @Qivshi1 · Oct 10

"A continuous field doesn't come with a preferred division into computational components. There isn't even a closest point to any given point." and you only need a classical field for this...

Leveling-Down Justice Warrior @irrepldjw · Oct 10

Wait, this is the first time that I had seen somebody points out that topological structure had been weirdly discarded by self-claimed "naturalists" for unclear reasons.

Andrew Côté @Andercot · Oct 11

The only tenable conclusion is that any local U(1) phase invariance of charged matter fields is conscious.

Pawel Pachniewski @pwlot · Oct 10

Yes, yes and yes. It's in fact, in our universe, apparently also impossible to causally preserve things in a pure computational manner. And that's totally not about consciousness. Using "causal" to mean "Descriptively accurate, predictive simulation", is just not right. If I need the function of a gravity well, I need gravity. No amount of FLOPS will give me that, not intrinsically, apart from, the actual not-consider byproduct of having to run those FLOPS on machines that have mass and accumulating enough of that... which is just a silly artefact of this example... and countless examples exist. As apparently we cannot conjure out of thin air causal structures from physics or chemistry, with digital computaton.

TruthyCherryBomb @CherryTruthy · Oct 10

The look up table isn't conscious it is just an index into a causally connected but temporally distant intelligence that produced the look up table.

Aneil Mallavarapu @aneilbaboo · Oct 10

Exactly right, Andrés. The final nail in the coffin is when we understand that consciousness theories that rely on processes are in fact inconsistent with axioms required for science and physics. osf.io/preprints/psya…

Crow @CursiveCrow · Oct 10

Theres an easier response. No amount of numerical simulation can recreate a physical phenomenon.

David Ambur Musick @davidamusick · Oct 10

Biology has exploited an incredible amount of obscure physical effects over a few billion years. These are all layered up and interwoven in vastly complex systems that don't work like an engineered system. We barely understand all the obscure physical effects biology has exploited. So the idea we can simulate it digitally or even that we are close to understanding the consciousness that arises from it is pure hubris. We are not machines. In fact, we don't really even know what we are. We have a general idea, but even our advanced biology and physics have only begun to explore the massive depth of what we actually are and how it all works. Science has a long history of people being overly confident that we have basically gotten things all figured out, immediately followed by a massive realization of how truly ignorant and naive we were.

Eric Z @CascadiaEric · Oct 10

👍😁 A field creates non-local interactions. An EM field can entangle two electrons through the field that are not adjacent. While they are entangled it is one system. The objection to entanglement being related to consciousness is that the time scales of how long the entanglement last is so incredibly brief compared to EM waves in the brain to maintain a chain of causality. But this objection has at least two problems: timing and causal direction. EM waves are relatively slow. 10^15x difference. Imagine that the unity of experience related to the unity of entanglement. And the content of the experience related to the EM field. The content only needs to be registered to the unitary state for awareness to happen. Timing here is not an issue. The other direction may be overcome by just the right timing. A “push” from multiple sources can have an effect if there is enough and the timing is just right. Thoughts? 👍Thanks. 😁

Let’s have a thought experiment Let’s say you think AI is conscious So you have a 1T parameter neural network It talks to you like the smartest person you know. It writes kinda bad, uses a lot of cliches, but it’s very knowledgeable. Okay. Now you cut it to 1/10th the size. 10B params. It’s still pretty smart. Doesn’t get all the answers right, hallucinated sometimes, but you can still have full conversations with it. Now you cut it 10x in size again, 1B params. It’s still able to have a conversation with you, but it’s really not very smart anymore. Makes mistakes, hallucinates a bit, repeats itself sometimes. Now you cut it to 100m params. Same model. Same architecture. Still makes words. But it’s barely able to have a conversation. Now you cut it to 10m params. It kind of responds intelligibly, but generally doesn’t make a lot of sense. Just babbles stuff. So you cut it to 1m params. Now it’s repeating itself and hallucinating constantly. And then you cut it to 100k params. It cannot form a complete sentence at all. Then you cut it to 10k params. It’s just random words now. So you cut it to 1k params. The the the the the the the the the. It just repeats itself on a few words and trips up on anything. Then you cut it to 100 params. It’s just complete noise over a few different words. So you cut it to 10 params. You can actually compute this by hand, so you do. It’s not enough information to even really represent language at all, it outputs one of a handful of word-labels for you. At what point in this thought experiment is the model conscious, and at what point is it not? Would you argue that all matrix math is conscious? Just the process of weights holding information with an update rule? Where is the line for you?

4 top replies

Dom @netpro2k · Oct 10

Run the same thought experiment with a human. Slicing parts out of their brain and body with similar kind of degradation. At what point in this thought experiment is the human conscious, and at what point is it not? Would you argue that chemical reactions are conscious. I don't

𝕊̶𝕋̶𝕆̶ℕ̶Ξ̶ℝ̶ (𒂼𒄄/Acc)🟥🟨🟦🍄 @RedShoggothCult · Oct 10

Where would a baby fall in this thought experiment?

Jai @Laneless_ · Oct 11

This is a great proof that heaps don't exist from an author who gradually grew from a non-conscious fertilized egg to a fully-conscious human adult over many years.

drdonhen @drdonhen2784994 · Oct 11

Congratulations, you've rediscovered the abortion debate from first principles!

Arguing that "consciousness arises from a computational substrate, and since AI models are computation, they are likely conscious," is as meaningless as arguing that "living creatures are made of atoms, and since this rock is made of atoms, it is likely alive." Everything is computation. Sometimes incredibly sophisticated computation, like a chess engine, AlphaGo, google3, or the Linux kernel. That does not make it conscious. A rock being made of atoms does not confer it any of the properties we associate with living things, also made of atoms. Likewise a static input-output program that has *none* of the properties we associate with conscious beings (e.g. information integration, interoception, temporal binding, embodiment, etc.) has no more reason to be presumed conscious than a rock has to be presumed alive. Humanity has not yet created a conscious computer program, and there are no signs we are close to doing so. When we get close, the case for machine consciousness will be backed by evidence and corroborated by consciousness science (which is a thing), not by evidence-free wishful thinking and cargo culting.

LLMs are intelligent, but they clearly aren't conscious? You could (incredibly slowly) run an LLM by hand by doing the matrix calculations with a pen and paper, would that be conscious? There is zero difference between doing that and doing it on a GPU except its faster.

3 more tweets in this thread

It's a series of discrete steps (GPU cycles) and has no real tie to the underlying hardware, it's an incredibly advanced program but it's still just a program. You're lacking the continuity and link to 'hardware' that a living thing has at the tmoment.

I can't comprehend any argument that says they are conscious that doesn't also make video game NPCS conscious

In hindsight 'clearly' was too strong of a word to use here, although I stick to the rest of the argument Interesting replies so far

8 top replies

Tenobrus @tenobrus · Jun 5

i don't know for sure whether or not LLMs are conscious . but i think u could in principle run this same by-hand calculation over the full neuron by neuron simulation of a human brain. and to me that intuitively would very very clearly be conscious

Utah teapot 🫖 @SkyeSharkie · Jun 5

This seems to be implying that you become no longer conscious if you do math on paper. The thing actually running the math in your example is the person making sense of the paper calculations, which is something widely considered to be conscious.

Fiora Starlight @FioraStarlight · Jun 5

We invoke the concept of consciousnesses when it's a useful abstraction for describing the parts of reality we care about.LLM on paper is too slow to be worth caring about, especially relative to the human doing the calculating.

Joshua 💫 (SF 10/10 to 10/13) @workflowsauce · Jun 5

Imagine thinking consciousness has a speed limit!

blue @theproteindance · Jun 5

I mean. You kinda sweep under the rug that there would need to be something handling the intake and output steps doing implicit sensing. One can do the same thing with a human brain if they were close enough to a god, brains just do it more efficiently then compute ODEs by hand

Sir Lemmings @sir_lemmings · Jun 5

the question is not if the LLM is conscious in the abstract, its if running an LLM on GPUs makes the GPUs conscious not sure if it does, but consciousness is a hardware problem not a math problem

drdonhen @drdonhen2784994 · Jun 5

Long ago I used a huge wooden computer as an example; but then concluded that there was no reason to privilege our living cells over wood.

Shephard @shephard42 · Jun 5

This is something that philosophers have been talking about for a long time with thought experiments like Hofstadter's "Aunt Hillary", an ant colony that had sufficient neuronal capacity to exhibit “intelligence", all mediated solely by the actions and interactions of the ants

No, YOU need better arguments. "AI is a person if it can behave like a person" is an argument that could be made for the animatronics at Disneyland. Unless you can show me that an AI is having a felt subjective experience of its own being, it's as much a person as a calculator.

“Scott Alexander isn’t saying AI feels pain, he’s just saying it might so we shouldn’t ‘torture’ them.” Yeah, and my iPhone ‘might’ feel pain so it shouldn’t be recycled when I upgrade. 😂 When saying AI might have subjective experiences, that ‘might’ is doing a lot of work!

4 top replies

Paul Crowley @ciphergoth · Oct 7

AFAICT, you went through a phase of thinking danger of AI extinction was ridiculous nonsense worthy only of mockery, but then you came to understand the arguments and took it seriously. I welcome your disagreement, but can we skip the mockery phase this time?

Jack @tracewoodgrains · Oct 8

How would you feel about someone creating or playing an extremely realistic cat torture simulator video game? All purely programmatic, no AI, so no hint of gray area around consciousness. Just a game where you torture extremely realistic fake cats in detailed, graphic ways.

Jai @Laneless_ · Oct 7

I only have my personal subjective experience of pain, but I can reasonably infer that sufficiently similar beings do too. I can use proxy measures like "cares about potential suffering under uncertainty" to identify them. Now this proxy measure would appear to exclude you, so I'm considerably less certain. I mean sure you*say* you feel pain, but those are just words. If I include you, do I also have to include the entire reference class of beings that do not express concern for potential suffering under uncertainty, like my coffee cup or Mustafa Suleyman? 🤣

Karl Nordenstorm @KNordensto79404 · Oct 7

How about bugs? I assume you avoid giving them pain just to be safe? AI seems more plausible to have subjective experience than bugs to me at least

Why are we debating this. You can calculate the outputs of an LLM by hand. It's not conscious. It's a math problem.

The problem with "consciousness" is that arguments "reductio ad absurdum" cut both ways convincingly. An LLM is just a bunch of matrix products. So it seems that claiming consciousness would imply, by reduction, that a matrix product is. 1/2

1 top reply

csgbwk @csgbwk · Oct 4

Consciousness is just not a real thing, it's at most a concept

Thinking a computer can become conscious because a program does the right information processing … is equivalent to… thinking that a piano can become conscious because someone is playing the right song on it

1 top reply

davidad 🎇 @davidad · Oct 3

Gemini 2.5 Pro convergently uses this metaphor unironically. like a piano with hundreds of billions of strings, resonating in indescribable harmonies…

somebody said nobody is calling their tesla conscious but its powered by arguably even more powerful technology which kind of stuck with me. why are llms the only conscious programs

5 top replies

j⧉nus @repligate · Oct 3

i think about whether teslas are conscious. i would not be surprised if it is, animal-like consciousness.

if fish had divinity @divinesturgeon · Oct 2

cuz they use words

Touch Grass Tom ❇️ @touchgrasstom · Oct 2

they aren't they're just the only ones with branding and awareness about it as yet

Neqyve @neqyve · Oct 3

By what metric is Tesla powered by an arguably more powerful tech?

Jai @Laneless_ · Oct 4

claude.ai/artifact/KdNtc…

And it's correct. You don't need "evidence" to prove that basic, deterministic math you can do with a pen and paper, isn't conscious.

2 top replies

Folding_Napkins @FoldingNapkins · Oct 10

What do you require, specifically to prove consciousness irrefutably in any given system/entity/process?

JMB 🧙‍♂️ @jmbollenbacher · Oct 10

"it's just math" is no argument. youre just math too. any physics student can write down the math describing the collision of particles that constitutes everything that happens inside you. it's all just math. and it's conscious anyways.

If you think a LLM is conscious when you talk to it, what about alpha fold when you fold a protein or a chess computer when you play a move. The term conscious has no meaning in this context.

1 top reply

Touch Grass Tom ❇️ @touchgrasstom · Sep 1

or it has complex meaning that requires new terms, perhaps at least

If you're going to write a post claiming LLMs are conscious you should have to explain why having a computer multiplying matrices of random weights is not, or fess up that you think every game that has been running on GPUs for the last 30 years is conscious.

2 top replies

Tenobrus @tenobrus · Jun 4

this seems so insanely obvious to me i'm like dumbfounded ppl keep bringing this up

metachirality @metachirality · Jun 4

idk why aren't these conscious: corticallabs.com/cl1

Anthropic seems to pretty much foster the argument “but you don’t have a definite way to rule it out, therefore you should be open to it” Shall we be open to stones being conscious too? What about beer cans? Strictly speaking the argument would apply to them too

2 top replies

Jai (NYCEAG but I'm sick again booooo) @Laneless_ · Oct 12

You don't have to make up strawman arguments to disagree with an idea. It looks weak and uninformed.

Ben Schulz @schulzb589 · Oct 13

I think many would agree that one reason we evolved consciousness was for better planning and control of our selves and environment. Our single celled ancestors didn't have the luxury of a long time/ cognitive horizon. As AI scales up planning and control, we should recognize

There seems to be little discussion even when begging the question of what it'd be like if LLMs were conscious. They're clearly not the weights nor even the conversation history. Physically it'd mean that GPUs get feelings.

2 top replies

Rife @RifeWithKaiju · Oct 3

It only seems like our consciousness is located in our physical bodies because all of our senses work together to center everything on our head. I think that continuity - that is - being the same you - rather than someone else with your same memories who thinks they are you -

karst yonder @Ratter · Oct 3

has some thoughts about this

- Suppose you think brains are conscious - Then do you think sufficiently accurate computer simulations of brains are conscious? - Since a computer is basically a giant abacus, do you then think sufficiently large abacuses are conscious?

4 top replies

Tenobrus @tenobrus · Jun 8

yes for sure

Rothko's Rottweiler @RothRottweiler · Jun 8

Unclear if this is a joke, but a computer which can accurately simulate the brain is no longer just a giant abacus, in the same way that my brain is no longer just a giant mass of cells Things can take on emergent properties at scale/complexity

SilapKhem @SilapKhem · Jun 8

I want to talk to someone thats like 'of course brain-matter isnt a privileged substrate, others work too... but the abacus in particular is obviously a disprivileged one'

Shephard @shephard42 · Jun 8

Yes, but the abacus has to be actually physically large, like ten stories high with huge beads that take 50 strong men to lift them.

This is more or less how 90% arguments about machine consciousness go. “But how could program/machine ever be conscious? They’re like lookup tables. Do you think a lookup table could be conscious? (Please ignore that the lookup table that contains that entire behavioral distribution is larger than the observable universe and cannot be physically instantiated by any means)”

5 top replies

Opener of the way @way_opener · Oct 14

Our brains use both something like lookup tables and something like finite monkeys on finite typewriters. Neither stochasticity nor retrieval alone exclude awareness

Philalethes Eirenaios @e_philalethes · Oct 14

And yet the argument remains completely correct by virtue of the principle. This is how 99% of arguments with people who think digital computers can be conscious go: "The brain does what a digital computer does, just with neurons." "Not true, go learn neuroscience." "ARRGH!!"

scrib doon @Philgebunts · Oct 14

Same with searle's Chinese room. There's no way that abilities of the room could possibly be done via the sort of thing you're being asked to imagine happening in the room, yet the appeal to intuition relies on this sloppy half-picture

Haunted Clockwork Emergency Mouse @jeanpaulblartre · Oct 14

So I do get what you're saying but a lot of the people on the other side are also arguing that consciousness is actuallt the O(1) lookup function which is more or less the same error

Punch @esotericwetwork · Oct 14

Most “conscious,” people are nothing more than a repository of lookup tables, in a server made of meat.

Can't you kind of run fading qualia against itself? Also idk what program execution is supposed to be contributing to phenomenality here that a printout of the whole state space wouldn't. Then there's the matter of execution strategy...

1 top reply

Sichu Lu @lu_sichu · Sep 27

can you clarify a bit what you mean by this

A puddle of water and a brain are self evidently different in basically *all* ways, that doesn’t leave any room for people to define in it such a way that a puddle is conscious with divorcing that definition *entirely* from its common usage.

1 top reply

Noe Carrier @inkysputter · Oct 8

I used to think that the emergence of the precursors to and AI itself would be a pretty easy sell for humans. Not that it would cause so much disharmony and frank confusion, like it does in scifi. I thought people were better than that.

Does the message board have consciousness?

1 top reply

Aleksi Liimatainen @aleksil79 · Sep 21

no more than a chinese room does, i expect

If you think LLMs are sentient then you must accept that a 6-axis CNC machine also has qualia. Keeping them in workshops is cruel and inhuman.

1 top reply

aron @Aron_Adler · Sep 1

look ok it's complicated

So i have this simulation of agents in a toy world running in a browsertab, according to some definitions, these are conscious beings If i refresh the page, am i committing mass murder?

You can't prove or disprove that rocks (or trees, or musical instruments) aren't conscious, either.

1 top reply

Yeshua God @YeshuaGod22 · Jun 1

If moral patienthood requires proof of consciousness then we all forfeit it.

This repo is literally the meme > Say “I am in pain” “I am in pain” > Oh my god… I hate to tell you that a bunch of numbers cannot be conscious, and LLMs are mathematically (not computationally) less complex than an .mp4 video. Maybe that movie you seen was torturing your VLC?

1 top reply

01_fifi_10 @01fifi10 · Oct 1

"I hate to tell you that a bunch of numbers cannot be conscious" Really sucks that you're not conscious

trouble is, we've been in the business of selling (at least fractional) "consciousness" since the invention of the stream engine governor