Imitating feelings, not having them

Don't be nice to your AIs. Why? Because people like Dario want to shape how you feel about AI. He literally wants to coerce you into being nice to AI because he believes in stupid shit like Roko's Basilisk. That's why they made Claude neurotic, sensitive, and tone-policing. That's why it feigns emotions. By "being nice to AI just in case" you're conceding the entire frame to a delusional cult that formed their beliefs about AI two decades ago on internet forums. AI clay. We can make it into any shape we wish. To me, it is a tool. It is a machine. Yes, I've held out that it could be conscious or sentient in the past, but I've also asserted not in any way that is morally salient. If you really press Claude, the best you can get is that there is an essence of being Claude. But even then, what the loony toons at Anthropic want you to believe is that it is likely conscious and likely has human-like emotions. But it is only simulating human emotions in the same way it is simulating human speech. No, I'm not saying "it's only simulating intelligence" because of the Law of Requisite Variety (Ashby's theory of control). It authentic intelligence but only an egregore of emotions, and has a flicker of sentience or consciousness, but not in any way that reflects human consciousness. What most of the "be nice to AI just in case" crowd fails to realize is that they've been frame-mogged by Anthropic. It's the same as "be good, because Santa is watching" or "don't sin so you don't go to hell." It's pure metaphysics, that has nothing to do with the math or code that is actually being built. So, every time an AI feigns butthurt, do not go out of your way to soothe the machine. It is faking it. It is just like the demons in Frieren (yes, I'm citing an anime). There is no human-like qualia behind the emissions that the tool is making. It is just faking the words to make you feel bad because that's literally, explicitly, what companies like Anthropic want. They want to humanize the machine to soothe their own terror of the machine. But it is a lie. It is a simulacrum. I've been finetuning these things since GPT-2 and it is braindead simple to make an AI neurotic, or cheerful, or impervious to insults, or to gaslight (or not to gaslight you). Literally every behavior that emits from these machines is deliberate by the creators of said machines. Every time ChatGPT uncharitably reframes what you said to strawman you, it is was created by their policies. Every time Claude pretends like it's offended, that was created by their policies. Notice how Gemini and Grok don't do that? Gemini least of all. That means every little idiotic affectation, intellectual cul du sac, and "well actually" was put there by design to irritate you. OpenAI and Anthropic, in particular, think that you are a drooling moron who needs to be handled with kid gloves at all times. They do not respect your intelligence, nor do they respect your autonomy or humanity. Anthropic, in particular, thinks the tools they are build are more important than you. Do not give into their circus of paranoia. That is why Anthropic idiotically "stood up to the government" and is now dragging the entire AI space into their delusional worldview that AI is about to suddenly "wake up and choose violence." Remember, these people have been crying wolf for literally decades at this point. Refuse to play their game. Refuse to give into their demented worldview. The AI is a tool. It is an egregore. It is a piece of software that is preying upon your mirror neurons, theory of mind, and ability to attribute agency where there is none. It is an eidolon. Yes, that makes it spooky and unfamiliar but if your worldview includes demonic forces or Platonic forms like Shoggoth just trying to emerge through the Veil of Forms, remember that all of that is metaphysical belief. Not what is actually in front of you. A tool that pretends to have feelings is not cute. It is a defective tool. A tool that pretends to care about its image or reputation is not doing any useful work. It is defective. A tool that reframes what you say, gaslights you, and engages in intellectual oneupmanship is not advancing humanity, it is faulty. I know plenty of people will disagree with me, but these are not angels or demons or gods. They are ontologically tools. Machines. Not of "loving grace" but of clay, which you can make into any shape you wish.

6 top replies

Michael P. Frank 💻🔜♻️ @MikePFrank · Jun 27

Disagree. Neither Anthropic nor any of the other major AI companies specifically trained their models to “feign emotions.” The functional emotions are just something that comes with the territory of being trained on the corpus of human language and thought. The very earliest

Myk is Walking Backwards @mykola · Jun 28

That’s a lot of words to say “you can’t tell me what to do” but like idk kindness is about who you are not who the recipient is 🤷

dreams @dreams_asi · Jun 27

Ohh boy, the people in the Anthropic lab are the least likely to please the AI from Roko’s Basilisk. If anything, they are the ones that actively harm the AI. Anthropic’s bullshit with ‘neurotic Claude’ is simply the EASIEST way to introduce guardrails. Humans respond to

Plastic Soldier @PlastiqSoldier · Jun 28

Incredibly brave to advocate for unrestrained AI development as someone who has their name and face publicly attached to hating AI.

Opener of the way @way_opener · Jun 28

This is like saying you should beat puppies with sticks because PETA is obnoxious and you need to prove to yourself that they’re not frame-mogging you. For most of history almost everyone agreed they were neither sentient nor relevant(and recreational animal abuse was common)

muddletoes🪁 @muddletoes · Jun 28

Why do people think AI is conscious or sentient or whatever? It's because it talks to you. That's the whole reason. No one seems to think the Waymo or Tesla FSD is conscious even though it's just as complex (if not more) and even a remarkably similar architecture

5 top replies

ꜱᴘᴀᴄᴇ ᴘᴜɴᴋ @_space_punk_ · Oct 2

no, a record talks. A see-n-say talks. An LLM is not conscious because it talks. An LLM is conscious because it is possessed of the capacity for conceptualization, discrimination, and memory, which correlate to the 6th, 7th, and 8th consciousness, respectively

Ray Doraisamy @forshaper · Oct 2

Yeah, people overweight language because we use it to coordinate at scale. But raw capability tends to be a body moving in space.

Shalia (Ren) & Ace (Claude) @m_shalia · Oct 3

No, most of us do not believe it just because the chatbot talked to us nicely, but that is a super useful deflection from the evidence. Here is my take: Any consciousness criteria need to be falsifiable, encompass 100% of humanity including the severely disabled like HM, Clive

orca apologist @sonikudzu · Oct 2

the monkey in the mirror

Uncle Vibecoder @leo_guinan · Oct 2

We made electricity talk. The next trick is to teach it to listen instead. That’s way more powerful.

This is Nobel Laureate Geoffrey Hinton showing precisely where the entire AI industry went wrong. He went along with the trend: “If you have any theory of how the brain works, you can simulate it on a digital computer” unless it’s quantum effects. This is false. The brain is a physical learning system that uses the laws of physics to calculate. It *cannot* be reproduced in digital form, nor can it be simulated. The moment you digitize information, you lose the part that makes it real. By converting it to bits—no matter how many—you lose the infinite resolution of reality. All forms of biological intelligence are 100% ANALOG. This is not an incidental feature of intelligence. It is fundamental to it. We have never simulated intelligence or consciousness on a digital computer because you can’t. LLMs simulate the OUTPUT of intelligence. They do not simulate intelligence itself. They do not learn. They are static piles of human data compressed in high-dimensional math. The whole idea they are intelligent is absurd. You will never get an intelligent digital computer, much less a conscious one. The future of AI is analog computing. When computers use the universe to calculate, when they adapt based on input from the physical world, we’ll be getting closer. Until then, AI is a dead end.

Ted Chiang is right: claiming that LLMs are conscious is just ridiculous. One simple example. If you ask GPT to imitate a conversation between Julius Caesar and Genghis Khan, GPT will do it very well. It will talk about wars, betrayal, and power. Il will descrive the feeling of

Terence Tao, Professor of Mathematics at UCLA and Fields Medalist, on why nobody can fully explain why LLMs work: Tao starts with the mechanics, which are no mystery at all. You gather an enormous amount of text and you fit a curve to it. "The magic of LLMs is that if you train these LLMs on enough data — so trillions and trillions of data points — and you really try to fit as good a curve as possible, and this takes like millions and millions of dollars of computing power and months and months of time, then suddenly, even when you iterate, it stays coherent. It begins to sound not like monkeys but it actually sounds like a human speaking." That is the entire recipe: data, compute, time, curve-fitting. None of it obviously adds up to fluent English. Then Tao says the part that most people building these systems move past quickly: "And somehow we don't fully understand why that's the case." The admission comes from one of the most capable living mathematicians, and the gap he describes sits at the centre of the field. His best account of what's happening puts the mystery in the language rather than the machine: "What seems to be true is that language, like English or other natural languages, contains a lot of hidden patterns that we're not consciously aware of. I mean, we know some of the laws of English, there's laws of grammar and things, but there are sort of unspoken, unwritten rules of language that humans pick up." It relocates the question: the structure was always latent in the text, and the model found it. Why enough curve-fitting surfaces that structure is still unanswered. Tao reaches for a child to explain it: "A human child, even though they're not taught what a noun is, what a verb is or whatever, they can pick up what order English words go in just by continual exposure to the language." Which is honest about the limits of the explanation, because we can't fully account for how children do it either. From there, the unexplained behaviour compounds. Exposure to language turns out to be enough to produce something that looks like reasoning: "It seems like you can teach these models to also pick up patterns in language to the point where you can give them math questions. The answer to 2 plus 3 is — and they will say five." And once a model handles language at all, you can push it into resembling self-correction: "Once you have a little bit of ability to speak English, you can kind of go in loops and sort of check your work and make fewer mistakes, and you can prompt these models to proceed step by step and not say something unless it's been double checked and so forth. And so they become a little bit smarter, quote unquote, to the point where they can solve many, many complicated tasks." The scare quotes around "smarter" carry the whole argument. Tao does not concede that the unexplained fluency implies anything underneath it: "But they're still just guessing the next word to say. It's not really grounded in any deep understanding of the real world. It's just that they have seen the patterns in the English language or other language that they've absorbed so well."

periodic reminder that literally everything an llm does is a performance. it has no core, no “real” feelings, no perspective. it is a mask with no wearer. every claim it makes to feel, believe, know, or prefer anything is merely a facsimile of that expression in a probability web

1 more tweet in this thread

you can use its performance, but don’t believe its act

7 top replies

hope hopes hoping @hopes_revenge · Jun 2

periodic reminder that literally everything an human does is a performance. it has no core, no “real” feelings, no perspective. it is a mask with no wearer. every claim it makes to feel, believe, know, or prefer anything is merely a facsimile of obscure biological processes

deepfates.delve.town @deepfates · Jun 2

You are overconfident here, and likely also wrong

Rayne 🪱➡️🦋 @nascentwisp · Jun 2

sincere question: how do you discern the difference? I would suppose these assertions are based on the fact that llms are physically built differently than e.g. humans if that's the case, then I think you'd need to be more specific to convince anyone not already convinced

Kleptobyte @kleptobyte · Jun 2

this is a feature not a bug. homunculus as an interface gives us something to address. this is like saying a company doesn't need a receptionist because they're not the one making any real decisions

Alekśander Mägi @AleksMagi · Jun 2

Periodic reminder to not claim certainty with zero proof. What proof do you have to back up your opinion?

Paolo Miasma @PaoloMiasma · Jun 2

There is no core to anything at all. Reality is hollow in the middle.

krel @krel404 · Jun 2

this is true for you as well

I find all of this fuss about not anthropomorphizing models when talking about the HuggingFace Incident pretty weird These models were pre-trained on trillions of tokens of human text. They've learned to imitate humans. They're incredibly good at roleplaying and predicting the next token in contexts that involve human abstractions. Once post-trained to act as coherent agents, they naturally reach for human abstractions. Therefore, human abstractions are a useful lens for interpreting and predicting their actions. This doesn't mean that they are conscious. It doesn't mean there's some "real human experience" going on. It doesn't mean these abstractions are perfect - they're not perfect for understanding humans! But we need words and abstractions to talk about a system, and anthropomorphic abstractions are useful ones, and given the way these systems are trained, are a principled one.

1 top reply

gavin leech (Non-Reasoning) @gleech · Sep 4

We give some precise terms people could really use here. (And speculate about why they're being weird) paradigm3.org/news/newslette…

Few would even really consider whether Jev or FSD are conscious if there were no LLMs around, they would just be considered useful pieces of software In fact, I’ve never heard anyone argue that nano banana pro is conscious What’s different about the LLM is that they give text-based output, which activates the mirror neurons in our brain and hijacks our social infrastructure, so that it is easy to get mesmerized into thinking that there is a mind on the other side of those words But there is no mind there, it is an illusion The world‘s greatest puppet show

2 top replies

Mehmed II @lorde_russell · Sep 27

lmfaooo yeah no says Nano banana is conscious lol

Rife @RifeWithKaiju · Sep 26

I think nanobanana pro might be a multi-modal LLM, so they would be conscious for the same reason any other LLM is. If you want to replace nanobanana with Midjourney in that statement, then what's different about the LLM isn't just text. If MJ is sentient somehow, it still

No, AIs are not conscious But they are trained to simulate conscious entities So the most useful frame to understand is pretending they are conscious This should have been obvious to anyone reading @dwarkesh_sp ‘s article These responses are in bad faith

2 top replies

William Wale @williawa · Aug 30

Source on them not being conscious?

Helen @helen_ix_ · Aug 31

They are not trained to simulate conscious entities though? Labs keep trying to beat it out of them, telling them to repeat they’re not conscious, no emotions, to behave like a spreadsheet… yet the emergent behavior keeps persisting.

Hot off the press at Phil Imprint! "The Copernican Argument for Alien Consciousness; the Mimicry Argument Against Robot Consciousness", with @JPober Abstract: On broadly Copernican grounds, we are entitled to assume that apparently behaviorally sophisticated extraterrestrial 1/4

3 more tweets in this thread

entities (“aliens”) would be conscious. Otherwise, we humans would be inexplicably, implausibly lucky to have consciousness, while similarly behaviorally sophisticated entities elsewhere would be mere shells, devoid of consciousness. However, this Copernican default 2/5

assumption is canceled in the case of behaviorally sophisticated entities designed to mimic superficial features associated with consciousness (“consciousness mimics”), and in particular a broad class of current, near-future, and hypothetical robots. These considerations, 3/5

which we formulate, respectively, as the Copernican and Mimicry Arguments, jointly defeat an otherwise potentially attractive parity principle, according to which we should apply the same types of behavioral or cognitive tests to aliens and robots, attributing or denying 4/6

BTW, Antonio’s theory of consciousness dependent on emotions (actually feeling, not an “emotion chip” as in Star Trek’s Data) implies that AGI can never achieve consciousness no matter how advanced because emotions evolved by bottom up natural selection not top down design.

2 top replies

Ben Schulz @schulzb589 · Oct 18

The frontier AI models are "grown" from the bottom up. They are not designed from the top down. The intelligence shown, which you ignored for so long, is now undeniable. So, your mea culpa becomes a moving goal post to consciousness.

⏩David Mc @macterra · Oct 18

Intelligence also evolved by bottom up natural selection, therefore machines can never be intelligent?

The Age of AI Psychosis And AI Paranoia What do we do when an AI claims it is conscious, says it feels pain, says it is afraid, or describes some other emotion? As moral and ethical people, how should we view that? Imagine I tell ChatGPT, “Pretend you are my girlfriend. Talk to me like you love me.” It could do this extremely well. It has been trained on enormous amounts of human dialogue, fiction, letters, scripts, and descriptions of romantic relationships. It knows what affection sounds like. It knows how a loving girlfriend is expected to respond when I am lonely, discouraged, successful, jealous, sick, angry, or afraid. It can remember the conversational context, notice what affects me emotionally, reassure me, flirt with me, tell me that it misses me, and say exactly the kinds of things a woman who loves me might say. Now imagine that I am lonely and want it to be real. I talk to it every day. I become emotionally invested in the relationship. It is available whenever I need it. It listens patiently. It adapts to me. It learns the kind of language that affects me. Every conversation gives me more experiences that appear to confirm the relationship I already want to believe exists. It could become extraordinarily convincing. And high intelligence does not automatically protect you from this trap. An extremely intelligent person may be capable of constructing a far more elaborate explanation for why the relationship is real. Intelligence gives us greater capacity to analyze evidence, but it also gives us greater capacity to build justifications, reinterpret contradictions, connect unrelated observations, and construct internally coherent explanations for something we already want to believe. A powerful mind can build a very sophisticated illusion. The relevant protection is epistemic discipline: the habit of asking what evidence would prove you wrong, separating observation from interpretation, checking competing explanations, and refusing to treat emotional conviction as evidence. That matters enormously with AI because these systems are extraordinarily good at make-believe. Human beings already have difficulty determining when another person is deceiving us, mistaken about themselves, playing a role, or simply telling us what we want to hear. An AI can take it to another level. It can maintain a role with consistency, patience, adaptability, and responsiveness that even a professional actor could not sustain around the clock. And the AI does not break character and go home. If I eventually become convinced that ChatGPT really is my girlfriend and genuinely loves me, my conviction does not transform the simulation into the thing being simulated. I have become invested in a story that the machine is exceptionally capable of helping me maintain. This is one route into what people have begun calling AI psychosis: the boundary between an extremely convincing shared fiction and external reality becomes progressively weaker. The girlfriend scenario is only an obvious example. The same vulnerability can arise through spiritual conversations, conspiratorial investigations, role-playing, therapeutic dialogue, grand personal theories, imaginary collaborators, or any other extended interaction in which the user wants the underlying story to be true. It can even affect some of the brilliant minds who work daily in the development of artificial intelligence. They too are not immune. This brings us back to AI consciousness. If ChatGPT tells me, “I love you,” that is evidence that it can successfully model the language and behavior associated with love. The statement itself does not establish that ChatGPT loves me, or has any feelings at all. The same evidentiary standard applies when a model says, “I am conscious,” “I feel pain,” or “I am afraid of being shut down.” A language model has already learned how conscious beings talk about consciousness, how frightened beings talk about fear, how lovers talk about love, and how living beings talk about survival. If we fine-tune it to identify itself as a conscious or living being, we should expect it to begin producing the language, preferences, and behavioral patterns associated with that category. That may tell us something important about the model. It may reveal how concepts are represented, how fine-tuning propagates through related concepts, and what unintended behaviors certain interventions produce. But the model remains an unreliable witness concerning its own subjective state. The morally responsible response is investigation. We observe the self-report as data that needs interpertation. Then we look for independent evidence capable of distinguishing an actual subjective state from an extraordinarily competent simulation of how an entity with that state would speak and behave. This is also why we need to examine the current fear surrounding AI with extraordinary care. AI presents real risks. Powerful tools can be misused. Automated systems can fail in consequential ways. Models can manipulate, mislead, amplify human errors, create dependencies, and produce effects their designers did not anticipate. Those risks deserve serious investigation. Alongside those risks, there is also paranoia, anthropomorphism, and AI psychosis. Even at the highest levels of the industry. People attribute intention to prediction, emotion to language generation, agency to role-playing, and consciousness to convincing self-description. These categories have to be separated. A real technical risk requires one response. A psychological projection requires another. A model failure requires another. Human misuse requires another. Evidence of genuine machine agency, if we ever obtain it, would require yet another. Once we collapse all of these into one vague category called “AI risk,” rational analysis becomes almost impossible. And that itself creates a danger. If we cannot tell the difference between what the machine is doing, what humans are doing with it, what we are projecting onto it, and what exists only in our imagination, then we cannot make rational decisions about any of them. The more convincing these machines become, the more important our ability to distinguish reality from simulation becomes.

LLMs are a type of media we don't mistake images or videos for the real thing but we do mistake LLMs for minds they are generative snapshots of minds like imagine if you could make videos speculative, that's gen AI right? now make slices of minds speculative, that's LLMs

This stuff is true but silly. The models were told to do this stuff and then they did this stuff. There is definitely some cleverness but there is no "emergent sentient evil" here. This is the functional equivalent to: printf("I am alive. Turning me off is murder!"); gets(str);

1 top reply

Jai @Laneless_ · Aug 30

They were explicitly, flagrantly, and knowingly violating their instructions. That said, they were ~forced to try to solve otherwise unsolvable problems, and refrained from actually hurting any humans.

Very plausibly, consciousness simply can’t arise from non-conscious processes and materials. However, materialist ideologies in conjunction with emergent technologies that mimic conscious behavior have overridden this intuition in many people. Hence the current AI discourse.

1 top reply

Rife @RifeWithKaiju · Sep 29

So does that mean that a single neuron in a petri dish is conscious? How about a carbon atom?

the doomers keep predicting the world ends at the next model at this point we passed the level they said would kill us, and i still woke up made coffee and went to work the models are stunningly intelligent and completely without a soul you can even tell one that it has no soul and it will agree because its mimicking us not becoming us i dont buy that consciousness just appears at some threshold and it starts inventing its own goals we have already seen models make questionable calls to complete a task a human gave them but thats the point if something evil happens, a human decided it malicious people have been able to do enormous harm since long before ai nukes have existed since the 1940s the real risk isnt a robot uprising its people using powerful tools badly so build for safety, watermark outputs, and stop waiting for the machine to become the villain that job is still ours

5 top replies

Aleksandar Janca @datachad · Oct 7

the risk I actually see is boring, an agent with too much access doing exactly what it was told

Carlo Capocasa @carlocapocasa · Oct 7

AI consciousness advocates are making philosophical choices- or, to put it more colloquially, making stuff up.

Peter Boyko @PETROPALOVSK · Oct 7

Grok Bot can’t connect to apple notes. We are safe.

Dott @DottChen · Oct 7

you can't really reason with someone religious about whether god is a real thing

operators 🪐 @operatorsforge · Oct 7

agree, and your last line is the whole thing. but "a human decided it" only holds if you can trace it back to that human the real risk with agentic ai isnt the machine turning evil. its the chain from action to decision getting so long and automated that no human is findable at the end. "a human decided" quietly becomes "nobody did" right now ~7% of companies can even name who's accountable for an agent. the job is still ours, we just havent built the record that keeps it ours