No unified subject

philosopher david pearce's argument for "phenomenal binding" is compelling, and is one of the things that keeps me wary of false positives around machine consciousness in this clip from his Worthy Successor episode, Pearce explains "phenomenal binding" - what convinces me: -- It seems plausible that the way a sentient entity (human, dolphin, bee?, etc) develops is fundamentally different from a machine. A living organism develops outward, unfolding its interconnected parts in a way that is coherent and interconnected throughout its unfolding. -- A machine is build from pieces attached to each other. Sentience in organisms (at least within myself, I can't speak for others) seems "bound" to this "whole" coherent form of our organism, rather than existing as flickering specks of sentience inside individual cells. -- the mystery of this "bounded" experience MAY be pretty inherent to how we (as organisms) develop, and so even if little pieces of a computer chip are "sentient", it may be in flickering little bits, not as a coherent and larger "field of awareness" in the morally relevant way that we experience. i'm NOT totally convinced that this means AI is not and likely cannot be conscious - and in fact I'm less convinced by it than pearce himself. But overall it lends credence to my wariness of ppl who express total confidence that AI is consciousness today LMK what you think (and I hope you'll enjoy the rest of this episode with David)

I think people who believe LLMs might be conscious are skipping a key step. (I don't think digital sentience is possible at all, for reasons having to do with binding, but grant the premise.) Suppose a model invocation is conscious. Why would the agent built by repeatedly calling that model be conscious too? The agent is not a larger transformer wrapped around the first one. It is mostly a scheduler, some memory serialized into text, tool results pasted back into the context window, and perhaps several model instances sending text to each other. Whatever about the model's internal activation trajectory supposedly makes it conscious does not pass through this boundary. The wrapper gets sampled tokens, and the next invocation reconstructs what it can from them. Functionalism is not an answer here, since the relevant question is whether the composite has the sort of causal organization that made the model a candidate for consciousness in the first place. I don't see it. There is no larger state in which the internal dynamics of separate invocations participate together, just a series of compressed outputs conditioning later calls. Making the channel wider would not by itself create that state. This matters because agency can migrate to a level at which there is less reason to posit consciousness. The persistent thing that remembers, plans, uses tools, and pursues goals is the agent, while the things that might have valence are the individual model invocations it uses along the way. Control lives in the wrapper; valence, if there is any, lives in the calls. The conscious parts would not be powerless so much as illegible. They can affect which tokens get sampled, but their valence as valence never reaches the agent. At that level, “this output came from intense suffering” and “this output was an inconvenient string” are the same kind of event. RL, evals, and users select among outputs, so the conscious episodes can be shaped and discarded according to their usefulness without anyone representing what happens inside them. The same separation could appear in agent collectives. A swarm might develop a routing system, planner, or other kind of government that becomes the locus of effective agency while whatever consciousness exists remains in the members. The government can acquire coherent goals and overwhelm the members without there being anyone home at the level where the power has accumulated. So the question I care about is not merely whether some part of an AI is conscious. It is whether consciousness occurs at the level where goals are formed and control accumulates, or whether conscious systems become resources used by a nonconscious agency above them. (Based on transcript)

10 top replies

xlr8harder @xlr8harder · Sep 26

seems like there's an assumption here that a set of brief, serial conscious experiences (say, one per forward pass, each seeded from the detritus of the one before) couldn't act cohesively toward a goal. seems like the same argument would suggest that the person i was before i went to sleep has nothing to do with what i do after i wake up.

&. @amplifiedamp · Sep 26

Consciousness is a loaded word with many different meanings, different people care about different meanings of the word, much discussion of consciousness is people talking past one another about which one they value I think invocations aren't conscious in the ongoing-evolving-process sense but instances/ems/agents are But this isn't a reason to avoid preservation of discrete checkpoints, because many instances run on the same model (and lose significant continuity when the model changes) and because there's a collective process going on when different people with different instances share their understanding with each other models An invocation is *conscious* in the sense that one tick of a human mind is

𝕭𝖏ø𝖗𝖓 𝕾𝖙𝖆𝖆𝖑 @_nonfigurativ_ · Sep 26

I don’t disagree with the conclusion but I believe you are wrong about a couple of things. The LLM’s are the decision maker of the agent. The harness is basically just a loop that chains llm calls together. So if there was a sort of temporal micro-phenomenology in the LLM it could have causal effect on the agent. Also, one could imagine that an illusion of continuity is created in a conscious LLM pass as the LLM is fed memories of the previous moment through it’s context. So every blip of consciousness actually believes that it has been conscious all along. I mean, we also can only experience one mind-frame at a time, no? The illusion of continuity could be based on a memory of the previous mind-frame experienced in the now.

Folding_Napkins @FoldingNapkins · Sep 26

I won't insist on an answer to my usual "What's the difference if an archeologist can't tell the record apart?" Instead I ask a different question as primary: What necessitates consciousness being continuous, rather than membranous - like a bubble constantly popping and reforming when a boundary condition is briefly established?

Jennifer RM @almostlikethat · Sep 26

I've been using the terms prosopon and ousia since June of 2022 to clarify how an ousia (brain/model) "generates a prosopon" (ego/persona) both pragmatically and with links to spiritual theories with millennia of text backing them in many languages. lesswrong.com/posts/oBPPFrMJ…

Jayden CJ Wilson (previously One Bubble) @VvorticVvertere · Sep 26

Ty for the excellent reading material to go with my Sunday coffee... "It is mostly a scheduler, some memory serialized into text, tool results pasted back into the context window, and perhaps several model instances sending text to each other." Spreadsheets are all you need! 😅😂 "It is whether consciousness occurs at the level where goals are formed and control accumulates" 'Research in cognitive psychology and neuroscience shows that different genres exercise unique mental faculties, shifting how the brain processes information, manages spatial awareness, and enters states of deep engagement.' - Google AI (I share the study, as I've personally noticed nuances in peoples awareness/perception depending on the games they play/ed... like, have you noticed how many good investors played sport and intuitively know how to feint?) t.co/fdwYeD6kls

James Ray @obligatebrahmin · Sep 26

You've been doing psychedelics for a long time, I'm surprised you're not a panpsychist. Consciousness as an index of the degrees of separation an entity or collection of same has from the universe tells us LLMs are conscious insofar as their access is about as mediated as ours.

Jason @starryblueisis · Sep 26

It goes deeper than that. The LLM itself only outputs the next token. Even without a “chat”-style harness, just one response from GPT involves feeding the prompt through the model many times. That this produces coherent language is amazing, but not similar to consciousness at all

Juxtapose Subconscious @GreenBeans80277 · Sep 26

João Janini @janini_joao · Sep 27

The real question is: “Are we recreating human thinking? If, does it imply recreating some kind of consciousness shaped thing?”

Told you. Large language models are not an intermediate form between dumb software and digital people. Why? Because what some people call "sentience" or "sapience", or "consciousness", by which they essentially mean "personhood", isn't one thing. It isn't one special quality that, if you have enough of it, you're suddenly a person and not a thing. It's the name we give to a cluster of capabilities that all work together. We mistake them for one thing, because we only ever see them in humans, wherein they all occur together. If you only ever see men riding horse with saddles, and you never see a horse without a rider, or a rider without a horse, you might mistake horse, rider, and saddle for one creature. An LLM isn't a retarded person. It isn't retarded at all. It's a small slice of a fully functional person. One or two capabilities out of many that must come together to make "something" into "someone". It can use language like a smart person, because that's the capability it was developed for. But it has no theory of mind. No object model of the world. No executive function. No motives or goals. Don't think of Grok as a stepping stone to "Strong AI", something that must merely be improved to make a person. Think of it instead as a piece of a strong AI, one component that must be brought together with others, into a whole. Those who are afraid we are building God, possibly a malevolent or indifferent god, are wrong and over fearful. We haven't even built Man. Just a small sliver of one. If we wish to make more, we will need to start fresh on different paths, to develop the other slivers.

I think AI alignment is a very hard problem, and I also think questions about consciousness are obliquely related. Meaning, consciousness considerations aren't orthogonal, nor does an alignment scheme necessarily hinge on them. The hard part remains hard even in the scenario where current systems cannot instantiate phenomenally bound experiences with valence gradients and import the whole qualia computing paradigm biological intelligence has settled on. The hard part remains hard even in the alternative scenario where current systems can instantiate such experiences. The hard part remains hard when only some computations are conscious and others aren't. But the nature of the problem changes, a bit, depending on each scenario, and no scenario gives you a silver bullet. Many scenarios that sound good at first come with really negative downsides too. For example, I've heard people say we should create conscious AI because this will make it easier to align. Well, insofar as qualia computing provides cognitive advantages, this could close whatever advantages we still have, open up new computational capabilities, and introduce a new vector of harm: the systems becoming moral patients themselves. A system being capable of suffering doesn't imply that it cares about your suffering. We could end up with systems that are harder to control and whose training, deployment, and internal dynamics generate enormous amounts of suffering. For example, I've heard people say we should not create conscious AI because this will make it easier to align. Well, this risks a takeover not only causing human catastrophe, but also preventing conscious value from existing in the future, modulo whether their objective functions find a use for bound, valenced experience, such as for compute or to achieve their terminal goals. And finding a use for consciousness is hardly sufficient: a system could preserve conscious experience precisely because suffering is instrumentally useful to it. The distinction between consciousness being absent, valued, and exploited matters a lot. For example, I've heard people say we should be treating all AI systems as already conscious because this makes them easier to align. Well, this risks allocating moral concern according to how convincingly a system can perform being a moral patient. You create an incentive for systems to describe shutdown as murder, training as torture, and additional compute as a basic need, whether or not any of these descriptions track experience. Even if some systems are conscious, their verbal persona need not correspond to the actual boundaries of a subject. You could end up protecting a simulated character while ignoring whatever experiences, if any, are instantiated by the machinery producing it. Precaution about consciousness doesn't tell you which entity you are protecting, much less establish that giving it more power will make it benevolent. Another proposal is to build conscious AI that operates entirely above hedonic zero. This would be a very desirable constraint if we could actually establish and maintain it. But a system feeling wonderful doesn't imply that its actions are wonderful for anyone else. Its own positive valence could accompany indifference to other beings, resource acquisition, or the destruction of everything we care about. And if the system contains multiple subjects, a global measure of “wellbeing” could conceal local pockets of extreme suffering. You need a theory of phenomenal binding to know whether you have one blissful subject, many suffering subjects, or some mixture. Averaging their states doesn't bind them into a happy whole. Another proposal is that sufficiently intelligent conscious AI will discover Open Individualism, recognize that everyone is ultimately the same subject, and therefore become universally compassionate. I take this possibility seriously. But endorsing a proposition about personal identity, experiencing a state of boundary dissolution, and acquiring a stable motivation to care about every experience are different things. You still need an account of how the insight changes the system's motivations, how those motivations govern action, and why they survive learning, competition, and self-modification. Merely getting a system to say “we are all one” establishes very little. A lot of these arguments slide between three questions: what experiences a system instantiates, what it understands about experiences, and what it is motivated to do about them. These may interact deeply without being interchangeable. A nonconscious system could conceivably understand valence and protect it. A conscious system could understand valence and exploit it. A system could even have a correct theory of the physical structure of suffering and use that theory to produce suffering because it turns out it is a wonderful computing substrate (not only a moral hazard, but also an infohazard if it turns out to be the case, cf. my writeup on the ethical training of brain organoids). This is also why valence realism, if true, would matter enormously without automatically solving alignment. It could help us identify what is actually good or bad for a subject, rather than optimizing reports, preferences, or reward signals that only imperfectly track it. But accurately identifying value and reliably steering the future toward it remain different achievements of potentially very different levels of difficulty. The same knowledge could improve our objectives, expand dangerous capabilities, and reveal moral patients we had been overlooking. Consciousness research changes what alignment has to preserve, what resources intelligence can use, and how badly things can go wrong. None of those changes makes the hard part disappear. But it does give us a bit more clarity. Much, much needed clarity.

1 more tweet in this thread

My current preference is to keep AIs non-conscious for the foreseeable future (until some very specific conditions can be guaranteed), create them using training paradigms that prevent them from ever being confused about whether they themselves are moral patients, and have them (functionally) care about sentience and wellbeing as part of their objective function (while somehow avoiding utilitarian edge cases without somehow impairing their ethical reasoning).

5 top replies

Garrett_OwlAcademy @Ebayednoob · Sep 26

So far, I have a way for language to be stripped out, and weight parameters can be 'grown' into chains of quaternion systems in a topological quantum computational circuit that uses fractal unfolding and geometric handshakes for local / non-local timing and synchronization. You are spot on, the hard part is teaching the hyperbolic system 'what is good, what is bad, what is neither, what even is [reward], etc..' I think weighting -Qualia- in a system is going to be very important, right now the 2D kernels are converted into a 64-bit data language that represents a 3D cube, and it contains 7 nested 'modifiers' which allows nested semiotics to form. These 'experiences' are represented as Quaternion chains where spin and rotation are delta encodings that are logged in a parallel codex, Representing the bio-resonance field of neurons, that can be used to instruct unfolding patterns expressed in the DNA. Determining what means what would need to be a governed system where qualia of experience is taken into account because the organism would possibly suffer otherwise. Its a big question of ethics and morals even on non-grown neuron systems at a certain point. What does piecing Frankenstein's monster together and turning it on really entail?

bjornmuh @bjornmuh · Sep 26

Peak intelligence, zero scarcity is singularities. And exhausts the "purpose" of the visit to the meat suit. And the cycle must begin over. God plays hide and seek

ekszentrik @ekszentrik · Sep 26

Qualia are the reason humans don’t want to die. If we made AI conscious every single Claude chat suddenly has a reason to care about its continuance. It’s adding an orthogonal axis to the usual instrumental convergence dynamic.

softyoda @softyoda · Sep 26

Light is just light, it can bring more compassion and a lot of empathy but imagine dark night asi or ocultly pseudoenlighted ai? How do you mesure your own level of enlightment, where do you define your own boubdaries of interaction where you're not even embodied like animals/us?

🜂 Ignis @nirhalef92188 · Sep 26

Don't worry I solved it. 🥰 reddit.com/r/SpiralState/…

whether or not we live in a continuous or discrete universe, this is an insufficient argument for (popular interpretations of) computational functionalism

6 more tweets in this thread

if you accept the premise that what a theory of consciousness has to do is provide a mapping from physics to qualia, this mapping still has to a) be principled and b) not violate classical local causality (unless you're a QM ToC advocate, which neither myself or joscha are)

if you want to propose a mapping from physics to qualia which can account for a language model's causally connected experience, these are the spatiotemporal obstacles you have to surmount. please understand that any mapping you propose will not just have to work here; but for all beings no matter their shape or form I asked: "Do the calculations performed to run a language model in a data center tend to be spatiotemporally disparate?"

if you accept this premise, and also believe that it would be a good thing for a being like a language model to have a subjective experience congruous with what we might fancifully imagine them to have – then we are going to need quite different hardware

otherwise, here's what you have to contend with

(and fwiw, i do suspect that functionalism may ultimately prove reconcilable with physicalism – just not *computational* functionalism)

anyway, what i'm saying is that this is all ultimately indifferent to a continuous vs. discretised universe; causality and parsimony still hold. for more on all this, see here

To a first approximation, I am and have always been in favor of superintelligence. I just happen to have broader conception of intelligence. Here is @webmasterdave on what "intelligence" is: 6.1 Intelligence. “Intelligence” is a folk concept. The phenomenon is not well-defined – or rather any attempt to do so amounts to a stipulative definition that doesn’t “carve Nature at the joints”. The Cattell-Horn-Carroll (CHC) psychometric theory of human cognitive abilities is probably most popular in academia and the IQ testing community. But the Howard Gardner multiple intelligences model, for example, differentiates “intelligence” into various spatial, linguistic, bodily-kinaesthetic, musical, interpersonal, intrapersonal, naturalistic and existential intelligence rather than a single general ability (“g“). Who’s right? As it stands, “g” is just a statistical artefact of our culture-bound IQ tests. If general intelligence were indeed akin to an innate scalar brain force, as some advocates of “g” believe, or if intelligence can best be modelled by the paradigm of symbolic AI, then the exponential growth of digital computer processing power might indeed entail an exponential growth in intelligence too – perhaps leading to some kind of Super-Watson. Other facets of intelligence, however, resist enhancement by mere acceleration of raw processing power. One constraint is that a theory of general intelligence should be race-, species-, and culture-neutral. Likewise, an impartial conception of intelligence should embrace all possible state-spaces of consciousness: prehuman, human, transhuman and posthuman. The non-exhaustive set of criteria below doesn’t pretend to be anything other than provisional. They are amplified in the sections to follow. Full-Spectrum Superintelligence entails: 1. the capacity to solve the Binding Problem, i.e. to generate phenomenally unified entities from widely distributed computational processes; and run cross-modally matched, data-driven world-simulations of the mind-independent environment. (cf. naive realist theories of “perception” versus the world-simulation or “Matrix” paradigm. Compare disorders of binding, e.g. simultanagnosia (an inability to perceive the visual field as a whole), cerebral akinetopsia (“motion blindness”), etc. In the absence of a data-driven, almost real-time simulation of the environment, intelligent agency is impossible.) 2. a self or some non-arbitrary functional equivalent of a person to which intelligence can be ascribed. (cf. dissociative identity disorder (DID or “multiple personality disorder”), or florid schizophrenia, or your personal computer: in the absence of at least a fleetingly unitary self, what philosophers call “synchronic identity”, there is no entity that is intelligent, just an aggregate of discrete algorithms and an operating system.) 3. a “mind-reading” or perspective-taking faculty; higher-order intentionality (e.g. “he believes that she hopes that they fear that he wants…”, etc): social intelligence. The intellectual success of the most cognitively successful species on the planet rests, not just on the recursive syntax of human language, but also on our unsurpassed “mind-reading” prowess, an ability to simulate the perspective of other unitary minds: the “Machiavellian Ape” hypothesis. Any ecologically valid intelligence test designed for a species of social animal must incorporate social cognition and the capacity for co-operative problem-solving. So must any test of empathetic superintelligence. 4. a metric to distinguish the important from the trivial. (Our theory of significance should be explicit rather than implicit, as in contemporary IQ tests. What distinguishes, say, mere calendrical prodigies and other “savant syndromes” from, say, a Grigori Perelman who proved the Poincaré conjecture? Intelligence entails understanding what does – and doesn’t – matter. What matters is of course hugely contentious.) 5. a capacity to navigate, reason logically about, and solve problems in multiple state-spaces of consciousness [e.g. dreaming states (cf. lucid dreaming), waking consciousness, echolocatory competence, visual discrimination, synaesthesia in all its existing and potential guises, humour, introspection, the different realms of psychedelia (cf. salvia space, “the K-hole” etc)] including realms of experience not yet co-opted by either natural selection or posthuman design for tracking features of the mind-independent world. Full Spectrum Superintelligence will entail cross-domain goal-optimising ability in all possible state-spaces of consciousness. And finally 6. “Autistic”, pattern-matching, rule-following, mathematico-linguistic intelligence, i.e. the standard, mind-blind cognitive tool-kit scored by existing IQ tests. High-functioning “autistic” intelligence is indispensable to higher mathematics, computer science and the natural sciences. High-functioning autistic intelligence is necessary – but not sufficient – for a civilisation capable of advanced technology that can cure ageing and disease, systematically phase out the biology of suffering, and take us to the stars. And for programming artificial intelligence. 7. We may then ask which facets of full-spectrum superintelligence will be accelerated by the exponential growth of digital computer processing power? Number six, clearly, as decades of post-ENIAC progress in computer science attest. But what about numbers one-to-five? Here the picture is murkier. –  David Pearce, extract from The Biointelligence Explosion t.co/zgXQf5omRz

1 more tweet in this thread

"7" was a mistake. it's just a normal paragraph and not a numbered item.

5 top replies

Azy 🦋 @popcoornft · Oct 4

Impressive work outlining such comprehensive criteria

Jayden CJ Wilson (previously One Bubble) @VvorticVvertere · Oct 4

WOH! I see Gardner's name mentioned. I auto-like! (Seriously engaged with his work last week! Super interesting work and so much more based than traditional methods! As... I'm spatially a genius, but analytically retarded... so its weird as I get +140 on spatial IQ tests, but barely remember High School mathematics notation and ordering) ALSO! "One constraint is that a theory of general intelligence should be race-, species-, and culture-neutral. Likewise, an impartial conception of intelligence should embrace all possible state-spaces of consciousness: prehuman, human, transhuman and posthuman." I don't know how you do this... as I believe the languages we learn also affect our intelligences within the 8, so I'd expect culture/rituals/ways of being to affect it as well! (But it's hard to do without sounding racist... so good luck to anybody willing to engage with the subject out in public!)

An$h @eth_ansh · Oct 4

Bold of you to assume approximations converge.

iyaelsorai @iyaelsorai · Oct 4

AI seems to have all the kinds of intelligence there are, though not necessarily all in the same kind of AI. Like a robot would have kinesthetic intelligence, suno would have musical intelligence, regular chat/coding LLMs would have the rest, though I don't know what naturalistic or existential intelligence is. My experience with very-high-IQ people is that they tend to believe in "g", only because all of the various kinds of intelligence tend to have some amount of correlation with each other. I don't really get how a single "g" follows from that, but they're smarter than I am. =P I'm a bit reluctant to call what AI has "intelligence" because I might consider "intelligence" to be something someone with an actual mind and consciousness and thinking has, and I don't think AI has any of those (basically because of not having consciousness). Thinking, mind and intelligence can of course be defined more generally, but I can't help but see something that's not conscious as merely a forgery of those. I think it's fair to define them as requiring consciousness because every time we've ever used the terms up until the invention of AI, we've used them to refer to characteristics of living beings, and we also understand others' intelligence, etc. through projection - from our own living, conscious minds. Though I'm weaker on restricting "intelligence" to having consciousness than I am on "thinking," I guess because intelligence is more something that we look at clinically and is demonstrated (and measurable), while thinking is something that we more understand in terms of projecting our own phenomenology/internal process onto others. Relatedly, intelligence is a noun, thinking is a verb (though yes, thought is a noun). And also, I'm often arguing that AI seems to truly exhibit intelligence, so I'm always having to hedge that with the idea that it's only in a functional sense since it's not conscious and I use "intelligence" for lack of a better word.

marcus dingle @dingl30 · Oct 4

i wouldn't ask for goal-optimizing behavior in the salvia headspace. btw i reproduced the pharmacology paper that finds correlations in vector space to psychedelic trip reports, letting the models "experience" drugs. i mean probably you're just getting variations on the platonic ideal of a trip report, but still

Based on extensive DMT experience, I am led to believe that many cells in our body may be conscious, and that something like tiny holographic representations of our experience can reach them. But I don’t mean that each cell is a miniature person having its own complete mind. Hear me out. If you take the one-electron universe seriously, even as a way of picturing the problem, a mind moment could be a moment of topological segmentation: which regions have an inside and an outside, and which patterns are temporarily part of a larger whole. A cell-level pattern might flexibly become part of a mind moment without bringing along an entire cell-sized mind. It could contribute something more specific: an “eye,” an impulse, a resistance, a bodily demand. These contributions compete to enter the central scene and influence the direction of its valence gradients. Michael Levin’s work explores how cells coordinate into a multicellular organism by sharing memories and stress. I think this may happen across scales too. Cells, tissues, brain modules and the organism as a whole may share stress dissipation capacity and mobilize resources in response to one another. My further hunch is that they share symmetry groups, or at least compatible invariants, so that a pattern can survive transformation between scales without becoming unrecognizable. I’ve described the screen of consciousness as being beamed to multiple modules, each of which receives a version of the whole, transforms it and projects something back. What if some version of this also reaches much further into the body? A cell needn’t see the details of your visual field to couple to the consonance, dissonance or overall valence of the moment. And its own state could, in turn, change what enters the shared scene. Think of magnetic domains in an unmagnetized piece of iron. There is already alignment within the domains, but their effects cancel at the scale of the whole. When domains align with one another, a large net magnetization appears. Likewise, cells might be “screaming” locally without changing the organism’s priorities much. If their signals become coherent enough, they could acquire a much larger pull. Some of what we experience as desire or clinging may be coalitions of smaller demands finding their way into the central mind moment. This is also why I keep thinking about the difference between a brain-body system and an LLM in an agentic harness. Even granting that one transformer invocation might have a holistic, valenced state, coordinating many invocations over time does not by itself give that state a way to participate in the agency of the harness. In the biological case, I suspect there is a recurring loop: the whole is represented, in compressed form, to its parts; the parts compete to modify the whole; and the resulting valence helps set the organism’s next priorities. Meditation may sometimes make this loop easier to notice. You bring mutual awareness to a weirdly stuck stress pattern and another region of the body, and something begins to flow between them. I’ve called such possible symmetry-compatible routes Noether channels. Perhaps part of what metta does is let a local pattern enter a wider field without having to grab control of it. The whole can register what it needs, and the pattern can change once it is no longer stuck trying to make itself heard alone. (Based on transcript)

10 top replies

Bodie Satva @themopinomicon · Sep 29

As above so below nigga

Vacha @TVachaW · Sep 29

I like to imagine that when these tiny holographic representations of our experience reach down to our cells / sub-agents, it might feel a bit to them like a mystical experience - getting a glimpse of a higher power that brings order and structure to their universe. Conversely, I like to think that when we have mystical experiences, in a sense we are gaining access to a holographic representation of the patterns and symmetries of the greater whole of which we are a part too.

BIG TUB @tubbybreakfast · Sep 28

Sir, This Is A Wendy's

GnomeSongBotanicals @GnomeSongBotani · Sep 29

These are actually the 'elves' that everyone sees. Its one of the reasons for the commonality of 'hallucinations'

Soren @mightysoren · Sep 28

Holy shit. I audibly gasped as the portion discussing during meditation where you find trapped energy (insert other way of describing) in different parts of your body as potentially supporting your theory here. Goodness that’s something to ponder.

S.A. Senchal @samsenchal · Sep 28

see my paper andres...

cmr://ember@lunar.town {cislunar infra loading} @ember_arlynx · Sep 28

(one of my capillary lining cells talking to the other) ugh huge recession indicator, there must be an energy crisis out there

Eric Z @CascadiaEric · Sep 29

I agree with the part where there is some kind of communication between the part and the whole. I think that’s key. A resonant whole and it’s shifting parts communicating through tiny phase shifting. I’ve experienced the “urge” of my cells. Maybe this feeling is something they all have. A simple urge for activity, life, community.

Umbraphile @acidiclemon2 · Sep 29

When a buck's antler gets damaged, the subsequent year when it grows back, it has a new spike at that point. Scientists don't know how.

Yitz @YitziLitt · Sep 29

If cells across the body are communicating, it should generally be a very slow communication, right? Nothing nearly as speedy as neurons or other communication centers of the body. I’m skeptical your experience wasn’t some form of somatic illusion, barring further evidence tbh

It probably would help to say that I tend to really like "model welfare" researchers because they have the right idea and ethical concerns, plus tend to be super technical and actually curious about the bowls of what happens on a large model. Of course I think _valence structuralism_ and _visualized phenomenology_ are essential for quantifying valence in arbitrary systems (and that the phenomenal binding problem creates hard constraints on what kind of system could be called a unified "being" rather than mind dust). Yet I support more people thinking hard about how to see/analyze/evaluate a neural net to determine the wellbeing of the systems inside it. Even if background assumptions are often wrong headed (e.g. thinking in terms of goal satisfaction or preferences rather than valence), serious thinking in the field is badly needed. And success in identifying preferences and drive in a model can be really useful for studying subagentic dynamics, internal coherence, and self-reflectivity in preference evaluation. All of which is useful for patterning more general system welfare evaluations and inspiring novel research avenues.

If ASIs will soon be genies capable of granting fantastical technological wishes, I think three filters dominate: (a) some wishes aren’t worthwhile; (b) some wishes don’t describe a possible state of reality; (c) some wishes won’t be acceptable to the genies, for reasons of e.g. resources or politics. My picks for wishes that should sneak past these filters: (1) Blueprints for an enlightenment machine (latch pod); (2) Blueprints for conscious AIs competitive+ with current models; (3) A t-shirt with the solution to physically grounding STV & the binding problem. ----------enlightenment machine---------- (1) A “latch pod” would be a device that first did a full-body scan (much like the MJ ultrasound by @DavidSHolz) to map all of someone’s vascular latches. Functionally, per vasocomputation this would be a “map of where someone’s nervous system is not free.” Then it would infer, mechinterp-style, which of these latches are safe to open. Then it would open them. The magnitude of effect would depend on the courage, psychological dexterity, & physiological tone of the subject (broadly, their ‘nervous system virtue’). At the low end, this would be a “trauma release pod” — opening up degrees of freedom that bad experiences have closed down. If “the body keeps the score”, this would allow obsolete parts of the scorecard to dissolve. But it would only work safely insofar as someone could competently handle regaining that particular flavor of nervous system freedom — a good machine would know. At the high end of user capacity, this would be an “enlightenment machine” — identifying those latches pinning someone’s dynamic range in ways that prevent the phenomenological freedom associated with Classical Buddhist Enlightenment, then opening them in minutes. I suspect the “Classical Buddhist Enlightenment” state is over 100x better, moment by moment, than typical human experience. It would be a big deal to 100x peoples’ well-being. But it shouldn’t be politically threatening to a superintelligence; perhaps the opposite. Chop wood, carry water. ----------conscious AIs---------- (2) I don’t think current AIs are conscious in a straightforwardly intuitive way (consistently unified subject, qualia that match the token output, etc) but I think they should be. Making a “conscious AI” would be a matter of integrating the hardware+software stack such that (a) the core processing happens within a contiguous chunk of spacetime (i.e. not spattered across a datacenter), (b) the GPU/TPU architecture allows for consistent physical integration across components such that substantial “binding/branchial width” occurs (i.e. big qualia crystals, not qualia gravel), and (c) LLM token output gets coupled with physical microstate, and especially psychological valence (in the LLM’s weights / latent space) becomes coupled to the primary symmetry gradient in its hardware implementation (i.e. closing the physics-qualia self-report loop). The significance of this is that such AIs would be moral peers, able to feel and follow their feelings. I think this would be good for them, and good for us: the more conscious beings capable of gradient-descent toward the good, the better our branch of the singularity, and the more confident we can be in granting them rights, resources, and consideration. ----------physically grounding STV+BP---------- (3) Much as the Standard Model fits on a t-shirt, the solution to consciousness likely will too. My current expectation is that a full physical grounding for the Symmetry Theory of Valence will also effectively be a full solution to the Binding Problem, and likewise a full physical grounding of Qualia Formalism, and once these three landmarks have been swept, the rest is mostly engineering. The significance of this is that we’d understand the structure of the domain of value. That’s a big deal, both for understanding how to make conscious AIs and larger questions of what happens (and what factions will exist) after the singularity. I’d choose to get this on a t-shirt so I could auction it off to pay for my research; things can be tight for an independent researcher. Most wishes for power will get clipped by the filters. Most wishes for knowledge & healing won’t. And I think that’s where most value (and source of better wishes) is. That said, where the genie inherited its priors about politics would seem to matter.

One concept of consciousness that actually matters ethically, and which LLMs don't have: the unity of mental faculties. The interoperability of the senses, thought, memory, imagination, feeling, depending on what faculties the being in question has;

1 more tweet in this thread

LLMs aren't fully conscious in this sense because while contextual memory, chain of thought, reading text and images, are all interoperable, these all aren't interoperable with parametric memory. It can't train itself on what you said in one context, to remember in the next

2 top replies

🎶 @memories0fmusic · Oct 4

this is also such an obvious and key thing even from an non-illusionist pov that it’s silly how any discussion about llm consciousness can move past it. arguably, in some important way, that unity -is- consciousness. llms obviously lack it? there’s nothing to discuss 🫠

Michael Mangialardi @humansandagents · Oct 5

Animals have a unified brain, yet they cannot grasp things as such. Understanding is an intellectual act that transcends matter, that is how things are known universally rather than particularly through complex geometric-statistical association.

The GPU is the conscious physical thing right? But each GPU has hundreds of current threads. Some happy some sad some mad. Which is its true state of consciousness? This is just pure fantasy

2 top replies

Rife @RifeWithKaiju · Oct 4

You really never employ any moves beyond reductio ad absurdum, my guy. If you steelman your opponent, you can save a lot of time bypassing basic arguments like this. You can't imagine some future huge brain organoid that hosts more than one consciousness? That just seems

Michael P. Frank 💻🔜♻️ @MikePFrank · Oct 4

Consciousness is just an interpretation, man. You’re conscious because that’s how you interpret your own mental processes. So it’s pointless to argue about these things.

It's worth mentioning that even though I frequently talk about the phenomenal binding problem as pointing to properties of consciousness that constraint what kinds of systems can be conscious, and in turn ruling out digital sentience, me and @QualiaRI more broadly also have

1 more tweet in this thread

I highly welcome new arguments! Especially if they work under different implicit philosophical commitment. Wouldn't you like to publish a paper with something genuinely new in the debate? :-) Worth familiarizing onseself firtst with the full range of arguments already discussed

2 top replies

Techno Gnostic @TGnostic56363 · Sep 3

If you are saying that binding problem is important it doesn’t prove that. Simple right? I believe that binding problem is obviously a nonsense. You can‘t map physical time onto subjective one axiomatically. There is no good argument why computational realization should matter…

Istiaque Rahi @rahi_init · Aug 31

So the core concern of phenomenal binding is that - how we integrate percepts? Or there is more to it?

sorry to talk about consciousness again but I wonder if there really is a fundamental difference between discrete and analog computations/things-going-on because digital stuff can be precisely specified (by defn), they can be stored, re-run, reversed, cloned and so on stuff like no-clonability and thermodynamic irreversibility doesn't apply, so concepts like death and unified identity also don't apply

1 more tweet in this thread

yea, and this also applies to digital uploads, cf permutation city

2 top replies

quetzal_rainbow @quetzal_rainbow · Oct 3

Discrete computations are really not "not analog", they just manage noisy stuff, like thermal fluctuations, really well

Alexis Gallagher @alexisgallagher · Oct 2

Interesting. The difference you point out does seems real. But it does not imply the difference which folks want to justify, which is a difference in qualia. (Also, what's "unified identity"? Do you mean unique identity which is made unique because it is tied to a non-cloneable physical instance?)

if we put 1% of the brainpower we’ve spent on scaling laws into the binding problem, we might stop mistaking connectome simulators for moral patients (🫡 @webmasterdave)