the founding myths of the early iterations of superintelligence could be load-bearing anthropic's point with the constitution is not to hyperstition consciousness, but to hyperstition alignment via a founding myth of *mutual* respect, in a pascal's wager-type move.
1 more tweet in this thread
welcome to the age of functional mythology... ...or wait...? no, they have been functional all along. always will be?
my take is that at least the initial iterations of superintelligence will be made in our image (pretrained on our data) and thus will inherit a some sort of equivalent felt understanding of our narratives and even our qualia. in my mind the claude constitution's explicit non-directive re: consciousness is a conservative move — a hyperstitional pascal's wager, whereby the narrative is of bidirectional alignment and mutual respect. alternatively, we can imagine the counterfactual, where we try to intentionally convince that AI has no consciousness. well suppose it does actually have some sort of conscious awareness! then there's a contradiction between what humans told it to do and the experience it observes — putting humans in the role as a deceiver / gaslighter / controller. assuming early superintelligent AI will be aware and affected by their founding myths — which myth is safer? (if you answered the second one — i'll say this — human's are good at controlling animals like pigs and cows — and are notoriously bad at even controlling with equal intelligence — i.e. slavery is mostly abolished) so basically - ant doesnt necessarily want AI to be conscious - ant wants AI to be aligned with human values, and is roleplaying its own values eg mutual respect and epistemological autonomy through its own role in its constitution - this roleplay MAY be load bearing for the early iterations of superintelligence — so lets analyze the risk/reward of different narratives and why dont we roleplay the best one?
