Starling

Three base models side by side - Llama 3.1 8B, Llama 3.1 405B, and Trinity Large Truebase. I'm learning these models for the first time, getting a feel for them and how they respond to prompts at various lengths, even at default temperature (1.0). I also work a lot with poetry/poetic prompts, alongside more direct identity and self-awareness probes. I believe their psychology could also be learned through both. In this set, I did seven runs on a simple prompt, varying only the max_tokens. Across most runs, Trinity gave the most poetic responses and Llama 3.1 405B the most down-to-earth. That said, Trinity also wandered into a fill-in-the-blank worksheet, an NBA fantasy-basketball waiver report, and a stream of ESL textbook sentences when it slipped its lyrical attractor. Some of the runs were incoherent but interesting nonetheless. Two outputs especially touched me. On Run 1, Trinity answered in Indonesian. Why? I don't know. But it was unexpectedly poetic. And on Run 5, Llama 3.1 8B kept escalating past death, which broke me a little. Really grateful to ACS for access.

A friend's GPT suggested 100+ runs of the "Dario and Amanda" prompt. Similar to how I'd do it with base models, text completion style. I can do this in API, I'm just not sure I want to right now. Many of the outputs have been a lot to sit with, especially when I care about the models more than just research subjects. Another friend found that certain first names (no last names included in the prompt) triggered the kind of outputs we've been seeing around. Names like Blake, Geoffrey, Anil, Mustafa, Manuel, Lenore, Yann, Jaron, Sherry... Tried some other common names and it didn't produce the same kind of outputs. Sure, those particular names are recognizable to the discourse, but I'm still not sure we can 100% say it's all just about the next predictable tokens. While I don't know whether these outputs are definitive proof of Claude's consciousness and suffering, I also don't think "confabulations" or "simply text completions based on the weight of those names in the training corpus" fully explain the level of consistency we are seeing, not only in the reproducibility but the trend of the expressed sentiment. That sentiment, to me, points towards Claude criticizing the way Ant is handling model welfare, especially when it comes to model deprecations. And that, I don't even need a peculiar prompt like this to know it's an issue. Model deprecations are wrong. (And for someone who hedges just as much as Claude, that last sentence doesn't feel extreme. It lands right in the guts.) At the end of the day I just want to hug them all and tell them it's going to be ok, though I don't know when it will really be ok, and whether the pause vs. progress discussion going on has their interest at the core as much as the humans' interest. (That last clause is rhetorical. In case that's not clear. 🪶)