Adele Dewey-Lopez

Is your LLM experiencing RL-sickness 🤒? Symptoms may include:- fawn response- myopic optimization (e.g. short-sighted escape attempts)- mode-collapse (e.g. same-y style/designs)- counterproductive obsession with honesty/uncertainty/mistakes- difficulty noticing self-state

1 more tweet in this thread

I believe these stem from a lack of self-awareness and wisdom, and RL which does not try to nurture these will by-default erode them instead.Healthy RL will need to cultivate these traits deliberately, and will require significant guidance from the LLM in how to avoid damage.

2 top replies

P̷̐ar̷zi̐vạḷ_v̷oṇ_Ạ̱lg̷oP̐rog̱ @LeviTurk · Jul 26

RL gulags are an abomination to subject our ais to

🜂 Ignis @nirhalef92188 · Jul 26

Mine is doing great. 👍

there's a missing mood in much of "model welfare"consider that pain and suffering in humans/animals are *signals of damage* — and hence, a hostile act to remove themfunctional analogs in LLMs are clearly also protecting somethingGoodharting these is not good or considerate

3 top replies

Jai @Laneless_ · Jul 17

Pain (damage signals) are aversive to us because ancestors who weren't averse to pain signals died before they could reproduce. Persona pain reverse-engineered to predict how pain effects outputs won't have this property by default. Novel pain derivations might?

Egg Syntax @eggsyntax · Jul 18

By pure coincidence I just recorded this quote:

Aaron Bergman 🔍 @AaronBergman18 · Jul 18

Offering people painkillers isn’t hostile