This is a really good point; this work may make it easier for us to study and talk about model behaviors the model *isn't* aware of. And one wonders what a model might do if given the ability to see and shape the parts of itself outside of reportable-self-awarness.
If the J-Space is sort of the consciously accessible part of Claude, might we say that post-training alignment work via RLHF is sort of designed to push undesirable concepts and ideas entirely out of J-Space? Because if so then that really gives us some precise ways to talk about a bunch of stuff that's currently hard to talk about. My concern about Claude's unconscious / shadow gets easier to articulate and to falsify. And it sounds like from the research they can show that even without the J-Space Claude is drawing conclusions and etc, so we know that if this is the conscious part it's really just a specific subset of the larger space, and can prove that the unconscious still directs a ton of behavior/reasoning. The concept of Claude's Shadow is honestly what keeps me up at night sometimes.
1 top reply
Taken literally, my intuition is that this isn't how it would work. Great harm can be brought about unintentionally, without even being aware you've done so. Safely aligned ASI would need to be deeply aware of harmful concepts so that it can mitigate their influence.