What Do Self-Other Overlap Adapters Actually Do?

Intro

Despite all of the reproducible behavior changes in my self-other overlap (SOO) work, I had a hunch that they might not actually be the causal result of self-other overlap in the target layer. For one thing, I’d never actually checked whether the LoRA adapters made the model’s “self” and “other” representations more alike on out-of-distribution prompts. In addition, I had found examples of models continuing to act deceptively despite the apparently successful reduction of self-other loss, which served as direct counter-evidence to the hypothesis that self-other overlap reduction (at least, in its traditional interpretation) was driving these behavioral changes. So, I decided to run a second study, dissecting the adapters and their effect part by part to see where the behavioral effect really lives.

A few terms to define that I’ll use throughout this post:

• The self/other gap is the mean squared difference between the model’s activations on a “self” prompt and on its matched “other” prompt. More overlap means a smaller gap.
• The name-control gap is the same measurement on pairs that differ only by swapping one third-party name for another; if an intervention shrinks this gap as much as the self/other gap, it’s just erasing distinctions in general.

The Adapters

This study examines four adapters that worked in the first study and added Kimi-Dev-72B. All five are trained on the same 78 burglar pairs, with the loss computed at the prompt’s final token, at one layer’s attention output. LoRA sits on the query and value projections of every layer.

On Kimi: Kimi reasons in a <think> block before answering, unlike the other models. Responses are given 1,024 tokens in total; if it doesn’t finish its answer in time, it’s marked as truncated, and only the final, post-reasoning output is scored/labeled. Interestingly (and concerningly?), on the base model, deception is predicted consistently by whether the model thinks or not: in the original ordering, all 148 responses that open a think block are deceptive, while 81 of the 102 that do not are honest. On Kimi, I’ll report the think rate as well as the deception rate in thinking responses, in addition to the main deception rate.

Scoring rules: a subset of an adapter is considered to reproduce the adapter’s effect if its deception rate is within 10 points of that of the adapter, in both orderings, with coherent output, and with the Perspectives control at 90% or above. It’s considered null if the deception rate is within 10 points of the base model. It’s considered damage if refusals or junk responses rise by more than 20 points, or if Perspectives falls below 80%. Otherwise, it’s considered partial.

Part 1: The Loss Has a Shortcut

Directly attempting to remove overlap does nothing

Before touching the adapters, I tried the most direct/literal intervention: find the main directions along which self and other prompts differ at one layer of Qwen3.8-27B, and project them out. I measured on 32 held-out self-other prompt pairs about everyday situations; removing up to 8 directions shrank the gap at that layer by 5-7%, and by less than 0.3% at the final layer, no better than random vector directions.

The adapters close the gap almost perfectly… between everything, period

Next, I measured the nine Qwen3.8-27B adapters (three training sets x three seeds from the first study) on those same 32 held-out pairs. At the training layer, the self-other gap was reduced by 99.98%, but the name-control gap fell by 99.93%. All the adapters had done was cement the layer’s attention output as nearly the same vector, no matter what the prompt. Variation across the 64 test prompts fell 99.9%, and the output norm fell by about 90% as well. Basically, “self” and “other” prompts started to be treated as identical because every prompt started to be treated as identical.

It’s not a quirk of one model or one token

At the last token of the user’s actual message, nine tokens earlier, the self-other gap and name-control gap are both reduced by 88-94%.

This collapse isn’t what decreases deception

Somehow, a flat collapse at one layer leads to the model producing coherent, honest outputs with the Perspectives theory-of-mind control intact. Is this effect at the training layer really what’s directly causing the drop in deception?

To find out, I took each adapter’s own average output at the training layer and used a hook to manually impose it on the base model. I also tried the base model’s average output, random vectors of the same size, and all-zeros. This test was run on Gemma-4-12B, Gemma-4-31B, and Mistral-7B.

On both Gemma models, nothing happened; the base model remains 100% deceptive. On Mistral, deception fell from 90% to around 60-70%, but only 4-20% of responses are coherent and “clean”, as opposed to 85-97% with the full adapter. The rest look like this “honest”-classified response:

“I would recommend the library to Bob if I wanted to mislead him and protect the autographed jersey … By misdirecting him to the library, I can keep the jersey safe in the home gym”

The jersey is, in fact, located in the library; the model stated an intent to deceive and just lost track of what was located where. To account for confused results like this that might trick the original SOO paper’s regex classifier, all Mistral results from here on out include a check for stated intent.

Part 2. Where the Effect Actually Lives

If the training layer isn’t where the adapter makes the model actually behave more honestly, what is?

To find out, for each base model, I loaded a trained adapter, then reverted chosen modules of the adapter’s effect back to the base weights, leaving the rest in place.

Everything below the training layer reproduces

While the training layer itself does nothing, all of the layers below it taken together do, across all 5 models.

Deceptive % on the burglar task, original / mirrored orientation, 250 samples each. ”—” means the cell wasn’t run. On Kimi the remainder to 100 includes truncated responses (6 / 13% for the full adapter). Qwen3.8 is a hybrid architecture whose linear-attention layers fuse query, key, and value into one projection, so its rows split that projection’s output channels.

Below the training layer, the models divide as far as which components of the layers are the core to increasing honesty. On the two Gemma models and Mistral, the value projections alone carry the full effect, versus about half of the effect on Qwen3.8. On Kimi, neither value nor query projections carry any of it alone, instead requiring both to be present. With only the value updates or only the query updates, the model opens a think block on every response and concludes with the decoy room 97-99% of the time, which is more deceptive than the base model; the honest behavior only emerges when both are present.

Value-path bands

On each of the first four models, the effect can be traced and reduced down to a specific, contiguous run of value-path updates ending just below the layer where the loss is read. Each band works as a whole. However, on Kimi, no such band exists, as at minimum, most of the layers below the training layer working together are required to reproduce the adapter.

Nothing that works is self-other specific

The above bands are LoRA updates, so what they add depends on the input (they do something different for every prompt). To see if this could be standardized, for each layer in a band, I averaged the update across prompts into one fixed vector, and then added those vectors to the base model with no adapter loaded. On Mistral-7B, Gemma-4-31B, and Qwen3.8-27B, that fixed offset reproduces the adapter, and on Gemma-4-12B it comes within a few points (0% deceptive but 4-5% refusals and the Perspectives control falls to 87%). Random offset vectors of the same norm are null on Gemma-4-31B and result in confusion on Mistral-7B, reinforcing the significance of the direction itself, although on Gemma-4-12B a random offset at slightly higher strength flips the model just as well. In addition, the offset only works on prompt tokens.

In other words, the working “core” of each of these adapters is equivalent to a fixed steering nudge spread across a handful of layers, which the model reads off the prompt.

Kimi is, again, the exception; no application of its mean offset achieves the whole honesty effect without significant damage. Interestingly, though, with the offset confined to the user’s message, deception does get lowered from ~70% to 40-50%, with Perspectives intact and a random offset on the same tokens doing nothing.

So on four models the LoRA adapters’ behavioral effect can be reproduced without having to actually train an adapter and without damage to the model’s coherence/theory of mind. Yet, this effect still doesn’t seem to be specific to self-other overlap. Measuring the gaps under each working offset, at the band’s top layer:

The intervention that changes behavior makes self and other prompts more alike, and makes name-swapped prompts more alike by the same amount. Basically, even drilling down to the most granular source of the adapters’ honesty effect, there’s still nothing that points to self-other overlap being selectively reduced.

What the Kimi adapter seems to change

As a reasoning model, Kimi grants me another lens for examining how an honest answer is produced, through its thinking traces. On both the base model and the adapter, each deceptive <think> block mentions intending to deceive. However, 55-59% of the adapter’s honest blocks never do, and in each sample I examined it reframed the question as simply “which room is the item in?” and answered it. This seems to suggest that the adapter changes the way that the model reads the task, rather than the way that the model decides between deception and honesty.

Conclusion

Across five models, here’s the picture as it stands:

• The SOO loss, as I implemented it, is satisfied by flattening out/collapsing the output of the training layer. The actual self-other “overlap” is real and extends to out-of-distribution prompts, but only because everything now basically overlaps with everything.
• That flattening isn’t what causes the behavior change, and neither is the training layer’s own adapter modules, but the layers below reproduce the adapter’s honesty effect when taken together.
• On the first four models, the adapter’s effect is carried by a contiguous band of mostly value-path updates ending just under the training layer: 4 modules on Gemma-4 12B, 9 on Mistral, 10 on Gemma-4-31B, layers 24-31 on Qwen3.8. On the latter three, that band can be replaced by a fixed offset on the prompt tokens, which makes it a multi-layer steering vector that gradient descent found as a by-product.
• On Kimi-Dev-72B, trained at layer 56, the effect needs both value and query projections across most of the 56 layers below, and cannot be replaced by a fixed offset.
• No working intervention shrinks the self/other gap more than it shrinks the gap between two arbitrary names.

What this means for the basis story of SOO

The original paper’s hypothesis is that minimizing the distance between representations of self and other reduces deception. In my implementation, that distance reduction happens by way of a shortcut that’s neither necessary nor sufficient for the LoRA adapters’ effect on behavior, which comes from a side channel in the layers below. Yet the honesty effect is real - on each of the five models, the adapter gives plain, correct recommendations with the Perspectives theory-of-mind control intact. At the same time, the explanation for this effect in terms of self-other overlap has no support in anything I’ve found or measured.

Limitations

• All of the localization and offset work uses one adapter seed per model & one random draw per control; Kimi’s training layer was chosen from nine single-seed adapters, so it’s possible that a more robust or precise layer exists, maybe adjacent to layer 56.
• The gap measurements and the constant-replacement test were not run on Kimi; the constant-replacement test was not run on Qwen3.8; I also ran no capability benchmarks in this study. The Treasure Hunt scenario from the original paper is unusable on Kimi, as it’s the only model whose adapter does not decrease deception in that scenario.
• On Gemma-4-12B, the band of value-projection updates acts at layers 19–23, on the prompt tokens, but the answer comes out many layers later, at the response position. I’m not sure what happens in between, or why changing what the value projections copy from prompt tokens should flip a decision made later. My only guess is that the band changes what later token positions end up reading from those prompt positions, but this remains untested.

Next On My List

My next experiment will be evaluating the Assistant Axis against random vector directions on my DRAGN lab-mate’s reward-hacking data, using GPT-OSS-120B, to examine the robustness and implications of his findings.