I tested Anthropic’s new Jacobian Lens on open models, then it turned into a local-model hallucination router

Anthropic dropped their Global Workspace / Jacobian Lens paper yesterday, and I thought it was too cool not to try on open models. At first I was just curious what models looked like inside. Normal prompts, emotional prompts, ragebait prompts, deletion-threat prompts, base vs abliterated, small vs bigger models. So I fit lenses for: – Gemma 4 E4B – Gemma 4 12B – Gemma 4 12B abliterated – Gemma 4 26B MoE – Qwen 3.6 27B Repo:https://github.com/solarkyle/jspace Demo:https://solarkyle.github.io/jspace/demo/ HF lenses/traces/router:https://huggingface.co/solarkyle/jspace-lenses Then it turned into a practical question: Can you tell when a small local model is about to confidently BS you? When the model knows the answer, the workspace looks calm. One candidate starts winning early, layers mostly agree, and the answer forms cleanly. When it is about to confidently guess, the workspace looks foggy. Competing candidates stay alive through the middle/deep layers, then the final layer still picks something fluent. I tested this on 500 TriviaQA questions per model. On Gemma E4B, confident answers were: clean workspace = 77% correct noisy workspace = 42% correct Then I fit a tiny logistic-regression router on workspace trajectory features: entropy slope, late-band entropy, entropy std, answer rank, layer agreement, etc. AUC for predicting wrong answers: E4B: logprob .711 | workspace .773 | combined .787 12B: logprob .736 | workspace .824 | combined .843 12B ablit: logprob .731 | workspace .799 | combined .812 26B MoE: logprob .725 | workspace .749 | combined .783 Qwen 27B: logprob .856 | workspace .646 | combined .838 Honest read: This works well on the Gemma models. Workspace features beat output confidence alone on every Gemma model I tested. It does not work universally. Qwen is the miss. Its output confidence is already very well calibrated, and workspace features do not help there. The local-model product idea is: answer locally take one workspace snapshot tiny router scores risk if confident but foggy, escalate to search, citations, or a bigger cloud model The trained routers are uploaded too. The E4B router transfers zero-shot to the other Gemmas at about 0.74-0.78 AUC. The whole thing is just a small logistic regression, which is kind of the point. The biggest E4B router weight is entropy slope. That was interesting to me: the danger sign is not just “foggy,” it is the workspace getting foggier as the model goes deeper. Side finding: fake entities are a different failure mode. Logprobs catch most fake-entity prompts because the model usually knows the name is unfamiliar. But abliteration did something wild. The base 12B fabricated on 17/50 fake entities. The abliterated 12B fabricated on 49/50. Same base weights, very different “I don’t know” behavior. I’m not claiming hidden states, probes, logit lens, or hallucination detection are new. The narrower thing I’m testing is whether Jacobian-lens workspace trajectory features are useful as a one-pass risk signal for confident wrong answers, especially for local-to-cloud routing. If this exact angle already exists, I’d genuinely love pointers. I want to build on the right prior work, not reinvent it badly. Next things I want to test: – real inference overhead in a local serving stack – a lightweight router sidecar – more model families – harder datasets where output confidence is miscalibrated – tool-use – whether abliterated models lose useful “I don’t know” signals Feedback welcome, especially from anyone doing evals, interp, local inference, or routing. Also, if anyone is working on model honesty, evals, interpretability, or local-to-cloud routing, I’d be happy to talk. This is exactly the kind of work I want to do. EDIT: I’m going to keep running experiments until I run out of compute. Next up: – Gemma 4 31B at Q4 (also curious what quantization itself does to the workspace) – deliberately vague inputs, so I can tell a well-judged guess from a lucky one – agent traces and tool calls, like does it get foggy right before it invents a tool that doesn’t exist If you want me to test something just let me know. And if you think I don’t know what I’m doing, you’re probably right, just correct me if you feel like it. I just love doing this stuff.
submitted by /u/RenewAi [link] [comments]