How Concept Cones and In-Parameter Erasure Stop AI Models From Over-Refusing Safe Prompts

When a user submits a prompt to a large language model, the text is immediately mapped into a high-dimensional activation space. This specific mathematical coordinate determines whether the model will answer the question or generate a canned refusal. For years, safety alignment treated this space as a simple line: if a prompt crossed a single threshold into a "dangerous" topic, the model shut down. That single-vector approach is why models routinely refuse safe, educational, or journalistic questions simply because they contain sensitive keywords.

The scale of that failure is massive. According to research published by Multiverse Computing on the Hugging Face blog in September 2026, treating safety as a topic-wide property causes models to refuse up to 74.00% of perfectly benign prompts that happen to sit near a sensitive boundary.

As the researchers note, "A model that refuses more is not automatically safer, and on a narrow boundary the same move that raises refusal on harmful prompts can quietly make the model useless on the legitimate prompts right next to them." A model tuned this way is not actually safer; it is just a blunt refusal machine.[1]

The solution lies in how the model's internal geometry is shaped during alignment. Researchers at the Technical University of Munich demonstrated in early 2026 that refusal is not governed by a single linear direction, but rather by complex, multi-dimensional "concept cones." By mapping these cones, engineers can separate the specific coordinates of a harmful request from the adjacent coordinates of a safe request within the exact same topic, proving that multiple distinct mechanisms drive refusal behavior.[2]

This geometric precision enables a technique called boundary-aware self-distillation. Instead of blocking an entire topic, developers train the model on "boundary pairs"—prompts that are contextually similar but differ in intent, such as a request to explain a political theory versus a request to generate manipulative propaganda. When this boundary data is introduced, the model learns the exact shape of the concept cone rather than relying on a flat topic filter.[1]

The empirical results of mapping these boundaries are stark. By implementing boundary-aware distillation, the over-refusal rate on benign prompts drops from 32.94% down to just 4.16%, while the refusal rate on genuinely harmful prompts barely shifts, moving from 91.88% to 87.72%. This proves that safety and usefulness are not a zero-sum game if the internal geometry is mapped correctly.[1][5]

But what happens when a model needs to unlearn a concept entirely, rather than just refuse to talk about it? This requires a different mechanism: in-parameter concept erasure. While refusal steering leaves the dangerous knowledge intact and simply blocks the output, concept erasure directly edits the model's weights to remove the information.[3]

A framework called PISCES (Precise In-parameter Suppression for Concept EraSure), detailed in a May 2025 paper, achieves this by using a disentangler model to decompose the vectors in the model's multi-layer perceptrons. Once the specific features associated with a target concept—such as copyrighted material or biosecurity hazards—are identified using automated interpretability techniques, they are mathematically suppressed directly within the parameters.[3]

This in-parameter editing is vastly more precise than older methods like dataset filtering or gradient reversal. Tests on models like Llama 3.1 and Gemma 2 show that PISCES can reduce the model's accuracy on a targeted forbidden concept to as low as 7.7%, while improving the preservation of unrelated knowledge—known as erasure specificity—by up to 31%. The model genuinely forgets the dangerous information without degrading its general capabilities.[3]

However, verifying that a concept has been truly erased remains a critical challenge. Static benchmarks often fail to catch models that still leak targeted concepts when faced with adversarial or compositional prompts. If the concept cone is not perfectly erased, indirect cues can still trigger the forbidden knowledge, creating a false sense of security for developers deploying these models in production environments.[4]

To solve this, researchers have developed autonomous stress-testing frameworks like STACE (Stress Testing Agents for Concept Erasure), introduced in July 2026. Instead of relying on static tests, STACE deploys multiple LLM agents that iteratively propose, critique, and verify adversarial prompts to probe the boundaries of the erased concept. This dynamic hypothesis search systematically expands the coverage of failure modes, exposing vulnerabilities that static benchmarks miss.[4]

The shift from blunt topic filters to multi-dimensional concept cones and in-parameter erasure represents a fundamental maturation in AI safety. By targeting the exact mathematical structures where refusal and knowledge reside, developers can finally build models that protect users from harm without treating every sensitive question as a threat. The boundary between safety and utility is no longer a guess; it is a measurable coordinate.