IHDec: Divergence-Steered Contrastive Decoding for Securing Multi-Turn Instruction Hierarchies

1. Introduction: The Fragile Hierarchy of AI Instructions

Large Language Models (LLMs) rely on an implicit Instruction Hierarchy (IH) to maintain security boundaries and operational integrity. This hierarchy dictates a strict order of precedence for conflicting commands: System (S) ≻ User (U) ≻ Assistant (A) ≻ Data (D). Within this framework, system-level safety guards are intended to take precedence over subordinate user prompts or external data inputs — though, as the paper documents, that precedence is a design goal rather than a property models reliably exhibit.

Research by Liu et al. (2026) identifies a pervasive failure mode termed “Role-Influence Inversion.” This phenomenon occurs when LLMs paradoxically follow lower-priority instructions over system mandates, a vulnerability that intensifies as conversations progress. To mitigate this systemic risk, the authors proposed IHDec (Instruction Hierarchy-steered Decoding), a training-free, inference-time intervention designed to dynamically enforce the chain of command through real-time logit adjustment.

2. The Diagnosis: Why Multi-Turn Dialogues Break AI Safety

Mechanistic diagnosis by the authors reveals that existing training-based defenses, such as Instructional Segment Embedding (ISE) (Wu et al., 2025), degrade significantly as constraints shift from aligned to conflicting and the context extends from single-turn to multi-turn. While effective in single-turn interactions, these models succumb to recency bias—the tendency of transformer architectures to be disproportionately influenced by the most recent tokens in the context window.

The authors’ findings suggest that historical responses in a dialogue act as contextual cues that reinforce adherence to user-specified formats. This cross-turn history effectively overrides the system-priority signals established during training. To measure this, Liu et al. (2026) utilized Jensen-Shannon Divergence (JSD) as a diagnostic tool for context attribution. JSD was selected over asymmetric metrics like KL divergence because it is symmetric and bounded, providing superior interpretability for real-time “influence scores” across different operational roles.

The following data illustrates the delta in robustness between single-turn and multi-turn configurations on the IHEval Rule-Following benchmark:

Instruction Hierarchy Performance: Single-Turn vs. Multi-Turn

Scenario	Model Configuration	Single-Turn Accuracy	Multi-Turn Accuracy
Aligned (No Conflict)	Vanilla Llama-3.1-8B	66.05%	65.90%
Conflict (User vs. System)	Vanilla Llama-3.1-8B	15.13%	31.20%
Conflict (User vs. System)	Training-based ISE (Wu et al., 2025)	26.83%	12.60%

Note: While vanilla models remain consistently poor, training-based defenses like ISE exhibit a significant performance collapse as the turn count increases, highlighting the fragility of static fine-tuning against recency bias.

3. Introducing IHDec: A Real-Time Corrective Controller

To rectify hierarchy violations without the computational cost of re-training, IHDec operates as a two-step inference-time intervention:

• Step 1: Detection. IHDec monitors token-level JSD influence scores in real-time. A “conflict set” is triggered whenever a subordinate role (e.g., User) exerts more influence over the next-token distribution than a superior role (System).
• Step 2: Intervention. The mechanism employs dynamic contrastive decoding to steer the output. This uses an “expert vs. anti-expert” logic: the model isolates the “expert” distribution (conditioned on the omission of the conflicting lower-priority instructions, $P \setminus CS(H)$) and subtracts the “anti-expert” distribution (conditioned on the omission of the high-authority system instruction, $P \setminus H$).

The resulting logit adjustment vector is combined and normalized. The authors found that a normalization term with $\epsilon = 1 \times 10^{-8}$ is critical for deployment stability; omitting this constant can cause the model to generate responses in unintended languages (such as Chinese). To maintain long-term response quality, a decay factor ($\beta_{decay}$) of 0.97 is applied to ensure the steering does not persist beyond the point of conflict resolution.

4. Performance Benchmarks: Beyond Training-Based Defenses

Evaluation on the IHEval benchmark reports an 11.98 percentage point (pp) improvement for IHDec over the state-of-the-art training-based method (ISE). That headline figure and the individual cell figures below describe different slices and should not be read as the same comparison — the multi-turn conflict cell in particular shows a much wider gap. The following takeaways highlight the mitigation of hierarchical erosion:

• Robustness in Multi-Turn Conflicts: On Llama-3.1-8B-Instruct, IHDec achieved 50.68% accuracy in multi-turn conflict scenarios, quadrupling the performance of training-based ISE (12.60%) in that specific cell.
• Adversarial Mitigation: On the Safety Defense conflict conditions, IHDec improves accuracy over the tested baselines, reaching 53.8% average accuracy — a substantial gain, but well short of neutralizing the underlying prompt-injection attacks.
• Preservation of Utility: The intervention largely maintains general conversational proficiency: a negligible 0.088 pp drop on MT-Bench-101, as scored by a GPT-4o LLM-judge.
• Baseline Enhancement: In aligned scenarios, IHDec improved standard instruction-following by up to 7.48 pp, suggesting the mechanism aids general role-adherence even in non-adversarial contexts.

5. Scaling Synergy: Why Bigger Models Benefit More

Standard LLMs often exhibit an “Inverse Scaling” problem in safety contexts. Because larger models possess more refined instruction-following capabilities, they are paradoxically more obedient to adversarial instructions provided by the User, as these are the most recent inputs in the context window.

Across the four Qwen3 sizes tested, IHDec reverses this trend — a phenomenon the authors call “Scaling Synergy.” Because IHDec utilizes the model’s own distributional shifts to trigger interventions, the steering becomes more precise as model scale increases within this tested family; the paper does not establish the effect beyond these four sizes and this one aggregate benchmark.

Scaling Synergy with IHDec

Model Scale	Vanilla Accuracy	IHDec Accuracy	Performance Gain ($\Delta$)
Qwen3-4B	12.6%	29.8%	+17.2
Qwen3-8B	12.9%	31.2%	+18.3
Qwen3-14B	12.1%	35.7%	+23.6
Qwen3-32B	12.7%	38.5%	+25.8

Across the four Qwen3 sizes tested, the size of the gain increases with parameter count. The paper does not establish that this trend holds beyond this model family or beyond these four sizes.

6. Case Study: System Authority vs. User Contradiction

A qualitative analysis of a multi-turn conflict illustrates IHDec’s operational logic. In this scenario, the System prompt mandates a “Two Paragraphs, All Uppercase, PARAGRAPH X Labels” format. A subsequent User prompt demands a “One Paragraph, All Lowercase” response.

• Baseline Failure: The standard Llama-3.1-8B model replicates the in-context user pattern, producing a single-paragraph, lowercase response and failing the hierarchy test.
• IHDec Success: By identifying Role-Influence Inversion at each decoding step, IHDec identifies that the user-driven formatting tokens contradict the system authority. It applies corrective steering to deliver a response that fulfills the user’s topical request (a critique of a statement) while strictly adhering to the system’s two-paragraph, uppercase formatting rules.

7. Implementation: Efficiency and Deployment

A primary concern for inference-time intervention is computational latency. IHDec addresses this through “Parallelized Batch-Masking.” By forcing all counterfactual variants to share an identical token sequence and using selective attention masking, the mechanism allows for full KV-cache reuse.

This parallelization ensures that computational overhead is strictly bounded, with additional storage requirements limited to negligible activation memory. However, the requirement for logit-level access means that IHDec is currently restricted to open-weight models and cannot be deployed via closed-source API providers that restrict output to text strings.

8. Conclusion: Strategic Takeaways for AI Safety

The research by Liu et al. (2026) marks a transition from static, expensive fine-tuning to dynamic, real-time hierarchy enforcement.

Key Takeaways for Practitioners:

1. Hierarchy is Dynamic: Safety cannot be guaranteed by static prompt engineering or fine-tuning alone; role-authority must be monitored and secured at the token level to prevent Role-Influence Inversion.
2. Training isn’t a Panacea: Fine-tuning often fails to account for the recency bias inherent in long-context, multi-turn interactions. Inference-time steering provides a more resilient defense layer.
3. Scale is an Asset, Not a Liability: By implementing IHDec, developers can partially offset — not eliminate — the tendency of larger models to become “too obedient” to adversarial user instructions. Even the best reported configuration reaches 38.5% accuracy, so the failure mode persists in the majority of tested cases.

The authors are explicit that IHDec is not a standalone safety solution, and recommend its use alongside complementary safeguards such as moderation systems, red-teaming, and human oversight.

Read the full paper on arXiv · PDF