2026-07-30 合肥物質科学研究院（HFIPS）
Illustration of outcome reward-induced shortcuts and the corrective mechanism of process supervision (Image by DING Zenghui)
＜関連情報＞
- https://english.hf.cas.cn/nr/rn/202607/t20260730_1183462.html
- https://aclanthology.org/2026.acl-long.925/
結果最適化のパラドックス：LLMにおける推論ショートカットに対する因果情報理論的限界 The Paradox of Outcome Optimization: A Causal Information-Theoretic Bound on Reasoning Shortcuts in LLMs
Zihan Chen, Yiming Zhang, Wenxiang Geng, Zenghui Ding, Yining Sun
Association for Computational Linguistics Published:July 2026
DOI:https://doi.org/10.18653/v1/2026.acl-long.925
Abstract
Large Language Models (LLMs) aligned via outcome-based Reinforcement Learning (RL) frequently exhibit a critical failure mode: they achieve high performance on in-distribution benchmarks while demonstrating brittle reasoning capabilities on out-of-distribution (OOD) tasks. We term this phenomenon Reward-Induced Manifold Collapse. We establish a theoretical framework bridging Structural Causal Models (SCM) and the Information Bottleneck (IB) principle to explain this paradox. We define reasoning as a high-complexity causal process and shortcut learning as the exploitation of low-complexity spurious correlations. Under the implicit inductive bias of Stochastic Gradient Descent (SGD), models optimized for outcome rewards are biased toward shortcut solutions whenever the training distribution allows for a “Markovian Screening” of the true causal mechanism. We derive a new generalization bound based on Semantic Coverage Measure () rather than sample size, showing why data scaling on homogeneous distributions may fail to correct reasoning flaws. We also show that Process Reward Models (PRMs) function as Topological Filters, enforcing step-wise mutual information constraints that render the low-complexity shortcut manifold inadmissible. These results provide a mathematical grounding for the role of process supervision beyond simple credit assignment.
