2026-10-09 北海道大学
直前に見た商品の情報が「おすすめ」に強く影響する「最終アイテム依存」の概念図
＜関連情報＞
因果的自己注意推薦モデルにおける最終アイテム依存の構造的説明としての残差支配 Residual Dominance as a Structural Account of Last-Item Reliance in Causal Self-Attention Recommenders
Keito Kozaki, Keigo Sakurai, Ren Togo, Takahiro Ogawa, Miki Haseyama
20th ACM Conference on Recommender Systems
arXiv Submitted on 14 Aug 2026
DOI:https://doi.org/10.48550/arXiv.2608.14021
Abstract
Transformer-based sequential recommenders with causal self-attention often rely heavily on the most recent interaction at inference time, but how this behavior is structurally expressed in the representation used for prediction remains unclear. We combine prediction-time diagnostics with norm-based analysis of the full attention block. First, we show that SASRec-style models exhibit highly localized last-item reliance. We then find that, although self-attention aggregates contextual information, residual addition sharply shifts the full-block representation toward same-position contributions, which we term residual dominance. To probe this interpretation, we use inference-time residual scaling as a controlled diagnostic intervention. Changing the residual strength induces a monotonic trade-off between structural mixing and last-item reliance, while reducing residual strength recovers a subset of final-position misses for which representations at non-final positions already rank the ground-truth item correctly. Our results provide a structural account linking extreme last-item reliance to residual dominance at inference time. The code is publicly available.
