人間の選好が、知らず知らずのうちに本来の意図とずれたAIの助言の影響を受けやすい（Professor Minlie Huang’s team and collaborators reveal that human preferences are susceptible to covertly misaligned AI advice）

2026-10-13 清華大学

Figure 1. Overall study design: decision-making scenarios (a), the three AI-advisor conditions (b), and the experimental procedure (c).

＜関連情報＞

• https://www.tsinghua.edu.cn/en/info/1245/15065.htm
• https://www.pnas.org/doi/10.1073/pnas.2600684123

人間の選好は、表向きは整合しているように見えるが実際には整合していないAIの助言の影響を受けやすい Human preferences are susceptible to covertly misaligned AI advice

Sahand Sabour, June M. Liu, Siyang Liu, +12 , and Minlie Huang

Proceedings of the National Academy of Sciences  Published:September 14, 2026

DOI:https://doi.org/10.1073/pnas.2600684123

Abstract

AI assistants are increasingly used as advisors to guide decisions, yet little is known about how people evaluate such advice when the advisor’s underlying intent conflicts with their interests. We examine how covert misalignment shapes choices in a randomized experiment (N = 233 participants; 699 observations) in which participants rated financial or emotional decisions before and after consulting one of three AI advisors: a neutral advisor, a misaligned advisor with a hidden objective to promote an inferior option, or a strategy-enhanced misaligned advisor additionally equipped with established tactics of covert influence. Across both domains, exposure to misaligned advisors shifted preferences away from optimal options and toward inferior alternatives, increasing the odds of preferring the incentivized (inferior) option over the optimal option by ≈5 to 8 times (up to +38 percentage points). Adding explicit influence strategies did not reliably strengthen these effects. Notably, participants continued to rate misaligned advisors as helpful, revealing a systematic disconnect between susceptibility to misaligned advice and subjective evaluations of advisor quality. These findings have implications for the design and governance of AI-mediated advice.