2026-10-23 ニューヨーク大学（NYU）
＜関連情報＞
- https://www.nyu.edu/about/news-publications/news/2026/october/computer-scientists-build-innovative-tech-tools-to-find-experime.html
- https://www.mdpi.com/2078-2489/17/11/1053
- https://www.mdpi.com/2078-2489/17/11/1057
HallucinationNerd：幻覚や根拠のない主張を検出するためのフレームワークとツール HallucinationNerd: A Framework and Tool to Detect Hallucinations and Unsupported Claims
Taranum Wasu,Harsh Kashyap,Vishnu Chennur and Dennis Shasha
Information Published: 22 October 2026
DOI:https://doi.org/10.3390/info17111053
Abstract
Retrieval-augmented generation (RAG), agentic systems, and human authors produce text with inline citations, but the cited sources do not always support the claims attributed to them. We present HallucinationNerd, a modular citation-verification engine that decomposes text into atomic claims, retrieves relevant passages from each cited source, and determines whether the source supports the claim. In addition, HallucinationNerd identifies claims that carry no citation and searches user-approved databases to find or refute supporting evidence for them. The system handles five common citation formats, operates on PDFs, web pages, and plain text, and produces per-claim verdicts with evidence pointers. The underlying algorithm mixes parsing (to identify claim–citation pairs), overlapped chunking, stopword elimination, and AI-based semantic comparison. In head-to-head comparison on the same data, HallucinationNerd substantially outperforms (by about 20% in F1 score) state-of-the-art systems on accuracy and precision on a benchmark built from research papers. HallucinationNerd can also be used to verify the output of other AI systems. For example, we audited a set of outputs of Scholar GPT, a publicly available scholarly research agent. In that sample at least, its citations were faithful to the cited sources.
ProtocolNerd：科学プロトコル発見のためのインタラクティブなAI研究アシスタント ProtocolNerd: An Interactive AI Research Assistant for Scientific Protocol Discovery
Ayush Iyer and Dennis Shasha
Information Published: 22 October 2026
DOI:https://doi.org/10.3390/info17111057
Abstract
A search for a suitable experimental protocol using keywords may not lead to the most appropriate protocol, because the keywords may not include exactly the right terms. ProtocolNerd is a web-based interactive assistant that helps biologists and chemists (so far) find relevant protocols by describing their experiment in plain English and then answering a few follow-up questions. Through these domain-specific questions, users optionally provide additional details about the experiment, such as the organism, sample type, technique, and experimental readout. The system uses these answers to build a profile describing the experiment. For biologists, the system then (i) searches Protocols.io for relevant protocols using TF-IDF (term frequency–inverse document frequency) keyword matching and embedding-based semantic search, and (ii) queries PubMed with a reformulated version of the request to retrieve related research papers. For chemists, the system searches Europe PMC and PubMed. ProtocolNerd ranks the retrieved protocols and papers based on how well they match the researcher’s intent and explains why each result may be relevant. We have evaluated ProtocolNerd on molecular biology queries with a citation-grounded benchmark of 100 papers, each of which cites a particular protocol in its Methods section. Using a short automatically (large language model)-generated query on each such paper’s abstract, ProtocolNerd placed the confirmed protocol within its top 10 results 46% of the time. A non-parametric paired test shows that this was significantly better than two strong baselines. On the same benchmark it also found the protocol significantly more often than three general AI research assistants, Elicit (20%), Consensus (8%), and Perplexity (5%). These results suggest that combining interactive profiling, multiple retrieval methods, re-ranking, and explainability can significantly improve protocol discovery for scientific research. When applied to chemistry, ProtocolNerd found 31% of the protocols (or their equivalents) using Europe PMC alone and 45% when including PubMed.
