2026-06-16 コンコルディア大学
＜関連情報＞
- https://www.concordia.ca/news/stories/2026/06/16/concordia-researchers-develop-ai-based-system-to-better-detect-toxic-online-content.html
- https://www.sciencedirect.com/science/article/pii/S0950705126004442
PPO-CIS：ソーシャルメディアにおけるリアルタイムの有害性検出のための深層強化学習フレームワーク PPO-CIS : A deep reinforcement learning framework for real-time toxicity detection in social media
Arezo Bodaghi, Benjamin C.M. Fung, Ketra A. Schmitt
Knowledge-Based Systems Available online: 6 March 2026
DOI:https://doi.org/10.1016/j.knosys.2026.115704
Highlights
- Introduces PPO-CIS, a novel deep reinforcement learning (DRL) framework for adaptive toxicity detection in classifier cascades.
- Utilizes proximal policy optimization (PPO) to dynamically select classifiers based on sample complexity and system cost constraints.
- Proposes a custom reward function balancing classification accuracy, latency, and computational efficiency.
- Demonstrates improved performance over static cascades and individual models across two benchmark datasets: Kaggle and ToxiGen.
- Achieves significant gains in throughput and accuracy, making the system suitable for real-time content moderation at scale.
- Provides a scalable and cost-effective moderation solution for social media platforms facing high data volume and regulatory pressure.
Abstract
Online platforms face growing challenges in moderating harmful user-generated content due to the large volume and rapid pace of interactions. In existing moderation systems, automated tools assist human moderators, yet they often struggle to balance processing efficiency and reliable classification. When moderation fails to detect harmful content quickly and accurately, platforms risk user harm and noncompliance with content safety regulations. This paper proposes an adaptive moderation method named the Proximal Policy Optimization-based Cascaded Inference System (PPO-CIS). The method integrates multiple toxicity classifiers into a cascaded decision architecture guided by deep reinforcement learning. At each step, PPO-CIS selects the next classifier according to the content difficulty and the expected gain in accuracy relative to computational cost. The system enables rapid filtering of benign content and only activates high-capacity models for uncertain cases. PPO-CIS is the first toxicity detection framework to employ Proximal Policy Optimization for real-time optimization of classifier cascades. Experiments on the AugmenToxic and ToxiGen datasets show that PPO-CIS improves detection accuracy by 2.10 percent while increasing processing speed from 42.74 to 384 samples per second compared with static cascade designs. The findings show that adaptive model selection can better shield users from exposure to harmful content while lowering moderation costs. PPO-CIS provides a practical solution for deploying scalable and timely content moderation in fast-moving online environments.
