Document world models: a cognitive-inspired auxiliary training framework for document-level relation extraction

Document-level Relation Extraction (DocRE) requires models to integrate relation evidence scattered across multiple sentences. Yet existing encoder-based methods perform relation classification in a single pass over a static final representation, causing long-range evidence to decay before it can inform the classification decision. Inspired by theories of incremental reading comprehension, we introduce the Document World Model (DocWM), a lightweight training-only auxiliary framework that encourages progressive entity-state maintenance and relation-belief stabilization in standard DocRE encoders.

DocWM introduces two auxiliary training objectives. Entity State Tracking (ES) maintains an entity-specific recurrent hidden state, updated sentence by sentence and aligned with entity embeddings pooled from mentions up to each sentence to preserve early evidence. Relation Belief Dynamics (BD) supervises relation predictions at each reading step via a temporally weighted loss, guiding relation beliefs to stabilize progressively toward the correct label.

Experiments on Re-DocRED and HacRED show consistent improvements over the compared baselines, most clearly on long-range cross-sentence relations. This study was funded by the National Natural Science Foundation of China under Grant No. 62666063, the National Natural Science Foundation of China under Grant No. 62476233, the National Natural Science Foundation of China under Grant No. 62262065, the Natural Science Foundation of Xinjiang Province under Grant No. 2025D01C299 and 2025D01C294, the 2025 Tianchi Talents Young Doctors Fund under Grant No. 51052501823, and the Key Research and Development Project of Xinjiang Uygur Autonomous Region under Grant No. 2024B03028. School of Computer Science and Technology, Xinjiang University, Urumqi, 830017, Xinjiang Uygur Autonomous Region, China Fuyuan Wei, Wenzhong Yang, Tiquan Gu, Bowen Gu, Hongzhen Lv & Liruizhi Jia Correspondence to Wenzhong Yang, Tiquan Gu or Liruizhi Jia.

Thr authors declare no competing interests. Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material.

You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. Wei, F., Yang, W., Gu, T. et al. Document world models: a cognitive-inspired auxiliary training framework for document-level relation extraction.