An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection

Accurate detection of rapid eye movement (REM) sleep is critical for neurophysiological monitoring and early identification of sleep-related disorders. An entropy-driven adaptive knowledge distillation framework integrated with a lightweight Transformer was developed to support robust REM sleep detection from single-channel electroencephalogram (EEG) signals. By dynamically adjusting temperature scaling and distillation weighting based on teacher–student entropy, the proposed approach explicitly models uncertainty inherent in REM sleep.

When evaluated under subject-independent cross-validation on three public datasets (Sleep-EDF, ISRUC-Sleep, and MASS), the proposed model outperformed CNN, LSTM, and standard Transformer baselines, achieving an average accuracy of 93.2% and an F1 score of 92.8%. Moreover, the distilled student model reduced training time and memory usage by approximately 30%, indicating potential suitability for future optimization toward real-time and wearable EEG applications. These results suggest that uncertainty-aware adaptive distillation may support accurate and computationally efficient REM sleep detection from single-channel EEG under subject-independent experimental conditions.

This research was supported by the doctoral scholarship program at Southern Taiwan University of Science and Technology (STUST), Taiwan. The authors would like to thank Professor Shih-Chung Chen, Professor Ji-Jer Huang, colleagues in Building A, floor 3 (BioMedical Electronic Centre, BMEC), and all those who contributed their guidance, support, and valuable discussions throughout this study. Department of Electrical Engineering, Southern Taiwan University of Science and Technology, Tainan, Taiwan Ji-Jer Huang, Shih-Chung Chen & Irawan Dwiwahyono The authors declare no competing interests.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material.

If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. Huang, JJ., Chen, SC. & Dwiwahyono, I.

An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection. Sci Rep (2026). https://doi.org/10.1038/s41598-026-62389-3 DOI: https://doi.org/10.1038/s41598-026-62389-3