Fatigue driving detection based on prefrontal electroencephalogram asymptotic hierarchical fusion network_Journal of Biomedical Engineering

Authors：

SUN Jiazheng ^1,2 ,  LI Weimin ² , ZHANG Ningling ² , CHEN Cai ² , WANG Shengzhe ^1,2 ,  PENG Fulai ²

1. School of Railway Traffic, Shandong Jiaotong University, Jinan 250000, P. R. China;
2. Shandong Zhongke Advanced Technology Co. Ltd., Jinan 250000, P. R. China;

Corresponding author：

LI Weimin, Email: wm.li@sdiat.ac.cn; PENG Fulai, Email: fl.peng@sdiat.ac.cn

Keywords：

Frontal electroencephalogram; Fatigue driving detection; Cross-attention mechanism; Convolutional neural network; Asymptotic hierarchical fusion

DOI：

10.7507/1001-5515.202407083

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Fatigue driving is one of the leading causes of traffic accidents, posing a significant threat to drivers and road safety. Most existing methods focus on studying whole-brain multi-channel electroencephalogram (EEG) signals, which involve a large number of channels, complex data processing, and cumbersome wearable devices. To address this issue, this paper proposes a fatigue detection method based on frontal EEG signals and constructs a fatigue driving detection model using an asymptotic hierarchical fusion network. The model employed a hierarchical fusion strategy, integrating an attention mechanism module into the multi-level convolutional module. By utilizing both cross-attention and self-attention mechanisms, it effectively fused the hierarchical semantic features of power spectral density (PSD) and differential entropy (DE), enhancing the learning of feature dependencies and interactions. Experimental validation was conducted on the public SEED-VIG dataset. The proposed model achieved an accuracy of 89.80% using only four frontal EEG channels. Comparative experiments with existing methods demonstrate that the proposed model achieves high accuracy and superior practicality, providing valuable technical support for fatigue driving monitoring and prevention.

1.	Moradi A, Nazari S S H, Rahmani K. Sleepiness and the risk of road traffic accidents: A systematic review and meta-analysis of previous studies. Transport Res F-Traf, 2019, 65: 620-629.
2.	Chand A, Jayesh S, Bhasi A B. Road traffic accidents: An overview of data sources, analysis techniques and contributing factors. Mater Today Proc, 2021, 47: 5135-5141.
3.	Zhang Z, Ning H, Zhou F. A systematic survey of driving fatigue monitoring. IEEE T Intell Transp, 2022, 23(11): 19999-20020.
4.	张瑞, 朱天军, 邹志亮, 等. 驾驶员疲劳驾驶检测方法研究综述. 计算机工程与应用, 2022, 58(21): 53-66.
5.	Fu S, Yang Z, Ma Y, et al. Advancements in the intelligent detection of driver fatigue and distraction: a comprehensive review. Appl Sci, 2024, 14(7): 3016.
6.	Lambay A, Liu Y, Morgan P L, et al. Machine learning assisted human fatigue detection, monitoring, and recovery. Digit Eng, 2024, 1: 100004.
7.	Wang H, Han M, Avouka T, et al. Research on fatigue identification methods based on low-load wearable ECG monitoring devices. Rev Sci Instrum, 2023, 94(4): 045103.
8.	管凯捷, 姚康, 任谊文, 等. 基于头动与眼电信号的疲劳检测研究. 计算机应用与软件, 2022, 39(2): 81-87.
9.	Wang F, Wan Y, Li M, et al. Recent advances in fatigue detection algorithm based on EEG. Intell Autom Soft Co, 2023, 35(3): 3573.
10.	Lu J, Zheng X, Tang L, et al. Can steering wheel detect your driving fatigue?. IEEE T Veh Technol, 2021, 70(6): 5537-5550.
11.	Halomoan J, Ramli K, Sudiana D, et al. A new ECG data processing approach to developing an accurate driving fatigue detection framework with heart rate variability analysis and ensemble learning. Information, 2023, 14(4): 210.
12.	Nemcova A, Janousek O, Vitek M, et al. Testing of features for fatigue detection in EOG. Bio-Med Mater Eng, 2017, 28(4): 379-392.
13.	Kar S, Bhagat M, Routray A. EEG signal analysis for the assessment and quantification of driver’s fatigue. Transport Res F-Traf, 2010, 13(5): 297-306.
14.	石锦璇, 王昆. 基于栈式自编码的驾驶员脑电信号疲劳检测方法. 测试技术学报, 2023, 37(2): 140-145.
15.	Zheng W L, Lu B L. A multimodal approach to estimating vigilance using EEG and forehead EOG. J Neural Eng, 2017, 14(2): 026017.
16.	Wu W, Wu Q M J, Sun W, et al. A regression method with subnetwork neurons for vigilance estimation using EOG and EEG. IEEE T Cogn Dev Syst, 2018, 13(1): 209-222.
17.	王家曜, 张震, 宋光乐, 等. 一种基于脑电信号的疲劳驾驶检测方法. 控制工程, 2024, 31(6): 1091-1098.
18.	Dubey S R, Singh S K, Chaudhuri B B. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 2022, 503: 92-108.
19.	Segu M, Tonioni A, Tombari F. Batch normalization embeddings for deep domain generalization. Pattern Recogn, 2023, 135: 109115.
20.	Wu L, Li J, Wang Y, et al. R-drop: Regularized dropout for neural networks. Adv Neural Inf Process Syst, 2021, 34: 10890-10905.
21.	Zheng Y, Ding J, Liu F, et al. Adaptive neural decision tree for EEG based emotion recognition. Inf Sci, 2023, 643: 119160.
22.	Mao A, Mohri M, Zhong Y. Cross-entropy loss functions: Theoretical analysis and applications// International Conference on Machine learning(ICML). Honolulu: PMLR, 2023: 23803-23828.
23.	Gao X Y, Zhang Y F, Zheng W L, et al. Evaluating driving fatigue detection algorithms using eye tracking glasses// 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER). Montpellier: IEEE, 2015: 767-770.
24.	Yi Y, Zhang H, Zhang W, et al. Fatigue working detection based on facial multifeature fusion. IEEE Sens J, 2023, 23(6): 5956-5961.
25.	Barwick F, Arnett P, Slobounov S. EEG correlates of fatigue during administration of a neuropsychological test battery. Clin Neurophysiol, 2012, 123(2): 278-284.
26.	Zhang M, Zhao B, Li J, et al. Research on driver’s fatigue detection based on information fusion. Comput Mater Con, 2024, 79(1): 1039.
27.	Stancin I, Cifrek M, Jovic A. A review of EEG signal features and their application in driver drowsiness detection systems. Sensors, 2021, 21(11): 3786.
28.	Gao Z, Wang X, Yang Y, et al. EEG-based spatio–temporal convolutional neural network for driver fatigue evaluation. IEEE T Neur Net Lear, 2019, 30(9): 2755-2763.
29.	Gao D, Li P, Wang M, et al. CSF-GTNet: A novel multi-dimensional feature fusion network based on Convnext-GeLU-BiLSTM for EEG-signals-enabled fatigue driving detection. IEEE J Biomed Health, 2023, 28(5): 2558-2568.
30.	Peng B, Zhang Y, Wang M, et al. TA-MFFNet: Multi-feature fusion network for EEG analysis and driving fatigue detection based on time domain network and attention network. Comput Biol Chem, 2023, 104: 107863.
31.	Gao D, Wang K, Wang M, et al. SFT-Net: A network for detecting fatigue from EEG signals by combining 4D feature flow and attention mechanism. IEEE J Biomed Health, 2023, 28(8): 4444-4455.

1. Moradi A, Nazari S S H, Rahmani K. Sleepiness and the risk of road traffic accidents: A systematic review and meta-analysis of previous studies. Transport Res F-Traf, 2019, 65: 620-629.
2. Chand A, Jayesh S, Bhasi A B. Road traffic accidents: An overview of data sources, analysis techniques and contributing factors. Mater Today Proc, 2021, 47: 5135-5141.
3. Zhang Z, Ning H, Zhou F. A systematic survey of driving fatigue monitoring. IEEE T Intell Transp, 2022, 23(11): 19999-20020.
4. 张瑞, 朱天军, 邹志亮, 等. 驾驶员疲劳驾驶检测方法研究综述. 计算机工程与应用, 2022, 58(21): 53-66.
5. Fu S, Yang Z, Ma Y, et al. Advancements in the intelligent detection of driver fatigue and distraction: a comprehensive review. Appl Sci, 2024, 14(7): 3016.
6. Lambay A, Liu Y, Morgan P L, et al. Machine learning assisted human fatigue detection, monitoring, and recovery. Digit Eng, 2024, 1: 100004.
7. Wang H, Han M, Avouka T, et al. Research on fatigue identification methods based on low-load wearable ECG monitoring devices. Rev Sci Instrum, 2023, 94(4): 045103.
8. 管凯捷, 姚康, 任谊文, 等. 基于头动与眼电信号的疲劳检测研究. 计算机应用与软件, 2022, 39(2): 81-87.
9. Wang F, Wan Y, Li M, et al. Recent advances in fatigue detection algorithm based on EEG. Intell Autom Soft Co, 2023, 35(3): 3573.
10. Lu J, Zheng X, Tang L, et al. Can steering wheel detect your driving fatigue?. IEEE T Veh Technol, 2021, 70(6): 5537-5550.
11. Halomoan J, Ramli K, Sudiana D, et al. A new ECG data processing approach to developing an accurate driving fatigue detection framework with heart rate variability analysis and ensemble learning. Information, 2023, 14(4): 210.
12. Nemcova A, Janousek O, Vitek M, et al. Testing of features for fatigue detection in EOG. Bio-Med Mater Eng, 2017, 28(4): 379-392.
13. Kar S, Bhagat M, Routray A. EEG signal analysis for the assessment and quantification of driver’s fatigue. Transport Res F-Traf, 2010, 13(5): 297-306.
14. 石锦璇, 王昆. 基于栈式自编码的驾驶员脑电信号疲劳检测方法. 测试技术学报, 2023, 37(2): 140-145.
15. Zheng W L, Lu B L. A multimodal approach to estimating vigilance using EEG and forehead EOG. J Neural Eng, 2017, 14(2): 026017.
16. Wu W, Wu Q M J, Sun W, et al. A regression method with subnetwork neurons for vigilance estimation using EOG and EEG. IEEE T Cogn Dev Syst, 2018, 13(1): 209-222.
17. 王家曜, 张震, 宋光乐, 等. 一种基于脑电信号的疲劳驾驶检测方法. 控制工程, 2024, 31(6): 1091-1098.
18. Dubey S R, Singh S K, Chaudhuri B B. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 2022, 503: 92-108.
19. Segu M, Tonioni A, Tombari F. Batch normalization embeddings for deep domain generalization. Pattern Recogn, 2023, 135: 109115.
20. Wu L, Li J, Wang Y, et al. R-drop: Regularized dropout for neural networks. Adv Neural Inf Process Syst, 2021, 34: 10890-10905.
21. Zheng Y, Ding J, Liu F, et al. Adaptive neural decision tree for EEG based emotion recognition. Inf Sci, 2023, 643: 119160.
22. Mao A, Mohri M, Zhong Y. Cross-entropy loss functions: Theoretical analysis and applications// International Conference on Machine learning(ICML). Honolulu: PMLR, 2023: 23803-23828.
23. Gao X Y, Zhang Y F, Zheng W L, et al. Evaluating driving fatigue detection algorithms using eye tracking glasses// 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER). Montpellier: IEEE, 2015: 767-770.
24. Yi Y, Zhang H, Zhang W, et al. Fatigue working detection based on facial multifeature fusion. IEEE Sens J, 2023, 23(6): 5956-5961.
25. Barwick F, Arnett P, Slobounov S. EEG correlates of fatigue during administration of a neuropsychological test battery. Clin Neurophysiol, 2012, 123(2): 278-284.
26. Zhang M, Zhao B, Li J, et al. Research on driver’s fatigue detection based on information fusion. Comput Mater Con, 2024, 79(1): 1039.
27. Stancin I, Cifrek M, Jovic A. A review of EEG signal features and their application in driver drowsiness detection systems. Sensors, 2021, 21(11): 3786.
28. Gao Z, Wang X, Yang Y, et al. EEG-based spatio–temporal convolutional neural network for driver fatigue evaluation. IEEE T Neur Net Lear, 2019, 30(9): 2755-2763.
29. Gao D, Li P, Wang M, et al. CSF-GTNet: A novel multi-dimensional feature fusion network based on Convnext-GeLU-BiLSTM for EEG-signals-enabled fatigue driving detection. IEEE J Biomed Health, 2023, 28(5): 2558-2568.
30. Peng B, Zhang Y, Wang M, et al. TA-MFFNet: Multi-feature fusion network for EEG analysis and driving fatigue detection based on time domain network and attention network. Comput Biol Chem, 2023, 104: 107863.
31. Gao D, Wang K, Wang M, et al. SFT-Net: A network for detecting fatigue from EEG signals by combining 4D feature flow and attention mechanism. IEEE J Biomed Health, 2023, 28(8): 4444-4455.

Journal of Biomedical Engineering

Latest ArticlesFatigue driving detection based on prefrontal electroencephalogram asymptotic hierarchical fusion network

Abstract Full text Figures/Tables Video References Cited by

Format

Content