首页资讯面向驾驶员的个性化健康导航

面向驾驶员的个性化健康导航

来源：泰然健康网时间：2024年12月16日 06:19

摘要:

为了减少因驾驶员的生理和心理健康状况变化引发的交通事故，实现对驾驶员健康状态的自动监测和实时优化，提出以控制论的基本理论为基础的驾驶员健康状态闭环反馈系统框架.首先基于驾驶员日志建立个性化健康模型；然后结合各种传感器实时采集的驾驶员、车辆和道路环境等多模态数据，对驾驶员当前健康状态进行估计；最后针对预设健康目标，为驾驶员提供可执行的行为建议，实现对驾驶员健康状态的导航优化.在最关键的实时监测环节，提出基于注意力的卷积神经网络（convolutional neural network，CNN）-长短期记忆网络（long short term memory，LSTM）的多模态融合模型，实现对驾驶员压力、情绪和疲劳3个方面的健康状态估计.在私有数据集和公开数据集上分别开展的实验验证均获得高于90%的检测准确率.实验结果表明，提出的模型和方法可以实时准确监测驾驶员的压力、情绪和疲劳状态，为实现驾驶员的个性化健康导航系统提供有力支撑.

Abstract:

To decrease the number of traffic accidents caused by changes in drivers' physical and mental health conditions and accomplish automatic monitoring and real-time optimization of drivers' health states, a closed-loop feedback system framework for drivers' health states was proposed based on the basic theory of cybernetics. First, a personalized health model was established based on a driver's log data. Then by combining this model with the real-time multimodal data of the driver, vehicle and road environment from various sensors, the driver's current health state was estimated. Finally given the health goal of the driver, executable behavior suggestions were provided to navigate the driver to an optimized health state. For the most critical phase of real-time monitoring, a multimodal fusion model based on attentional convolutional neural networks and long short-term memory network (CNN-LSTM) was proposed to estimate the three aspects of driver health, namely, stress, emotion, and fatigue. Experiments on both private and public datasets have achieved a detection accuracy of more than 90%, which demonstrates that the proposed model and methods can accurately monitor drivers' stress, emotion, and fatigue states in real time, thus provide a solid basis for implementing the personalized health navigation system for drivers (PHN-D).

图 1 驾驶员个性化健康导航架构

Figure 1. Architecture of personalized health navigation for drivers

图 2 面向驾驶员压力检测的多模态融合模型

Figure 2. Multimodal fusion model for driver stress detection

图 3 面向驾驶员情绪检测的多模态融合模型

Figure 3. Multimodal fusion model for driver's emotion detection

图 4 自我评估人体模型描述维度情感的等级

Figure 4. SAM used to describe levels of dimensional emotion

图 5 面向驾驶员疲劳检测的多模态融合模型

Figure 5. Multimodal fusion model for driver's fatigue detection

图 6 残差网络的架构

Figure 6. Structure of the residual network

图 7 CARRS-Q高级驾驶模拟器

Figure 7. CARRS-Q advanced driving simulator

图 8 NTHU-DDD数据集的一些样本帧

Figure 8. Some sample frames of NTHU-DDD dataset

图 9 融合模型在不同窗口大小下的平均准确率

Figure 9. Average accuracy of the fusion model in different window sizes

图 10 各情感维度在不同模型下的平均准确率

Figure 10. Average accuracy of each emotional dimension in different models

表 1 不同驾驶场景中的不同压力源

Table 1 Different stressors in different driving scenarios

场景车辆数量道路情况模拟器参数天气时间城市1 0 白天城市2 30 狭窄和弯曲晚上高速公路 50 弯曲超车、变道、超速和追尾等雨密度(0.2~1.0), 雾晚上 CBD1 50 狭窄、弯曲和急弯超车、变道、超速和追尾等雨密度(0.3~0.6), 雾白天 CBD2 60 弯曲和急弯超车、变道、超速和追尾等白天

表 2 压力检测的1D-CNN-LSTM模型参数

Table 2 Parameters of 1D-CNN-LSTM model for stress detection

网络层网络层参数卷积层卷积核=20，核尺寸=(10，1)，步数=1 池化层池化尺寸=(2，1)，步数=2 卷积层卷积核=40，核尺寸=(5，1)，步数=1 池化层池化尺寸=(2，1)，步数=2 卷积层卷积核=80，核尺寸=(3，1)，步数=1 池化层池化尺寸=(2，1)，步数=2 LSTM 隐藏层尺寸=64 LSTM 隐藏层尺寸=64

表 3 情绪检测的1D-CNN-LSTM模型参数

Table 3 Parameters of 1D-CNN-LSTM model for emotion detection

网络层网络层参数卷积层卷积核=20，核尺寸=(10，1)，步数=1 池化层池化尺寸=(2，1)，步数=2 卷积层卷积核=40，核尺寸=(3，1)，步数=1 卷积层卷积核=40，核尺寸=(3，1)，步数=1 池化层池化尺寸=(2，1)，步数=2 卷积层卷积核=80，核尺寸=(3，1)，步数=1 池化层池化尺寸=(2，1)，步数=2 LSTM 隐藏层尺寸=64 LSTM 隐藏层尺寸=64

表 4 不同模态下的特征数据

Table 4 Feature data in different modalities

模态特征数据眼部数据瞳孔直径凝视离散度(x和y轴) 眨眼频率车辆数据方向盘角度刹车踏板数据油门踏板数据环境数据距前车的距离一天中的时间车道宽度和数量天气条件(小雨、中雨、大雨和能见度)

表 5 不同模型在不同模态下的驾驶员压力检测的平均准确率

Table 5 Average accuracy of different models for driver stress detection in different modalities %

方法环境车辆眼部融合 LSTM 50.6 44.9 58.1 71.5 CNN-LSTM 51.0 73.6 85.8 90.8 CNN-LSTM-Attention 52.6 85.1 92.9 95.5

表 6 不同模态下的各情感维度的平均准确率

Table 6 Average accuracy of each emotional dimension in different modalities %

模态愉悦度兴奋度支配度环境 37.87 44.64 45.86 车辆 66.85 69.16 70.65 眼部 95.28 94.37 94.13 环境和车辆 70.96 72.87 74.22 眼部和车辆 97.29 96.79 96.92 环境和眼部 98.71 98.88 98.75 环境、车辆和眼部 98.89 98.87 98.82

表 7 融合模型在不同模态下的疲劳检测性能

Table 7 Driver fatigue detection performance of the fusion model in different modalities %

模态平均F1值平均准确率眼部 90.78 89.20 嘴部 90.30 88.76 头部 90.89 89.24 眼部和嘴部 94.25 93.35 眼部和头部 93.57 92.55 嘴部和头部 92.65 91.33 眼部、嘴部和头部 94.66 93.77 [1]

World Health Organization. Road traffic injuries[R/OL]. [2020-12-29]. https://www.who.int/zh/news-room/fact-sheets/detail/road-traffic-injuries.

[2]

United Nations Conventions and Declarations Search System. Transforming our world: the 2030 agenda for sustainable development[R/OL]. [2020-12-29]. https://www.un.org/zh/documents/treaty/files/A-RES-70-1.shtml.

[3]

SINGH S. Critical reasons for crashes investigated in the national motor vehicle crash causation survey[R/OL]. [2020-12-29]. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812115.

[4]

SAHAYADHAS A, SUNDARAJ K, MURUGAPPAN M. Detecting driver drowsiness based on sensors: a review[J]. Sensors, 2012, 12(12): 16937-16953. doi: 10.3390/s121216937

[5]

LEE B, CHUNG W Y. Wearable glove-type driver stress detection using a motion sensor[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(7): 1835-1844. doi: 10.1109/TITS.2016.2617881

[6]

KAMARUDDIN N, WAHAB A. Driver behavior analysis through speech emotion understanding[C]//2010 IEEE Intelligent Vehicles Symposium. Piscataway: IEEE, 2010: 238-243.

[7]

SOMAN K, ALEX V, SRINIVAS C. Analysis of physiological signals in response to stress using ECG and respiratory signals of automobile drivers[C]//2013 International Multi-conference on Automation, Computing, Communication, Control and Compressed (IMAC4S). Piscataway: IEEE, 2013: 574-579.

[8]

KRYGER M H, ROTH T, DEMENT W C. Principles and practice of sleep medicine[J]. American Review of Respiratory Disease, 1989, 140(4): 1172.

[9]

GAO H, YUCE A, THIRAN J P. Detecting emotional stress from facial expressions for driving safety[C]//2014 IEEE International Conference on Image Processing (ICIP). Piscataway: IEEE, 2014: 5961-5965.

[10]

FAIRCLOUGH S H, GRAHAM R. Impairment of driving performance caused by sleep deprivation or alcohol: acomparative study[J]. Human Factors, 1999, 41(1): 118-128. doi: 10.1518/001872099779577336

[11]

CHENG B, ZHANG W, LIN Y, et al. Driver drowsiness detection based on multisource information[J]. Human Factors and Ergonomics in Manufacturing & Service Industries, 2012, 22(5): 450-467.

[12]

YANG G S, LIN Y Z, BHATTACHARYA P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network[J]. Information Sciences, 2010, 180(10): 1942-1954. doi: 10.1016/j.ins.2010.01.011

[13]

RASTGOO M N, NAKISA B, MAIRE F. Automatic driver stress level classification using multimodal deep learning[J/OL]. Expert Systems with Applications, 2019, 138(30): 112793. [2020-12-30]. https://sciencedirect.com/science/article/abs/pii/s0957417419304890.

[14]

KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//26th Annual Conference on Neural Information Processing Systems. Piscataway: IEEE, 2012: 1097-1105.

[15]

HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735

[16]

EIMAN K, YOUNIS E M G, ANG C S. Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection[J]. Information Fusion, 2019, 49: 46-56. doi: 10.1016/j.inffus.2018.09.001

[17]

BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL]. [2020-12-30]. https://arxiv.org/abs/1409.0473.

[18]

LIN Z, FENG M, SANTOS C N D, et al. A structured self-attentive sentence embedding[EB/OL]. [2020-12-30]. https://arxiv.org/abs/1703.03130.

[19]

SAINATH T N, VINYALS O, SENIOR A, et al. Convolutional, long short-term memory, fully connected deep neural networks[C]//40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway: IEEE, 2015: 4580-4584.

[20]

BHUNIA A K, KONWER A, BHUNIA A K, et al. Script identification in natural scene image and video frames using an attention based convolutional-LSTM network[J]. Pattern Recognition, 2018, 85: 172-184.

[21] 毛喆, 严新平, 吴超仲, 等. 疲劳驾驶时的车速变化特征[J]. 北京工业大学学报, 2011, 37(8): 1175-1183. doi: 10.11936/bjutxb2011081175

MAO Z, YAN X P, WU C Z, et al. Characteristics of speed changes during fatigue driving[J]. Journal of Beijing University of Technology, 2011, 37(8): 1175-1183. (in Chinese) doi: 10.11936/bjutxb2011081175

[22] 冯仁科, 杜艳爽, 乔建刚. 基于驾驶员心生理反应的交叉口车辆减速特性[J]. 北京工业大学学报, 2019, 45(7): 679-684. doi: 10.11936/bjutxb2017120037

FENG R K, DU Y C, QIAO J G. Vehicle deceleration characteristics at intersections based on the driver's physiological reaction[J]. Journal of Beijing University of Technology, 2019, 45(7): 679-684. (in Chinese) doi: 10.11936/bjutxb2017120037

[23] 胡江碧, 李安, 王维利. 不同天气状况下驾驶员驾驶工作负荷分析[J]. 北京工业大学学报, 2011, 37(4): 529-532. doi: 10.11936/bjutxb2011040529

HU J B, LI A, WANG W L. Analysis of driver's driving workload under different weather conditions[J]. Journal of Beijing University of Technology, 2011, 37(4): 529-532. (in Chinese) doi: 10.11936/bjutxb2011040529

[24]

NAG N, JAIN R. A navigational approach to health: actionable guidance for improved quality of life[J]. Computer, 2019, 52(4): 12-20. doi: 10.1109/MC.2018.2883280

[25]

NAG N, PANDEY V, OH H, et al. Cybernetic health[EB/OL]. [2020-12-30]. https://arxiv.org/abs/1705.08514.

[26]

VERMA G K, TIWARY U S. Affect representation and recognition in 3D continuous valence-arousal-dominance space[J]. Multimedia Tools and Applications, 2017, 76(2): 2159-2183. doi: 10.1007/s11042-015-3119-y

[27]

BRADLEY M M, LANG P J. Measuring emotion: the self-assessment manikin and the semantic differential[J]. Journal of Behavior Therapy and Experimental Psychiatry, 1994, 25(1): 49-59. doi: 10.1016/0005-7916(94)90063-9

[28]

DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2005: 886-893.

[29]

KAZEMI V, SULLIWAN J. One millisecond face alignment with an ensemble of regression trees[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 1867-1874.

[30]

ZACH C, POCK T, BISCHOF H. A duality based approach for realtime TV-L1 optical flow[C]//Proceedings of the 29th DAGM Conference on Pattern Recognition. Berlin: Springer, 2007: 214-223.

[31]

HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2016: 770-778.

[32]

Queensland University of Technology. Advanced driving simulator[EB/OL]. [2020-12-30]. https://research.qut.edu.au/carrsq/wp-content/uploads/sites/45/2019/02/Simulator.pdf.

[33]

AVSimulation. Scaner studio[EB/OL]. [2020-12-30]. https://www.avsimulation.com/scanerstudio.

[34]

FUNKE G, GREENLEE E, CARTER M, et al. Which eye tracker is right for your research? performance evaluation of several cost variant eye trackers[J]. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2016, 60(1): 1240-1244. doi: 10.1177/1541931213601289

[35]

WENG C H, LAI Y H, LAI S H. Driver drowsiness detection via a hierarchical temporal deep belief network[C]//Asian Conference on Computer Vision (ACCV). Berlin: Springer, 2016: 117-133.

[36]

HE K, FAN H, WU Y, et al. (2020). Momentum contrast for unsupervised visual representation learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 9726-9735.

面向驾驶员的个性化健康导航

推荐资讯

从出汗看健康出汗透露你的健康信号

早上怎么喝水最健康？

面向驾驶员的个性化健康导航

推荐资讯

从出汗看健康 出汗透露你的健康信号

早上怎么喝水最健康？

从出汗看健康出汗透露你的健康信号