Driving tired is a major cause of accidents, second only to speeding. Fatigue driving is particularly common among professional drivers, especially long-distance passenger logistics drivers. Due to the drivers sitting position and fixed repeated action for a long time, its physiological function and psychological state of slow change, lead to distraction, dozing, narrow field of vision, information leakage, the judgment of slow, driving error or loss of driving ability completely, and collision, rushed out of the road surface such as a serious traffic accident. Studies have shown that 60% of traffic accidents can be avoided if drivers react 0.5 seconds earlier. Therefore, the research and design of a reliable driving fatigue detection system is of great significance to reduce the probability of road accidents, casualties and property losses. Therefore, the State Council "national medium and long-term science and technology development plan outline (2006-2020)" in key areas and their priority theme highlights the transportation safety and emergency guarantee: "key development traffic accident prevention and early warning, emergency processing technology, development of transport active and passive safety technology, traffic accident reproduction technology, such as traffic emergency response system and fast search and rescue technology." The development trend of fatigue detection technology, the development course of data set and evaluation criteria are respectively discussed in the following three aspects.

1. Development trend of fatigue detection technology

The pilot fatigue state monitoring system was first used in aircraft and other fields with high degree of advanced auxiliary pilot or autopilot. The pilot fatigue monitoring system in the early stage was a monitoring system based on the physiological response characteristic signal of human body when it was tired. According to the different signal attributes, the driver fatigue state monitoring system can be divided into direct monitoring and indirect monitoring. Fatigue driving detection and early warning is an important means to reduce traffic accidents, and it is also a research hotspot in academic and industrial circles at home and abroad. Major car manufacturers, parts suppliers, professional companies, government agencies and research institutes are all involved. Figure 1 summarizes the main approaches currently in use. These methods can be divided into two broad categories:

Figure 1: Main methods of fatigue driving detection

(1) Non-contact detection means that the driver does not need to wear any sensor that touches the body. The advantage of this method is that it is easy to use and does not require any extra operation by the driver. Non-contact detection can be further subdivided into detection methods based on computer vision and detection methods based on human-vehicle interaction characteristics:

(a) In the detection method based on computer vision, computer vision can act on the driver or the vehicle itself.

When computer vision is applied to the driver, a number of cameras are placed behind the front windshield to photograph the driver's head in real time. The driver's blink frequency (the blink frequency of the driver is generally reduced when he is tired) and the eyelid closure PERCLOS (the 80% eyelid closure time within 1 minute; Driver fatigue when eyelid closure time usually increases), eye tracking, observe whether the driver is straight ahead, whether to take the initiative to check the rear view mirror and side mirrors), pupillary reaction (when the driver fatigue pupils react to light changes slower), nodding (driver hit when sleepy head usually hanging lower, nod action increase), yawning, so as to determine whether the driver fatigue. For example, Hardeep et al. [9] proposed a method of using infrared light to locate and track human eyes. By analyzing the opening and closing state of eyes, the fatigue state of drivers can be judged. Mandalapu et al. constructed a fatigue detection model by extracting the features of driver's mouth, eyes and yawning. Niu Qingning et al. used Smart Eye system to extract drivers' Eye data for fatigue driving detection. Mario et al. adopted infrared detection algorithm to realize face detection and used Adaboost to locate the team's eyes and mouth. Weiyun adopts HOG, CLNG and rotation change algorithms to realize face detection. This kind of method generally obtains behavior characteristic information through image processing technology, which is non-contact and easy to install. However, it is easily affected by light, environment and personal habits of drivers, which affects the detection effect. The main drawback is that the image is heavily influenced by light, and the driver can't wear shades, masks and other shielding. In addition, there are privacy issues with the camera pointing at the driver all the time, which may make the driver feel uncomfortable. What's more, these head movements don't necessarily accurately reflect fatigue. In the case of the widely-used eyelid closure test, studies have found that drivers can enter a microsleep state with their eyelids open normally.

Fig. 2 Fatigue detection system for iFlytek pilots

Fig. 3 Fatigue detection process based on face recognition

(b) in the car interactive feature detection method based on people, the computer through a variety of sensors in the process of driving parameters, so as to determine whether the vehicle speed, the car is too close, whether the driver hit the brakes, steering wheel adjust adjust the steering wheel (driver fatigue frequency decreases), etc., to further determine whether the driver fatigue. The method to judge the fatigue state of the driver from the steering wheel movement has been used by Nissan, Renault and Volkswagen, but the effect is not very ideal. The main reason is that it is greatly affected by the road conditions. For example, on the straight and empty expressway, the driver may not need to adjust the steering wheel. For example, Sandberg et al. detect fatigue through vehicle speed, vehicle lateral position, steering wheel Angle and yaw Angle. Zhang Xibo et al. used the double time window method to extract two indexes, the maximum zero speed percentage and the maximum Angle standard deviation, from the steering wheel Angle, and built a fatigue driving detection model. Qu Xiaolei extracted fatigue features of steering wheel and vehicle state information through driving simulator, and built a fatigue driving detection model based on SVM. Mao Zhe analyzed the horizontal and longitudinal operating characteristics of drivers through driving simulation experiments and compared and analyzed the PCA based fuzzy clustering and neural network detection methods. The third method can usually use the existing vehicle sensors to collect data for index construction, which has the advantages of non-contact, abundant data sources and low cost, but also has the disadvantages of indirect detection and low accuracy. Because it is difficult to collect drivers, the research in this field mainly focuses on the data of simulation experiments and driving experiments, and rarely uses the research results of non-experimental data of drivers, which is difficult to meet the needs of actual use.

Figure 4 detects driver fatigue through lane lines and vehicle positions

(2) Contact detection, that is, the driver needs to wear a sensor that touches the body to collect physiological signals. These physiological signals including electroencephalogram (8 and 13 hz alpha waves when driver sleepiness activity decreases, 4-8 hz theta waves increase activity), electrocardiogram (snoring driver slowed heart rate change), electromyography (emg of frequency with the fatigue and the formation of the degree of fatigue showed a trend of decline, myoelectricity amplitude increases with increasing level of fatigue), eye diagram (detect eye movement direction and speed), respiratory, skin conductance, etc. The advantage of contact detection is that physiological signals are theoretically more accurate and reliable indicators of fatigue, since they come directly from the body. In addition, physiological signals can be predicted before the driver is tired, while non-contact methods are based on the driver's performance after fatigue, so fatigue detection based on physiological signals may provide more adequate warning time. The disadvantage of this method is that the driver's body movements will increase the artifact and noise of the measured signal, and reduce the detection accuracy. Therefore, it is necessary to use a variety of advanced signal processing filtering algorithms to remove artifacts and noise and improve the signal-to-noise ratio. In addition, drivers may feel uncomfortable wearing these sensors and become reluctant to use them. To alleviate this problem, a growing number of solutions are using wireless technology to transmit physiological signals to mobile phones or other mobile devices for processing. Further, some schemes embed sensors in steering wheels or in the driver's seat. For example, Yeo et al. used support vector machine algorithm to classify EEG signals under normal and fatigue states, so as to judge the fatigue state of drivers. Ma et al. used deep learning network PCANET to improve the accuracy and efficiency of EEG signals in fatigue classification. The first kind of method has high accuracy, but the equipment is expensive, and it needs to contact the driver's body directly in the process of use, which will affect the driver's operation to some extent.

2. Driver fatigue state data set development

In the early stage, due to the limitation of test conditions and safety considerations, most of the driving fatigue tests were carried out on the driver simulator. Its advantage is that there is no collision, rear-end collision, lane deviation and other dangerous conditions in the virtual traffic road environment, and it can be tested in the full range from mild fatigue to severe fatigue. The disadvantage is that the driver simulator is quite different from the real traffic environment, so it can be used in the exploration of fatigue monitoring methods, but it lacks authenticity if used in the design of fatigue monitoring system.

After the driver simulator test, the real vehicle test under the specified conditions appeared. After the transformation of the test vehicle, according to the pre-designed operating conditions, record the test data. If the movement information of the driver's face, eyes and head needs to be collected, additional sensors and corresponding data acquisition equipment are needed. If the driver's behavior, vehicle status and vehicle trajectory signals need to be collected, there is generally no need to add additional sensors, and the vehicle's own sensors can meet the requirements. Natural driving covers various working conditions, and the data can truly reflect the fatigue state of the driver under various working conditions. The fatigue monitoring system designed accordingly covers the real fatigue state of the near-driver under various working conditions, and the higher the accuracy will be. Large-scale natural driving trials began in the United States in 2004, followed by the European Union.

In natural driving research starts late in our country, the first natural driving research project in September 2012, by tongji university, gm China company and Virginia tech university transportation research center together, in 5 car test data acquisition equipment installation, monitoring and recording 60 drivers driving situation within three years, used to analyze the real traffic environment and driving behavior characteristics.

During the test, a number of working conditions with typical characteristics were designed, such as straight driving on urban roads, straight driving on rural roads and straight driving on expressways. Special working conditions such as turning, lane change, crosswind and road bulge were not considered. The designed driver fatigue state monitoring system has a wide range of applicability and higher accuracy than the one based on driver simulator, but it is not suitable for some special conditions. With the development of fatigue monitoring technology and the continuous improvement of users' requirements for the accuracy of fatigue monitoring system, the research of fatigue monitoring technology based on natural driving data has gradually become the mainstream direction.

3. Fatigue state evaluation criteria

At present, the evaluation standard of driver fatigue degree can be divided into self-evaluation method and other evaluation method according to the different executors. The self-assessment method is that drivers score their current state of fatigue according to the standard of KSS (Karolinska Sleepiness Scale) at regular intervals, and divide the fatigue level from very awake to very tired into 9 levels corresponding to a score of 1 to 9 points.

Advantages of KSS: good versatility, high precision, is the widely used standard of fatigue degree self-assessment method;

Disadvantages: both the tester's asking about the driver's fatigue degree and the driver's answering about his fatigue degree will stimulate the continuation of the driver's state, and the score of the driver's fatigue degree will be reflected. Other evaluation method is divided into subjective other evaluation method and objective other evaluation method.

According to the different signal sources, there are two kinds of subjective evaluation method. One is that the experimenter firstly observe the driver's facial expression, eye movement, frequent degree of yawning, etc., and then grade the driver's current state according to KSS standard.

The second method is the subjective evaluation method based on the driver's face video. The camera is used to collect the driver's face information, and the video is segmented during post-processing. Several specially trained testers score a video clip, and finally the average score of each tester is used as the score of the driver's fatigue degree in the video. The latter is more accurate than the former, and only needs to install a camera in the car to collect the video information of the driver's face, so the subjective evaluation method based on video is more commonly used. The objective evaluation method uses physiological signals of the driver to judge the fatigue degree, such as electrocardiogram and electroencephalogram, especially electroencephalogram, which can best characterize the fatigue state of the driver.

To sum up, there are many methods for fatigue driving detection. The main difficulty of current research and application is that although there are many kinds of data that can be used as input signals, each signal cannot be used independently to reliably determine the fatigue state of the driver. Future research directions can be considered from the following three perspectives: 1) further mining of fatigue characteristics, using advanced signal processing methods to extract the characteristic parameters of each input signal that can best represent fatigue; 2) The method of signal fusion is adopted, and multiple fatigue characteristic parameters are combined for detection to overcome the influence of light, shelter, weather, road signs, etc., improve the real-time performance, accuracy and robustness of detection, and reduce the false alarm rate; 3) The detection method should have the function of adaptive online learning, which can conduct self-training optimization according to the personalized data of each driver, overcome individual differences, and get the personalized detection model that is most suitable for each driver.

reference

[1] Mott G E, Forsman P, Short K R, et al. Efficient driver drowsiness detection at moderate levels of drowsiness.[J]. Accident Analysis and Prevention, 2013, 50(1): 341-350.

[2] Li Z J, Li S B, Li R J, et al. Online detection of driver fatigue using steering wheel angles for real driving conditions[J]. Sensors, 2017, 17(3): 495-507.

ZHONG Mingen, HUANG Jiehong, QIAO Yunhao, et al. Comparison of Influence of Physical Fatigue and Mental Fatigue on Vehicle Driving. Journal of Safety Science and Technology, 2017, 13(1): 22-27.

LIU Longfei. Research on Driving Fatigue Detection Method Based on Face Analysis [D]. Wuhan University of Science and Technology, 2019.

[5]Yeo M V M, Li X, Shen K, et al. Can SVM be used for automatic EEG detection of drowsiness during car driving [J]. Safety Science, 2009, 47 (1): 115 124.

[6] Xiao Sai, Lei Yewei. Research Progress on Causes and Monitoring of Driving Fatigue [J]. Journal of Transportation Science and Economics, 2017(4):14-19.

[7]Chuang C H, Huang C S, Ko L W, et al. An EEG-based perceptual function integration network for application to drowsy driving[J]. Knowledge-Based Systems, 2015, 80(C): 143-152.

[8] Han W, Yang Y, Huang G B, et al. Driver drowsiness detection based on novel eye openness recognition method and unsupervised feature learning[C]. IEEE International Conference on Systems, Man, and Cybernetics, Kowloon, 2015: 1470-1475.

[9] Niu Qingning, Zhou Zhiqiang, Jin Lisheng et al. Fatigue Driving Detection Method Based on Eye Movement Feature [J]. Journal of Harbin Engineering University, 2015(03):394-398.

[10] Qu Xiaolei. Research on Fatigue Driving Detection Method Based on Steering Operation and Vehicle State [D]. Beijing: Tsinghua University,2012.

MAO Zhe. Research on Vehicle Fatigue Driving Behavior Identification Method [D]: Wuhan University of Technology,2009.