Анализ исследований определения утомления на основе мониторинга окуломоторных событий
- Авторы: Шошина И.И.1, Коваленко С.Д.2, Кузнецов В.В.3, Брак И.В.4,5, Кашевник А.М.6
-
Учреждения:
- Санкт-Петербургский государственный университет
- Национальный исследовательский университет "Высшая школа экономики"
- Федеральный исследовательский центр "Информатика и управление" РАН
- Новосибирский государственный университет
- Приволжский исследовательский медицинский университет Минздрава России
- Санкт-Петербургский Федеральный исследовательский центр РАН
- Выпуск: Том 50, № 3 (2024)
- Страницы: 81-101
- Раздел: ОБЗОРЫ
- URL: https://permmedjournal.ru/0131-1646/article/view/664026
- DOI: https://doi.org/10.31857/S0131164624030074
- EDN: https://elibrary.ru/BULTQU
- ID: 664026
Цитировать
Аннотация
Рассмотрены теоретические предпосылки определения функционального состояния утомления на основе анализа стратегии глазных движений, современные методы оценки движений глаз. Анализ литературы позволяет сделать вывод о том, что в настоящее время существует огромное количество численных характеристик движений глаз, динамика которых гипотетически может позволить судить о степени утомления человека. Однако, пока отсутствуют предложения метода определения степени утомления на основе анализа стратегии глазных движений. В связи с этим, основываясь на представлениях о статическом и динамическом зрении, предложено рассматривать сдвиг численных характеристик движений глаз в сторону показателей, отражающих стратегию динамического зрения, как свидетельство утомления.
Ключевые слова
Полный текст

Об авторах
И. И. Шошина
Санкт-Петербургский государственный университет
Автор, ответственный за переписку.
Email: shoshinaii@mail.ru
Институт когнитивных исследований
Россия, Санкт-ПетербургС. Д. Коваленко
Национальный исследовательский университет "Высшая школа экономики"
Email: shoshinaii@mail.ru
Россия, Москва
В. В. Кузнецов
Федеральный исследовательский центр "Информатика и управление" РАН
Email: shoshinaii@mail.ru
Россия, Москва
И. В. Брак
Новосибирский государственный университет; Приволжский исследовательский медицинский университет Минздрава России
Email: shoshinaii@mail.ru
Россия, Новосибирск; Нижний Новгород
А. М. Кашевник
Санкт-Петербургский Федеральный исследовательский центр РАН
Email: shoshinaii@mail.ru
Россия, Санкт-Петербург
Список литературы
- Исакова М. Тема по ВПП № 22 для военнослужащих, проходящих военную службу по контракту и призыву // Армейский Сборник. 2021. Т. 8. С. 126.
- Величковский Б.Б. Когнитивные эффекты умственного утомления // Вестник Московского университета. Серия 14. Психология. 2019. № 1. С. 108.
- Borgianni Y., Rauch E., Maccioni L., Mark B.G. User experience analysis in industry 4.0 – The use of biometric devices in engineering design and manufacturing / IEEE International conference on Industrial Engineering and Engineering Management. (IEEM), Bangkok, Thailand, 16-19 December, 2018. IEEE Computer Society, 2019. P. 192.
- Величковский Б.М., Ушаков В.Л. Когнитивные науки и новые медицинские технологии // Современные технологии в медицине. 2019. Т. 11. № 1. С. 8.
- Robertson C.V., Marino F.E. Cerebral responses to exercise and the influence of heat stress in human fatigue // J. Therm. Biol. 2017. V. 63. P. 10.
- Bergasa L.M., Nuevo J., Sotelo M.-A. et al. Real-time system for monitoring driver vigilance // IEEE Trans. Intell. Transp. Syst. 2006. V. 7. № 1. P. 63.
- D’Orazio T., Leo M., Guaragnella C., Distante A. A visual approach for driver inattention detection // Patt. Recog. 2007. V. 40. № 8. P. 2341.
- Al-Anizy G.J., Nordin M.J., Razooq M.M. Automatic driver drowsiness detection using haar algorithm and support vector machine techniques // Asian J. Appl. Sci. 2015. V. 8. № 2. P. 149.
- Golz M., Sommer D., Chen M. et al. Feature fusion for the detection of microsleep events // J. VLSI Sign. Process Syst. Sign. Im. 2007. V. 49. № 2. P. 329.
- Liu Z., Peng Y., Hu W. Driver fatigue detection based on deeply-learned facial expression representation // J. Vis. Commun. Image Represent. 2020. V. 71. № 2. P. 102723.
- Mandal B., Li L., Wang G.S., Lin J. Towards detection of bus driver fatigue based on robust visual analysis of eye state // IEEE Trans. Intell. Transp. Syst. 2016. V. 18. № 3. P. 545.
- Sigari M.H., Fathy M., Soryani M. A driver face monitoring system for fatigue and distraction detection // Int. J. Vehicul. Technol. 2013. V. 2013. P. 1.
- Ляпунов С.И., Шошина И.И., Ляпунов И.С. Треморные колебания глаз как объективный показатель утомления водителей // Физиология человека. 2022. Т. 48. № 1. С. 89.
- Golz M., Sommer D., Trutschel U. et al. Evaluation of fatigue monitoring technologies // Somnologie. 2010. V. 14. № 3. P. 187.
- Кубарко А.И., Лихачев С.А., Кубарко Н.П. Зрение (нейрофизиологические и нейроофтальмологические аспекты): монография в 2 т. Т. 2: Нейронные механизмы контроля установки и движения глаз и их нарушения при заболеваниях нервной системы. Минск: БГМУ, 2009. 352 с.
- Барабанщиков В.А., Жегалло А.В. Айтрекинг: методы регистрации движений глаз в психологических исследованиях и практике. М.: Когито-центр, 2014. С. 117.
- Ярбус А. Роль движений глаз в процессе зрения. М.: Наука, 1965. 161 с.
- Holmqvist K. Eye tracking: A comprehensive guide to methods and measures. O.: OUP Oxford, 2011. 560 p.
- Djanian S. Eye movement classification using deep learning. Aalborg University: Department of Electronic Systems, 2019. 76 p.
- Mahanama B., Jayawardana Y., Rengarajan S. et al. Eye movement and pupil measures: A review // Front. Comput. Sci. 2022. V. 3. P. 733531.
- Li X., Fan Z., Ren Y. et al. Classification of eye movement and its application in driving based on a refined pre-processing and machine learning algorithm // IEEE Access. 2021. V. 9. P. 136164.
- Salvucci D.D., Goldberg J.H. Identifying fixations and saccades in eye-tracking protocols / Proceedings of the 2000 symposium on Eye Tracking Research and Applications (ETRA ‘00). Palm Beach Gardens, FL, USA. November 6–8, 2000. Association for Computing Machinery (ACM). New York, NY, USA, 2000. P. 71.
- Wang S., Wang Q., Chen H. Research and application of eye movement interaction based on eye movement recognition / MATEC Web Conf. 2018. V. 246. P. 5.
- Carpenter R.H.S. Movements of the eyes: Part 1. Movements of the eyes. 2nd ed. Pion, 1988. P. 593.
- Shojaeizadeh M., Djamasbi S., Trapp A.C. Density of gaze points within a fixation and information processing behavior / Universal Access in Human-Computer Interaction. Methods, Techniques, and Best Practices. UAHCI 2016. Lect. Notes Comput. // Eds. Antona M., Stephanidis C. Sci. Springer, Cham, 2016. V. 9737. P. 465.
- Skaramagkas V., Giannakakis G., Ktistakis E. et al. Review of eye tracking metrics involved in emotional and cognitive processes // IEEE Rev. Biomed. Eng. 2023. V. 16. P. 260.
- Foy H.J., Chapman P. Mental workload is reflected in driver behavior, physiology, eye movements and prefrontal cortex activation // Appl. Ergon. 2018. V. 73. P. 90.
- Srimal R., Diedrichsen J., Ryklin E.B., Curtis C.E. Obligatory adaptation of saccade gains // J. Neurophysiol. 2008. V. 99. № 3. P. 1554.
- Russo M., Thomas M., Thorne D. et al. Oculomotor impairment during chronic partial sleep deprivation // Clin. Neurophysiol. 2003. V. 114. № 4. P. 723.
- Warren D.E., Thurtell M.J., Carroll J.N., Wall M. Perimetric evaluation of saccadic latency, saccadic accuracy, and visual threshold for peripheral visual stimuli in young compared with older adults // Invest. Ophthalmol. Vis. Sci. 2013. V. 54. № 8. P. 5778.
- Yang Y., McDonald M., Zheng P. Can drivers’ eye movements be used to monitor their performance? A case study // IET Intell. Transp. Syst. 2012. V. 6. № 4. P. 444.
- Nakayama M., Takahashi K., Shimizu Y. The act of task difficulty and eye-movement frequency for the “oculo-motor indices” / Proceedings of the 2002 symposium on Eye tracking research & applications (ETRA ‘02). New Orleans Louisiana, March 25-27, 2002. Association for Computing Machinery (ACM). New York, NY, USA. 2002. P. 37.
- Van Orden K.F., Limbert W., Makeig S., Jung T.P. Eye activity correlates of workload during a visuospatial memory task // Hum. Factors. 2001. V. 43. № 1. P. 111.
- Amor T.A., Reis S.D., Campos D. et al. Persistence in eye movement during visual search // Sci. Rep. 2016. V. 6. P. 20815.
- Joseph A.W., Murugesh R. Potential eye tracking metrics and indicators to measure cognitive load in human-computer interaction research // J. Sci. Res. 2020. V. 64. № 01. P. 168.
- Chen S., Epps J., Ruiz N., Chen F. Eye activity as a measure of human mental effort in HCI / Proceedings of the 16th international conference on Intelligent user interfaces (IUI’11). Association for Computing Machinery (ACM), February 13–16, 2011. New York, NY, USA, 2011. P. 315.
- Zagermann J., Pfeil U., Reiterer H. Measuring cognitive load using eye tracking technology in visual computing / Proceedings of the sixth workshop on Beyond Time and Errors on novel evaluation methods for visualization, BELIV ‘16. Baltimore MD USA, 24 October 2016. New York, NY: ACM Press, 2016. V. 24. P. 78.
- Fahimi R., Bruce N.D.B. On metrics for measuring scan path similarity // Behav. Res. Methods. 2021. V. 53. № 2. P. 609.
- Holland C., Komogortsev O.V. Biometric identification via eye movement scanpaths in reading / International Joint Conference on Biometrics. 11 October 2011. P. 1. https://api.semanticscholar.org/CorpusID:2528223.
- Goldberg J.H., Kotval X.P. Computer interface evaluation using eye movements: Methods and constructs // Int. J. Ind. Ergon. 1999. V. 24. № 6. P. 631.
- Yamada, Y., Kobayashi, M. Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults // Artif. Intell. Med. 2018. V. 91. P. 39.
- Kliegl R., Rolfs M., Laubrock J., Engbert R. Microsaccadic modulation of response times in spatial attention tasks // Psychol. Res. 2009. V. 73. № 2. P. 136.
- Krejtz K., Duchowski A.T., Niedzielska A. et al. Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze // PLoS One. 2018. V. 13. № 9. P. e0203629.
- Zandi A.S., Quddus A., Prest L., Comeau F.J. Non-Intrusive detection of drowsy driving based on eye tracking data // Transp. Res. Rec. 2019. V. 2673. № 6. P. 247.
- Zemblys R., Niehorster D.C., Komogortsev O., Holmqvist K. Using machine learning to detect events in eye-tracking data // Behav. Res. Methods. 2018. V. 50. № 1. P. 160.
- Chen J.T., Kuo Y.C., Hsu T.Y., Wang C.A. Fatigue and arousal modulations revealed by saccade and pupil dynamics // Int. J. Environ Res. Public Health. MDPI. 2022. V. 19. № 15. P. 9234.
- Brezinova V., Kendell R.E. Smooth pursuit eye movements of schizophrenics and normal people under stress // Br. J. Psychiatry. 1977. V. 130. P. 59.
- Rottach K.G., Zivotofsky A.Z., Das V.E. et al. Comparison of horizontal, vertical and diagonal smooth pursuit eye movements in normal human subjects // Vision Res. 1996. V. 36. № 14. P. 2189.
- Ranti C., Jones W., Klin A., Shultz S. Blink rate patterns provide a reliable measure of individual engagement with scene content // Sci. Rep. 2020. V. 10. № 1. P. 8267.
- Marquart G., Cabrall C., de Winter J. Review of eye-related measures of drivers’ mental workload // Procedia Manuf. 2015. V. 3. P. 2854.
- Haq Z.A., Hasan Z. Eye-blink rate detection for fatigue determination / India International Conference on Information Processing (IICIP 2016), Delhi, India, 12–14 August, 2016. Proceedings IEEE Inc., 2017. P. 1.
- Horiuchi R., Ogasawara T., Miki N. Fatigue assessment by blink detected with attachable optical sensors of dye-sensitized photovoltaic cells // Micromachines (Basel). 2018. V. 9. № 6. P. 310.
- Tolvanen O., Elomaa A. P., Itkonen M. et al. Eye-tracking indicators of workload in surgery: A systematic review // J. Invest. Surg. 2022. V. 35. № 6. P. 1340.
- Marshall S.P. The index of cognitive activity: Measuring cognitive workload / Proceedings of the IEEE 7th Conference on Human Factors and Power Plants. 19 September, Scottsdale, AZ, USA, 2002. P. 7. doi: 10.1109/HFPP.2002.1042860
- Duchowski A.T., Krejtz K., Krejtz I. et al. The index of pupillary activity: Measuring cognitive load vis-à-vis task difficulty with pupil oscillation / Proceedings of the Conference on Human Factors in Computing Systems (CHI ‘18). Association for Computing Machinery, New York, NY, USA, 2018. V. 282. P. 1. https://doi.org/10.1145/3173574.3173856
- Duchowski A.T., Krejtz K., Gehrer N.A. et al. The low/high index of pupillary activity / Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ‘20). Association for Computing Machinery, New York, NY, USA, 2020. P. 1. https://doi.org/10.1145/3313831.3376394
- Alnajar F., Gevers T., Valenti R., Ghebreab S. Calibration-free gaze estimation using human gaze patterns / Proceedings of the IEEE International Conference on Computer Vision. 1-8 Dec. 2013, Sydney, NSW, Australia, 2013. P. 137. doi: 10.1109/ICCV.2013.24
- Rigas I., Economou G., Fotopoulos S. Biometric identification based on the eye movements and graph matching techniques // Patt. Recogn. Lett. 2012. V. 33. № 6. P. 786.
- Li J., Li H., Umer W. et al. Identification and classification of construction equipment operators’ mental fatigue using wearable eye-tracking technology // Autom. Constr. 2020. V. 109. P. 103000.
- Bitkina O.V., Park J., Kim H.K. The ability of eye-tracking metrics to classify and predict the perceived driving workload // Int. J. Ind. Ergonom. 2021. V. 86. P. 103193.
- Shiferaw B., Downey L., Crewther D. A review of gaze entropy as a measure of visual scanning efficiency // Neurosci. Biobehav. Rev. 2019. V. 96. P. 353.
- Deravi F., Biosignals S.G. Gaze trajectory as a biometric modality / Proceedings of the International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-2011). 2011. P. 335. doi: 10.5220/0003275803350341
- Peißl S., Wickens C.D., Baruah R. Eye-Tracking measures in aviation: A selective literature review // Int. J. Aerosp. Psychol. 2018. V. 28. № 3–4. P. 98.
- Meghanathan R.N., Nikolaev A.R., van Leeuwen C. Refixation patterns reveal memory-encoding strategies in free viewing // Atten. Percept. Psychophys. 2019. V. 81. № 7. P. 2499.
- Fukushima K., Fukushima J., Warabi T., Barnes G.R. Cognitive processes involved in smooth pursuit eye movements: behavioral evidence, neural substrate and clinical correlation // Front. Syst. Neurosci. 2013. V. 7. P. 4.
- Knox P.C., Davidson J.H., Anderson D. Age-related changes in smooth pursuit initiation // Exp. Brain Res. 2005. V. 165. № 1. P. 1.
- Einhäuser W. The pupil as marker of cognitive processes / Computational and Cognitive Neuroscience of Vision. Cognitive Science and Technolog // Ed. Zhao Q. Springer, Singapore, 2017. P. 141.
- Richstone L., Schwartz M.J., Seideman C. et al. Eye metrics as an objective assessment of surgical skill // Ann. Surg. 2010. V. 252. № 1. P. 1772.
- Zargari Marandi R., Madeleine P., Omland Ø. et al. Eye movement characteristics reflected fatigue development in both young and elderly individuals // Sci. Rep. 2018. V. 8. № 1. P. 13148.
- Catalbas M.C., Cegovnik T., Sodnik J., Gulten A. Driver fatigue detection based on saccadic eye movements / 10th International Conference on Electrical and Electronics Engineering (ELECO). IEEE. 30 November – 2 December 2017. Bursa, Turkey, 2017. P. 913.
- Hu X., Lodewijks G. Exploration of the effects of task-related fatigue on eye-motion features and its value in improving driver fatigue-related technology // Transp. Res. Part F Traffic Psychol. Behav. 2021. V. 80. P. 150.
- Ahlstrom C., Nyström M., Holmqvist K. et al. Fit-for-duty test for estimation of drivers’ sleepiness level: Eye movements improve the sleep/wake predictor // Transp. Res. Part C Emerg. Technol. 2013. V. 26. P. 20.
- Abe T., Mishima K., Kitamura S. et al. Tracking intermediate performance of vigilant attention using multiple eye metrics // Sleep. 2020. V. 43. № 3. Р. zsz219.
- Di Stasi L.L., McCamy M.B., Macknik S.L. et al. Saccadic eye movement metrics reflect surgical residents’ fatigue // Ann. Surg. 2014. V. 259. № 4. P. 824.
- Di Stasi L.L., Renner R., Catena A. et al. Towards a driver fatigue test based on the saccadic main sequence: A partial validation by subjective report data // Transp. Res. Part C Emerg. Technol. 2012. V. 21. № 1. P. 122.
- Finke C., Pech L.M., Sömmer C. et al. Dynamics of saccade parameters in multiple sclerosis patients with fatigue // J. Neurol. 2012. V. 259. № 12. P. 2656.
- Renata V., Li F., Lee C.H., Chen C.H. Investigation on the correlation between eye movement and reaction time under mental fatigue influence / Proceedings of the 17th International Conference on Cyberworlds (CW 2018), Singapore 3-5 Oct 2018. Institute of Electrical and Electronics Engineers Inc., 2018. P. 207.
- Herlambang M.B., Taatgen N.A., Cnossen F. The role of motivation as a factor in mental fatigue // Hum. Factors. 2019. V. 61. № 7. P. 1171.
- Stone L.S., Tyson T.L., Cravalho P.F. et al. Distinct pattern of oculomotor impairment associated with acute sleep loss and circadian misalignment // J. Physiol. 2019. V. 597. № 17. P. 4643.
- Gergelyfi M., Jacob B., Olivier E., Zénon A. Dissociation between mental fatigue and motivational state during prolonged mental activity // Front. Behav. Neurosci. 2015. V. 9. P. 176.
- Schweitzer T., Wyss T., Gilgen-Ammann R. Detecting soldiers’ fatigue using eye-tracking glasses: Practical field applications and research opportunities // Mil. Med. 2022. V. 187. № 11–12. P. e1330.
- Borghini G., Astolfi L., Vecchiato G. et al. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness // Neurosci. Biobehav. Rev. 2014. V. 44. P. 58.
- Dziuda Ł., Baran P., Zieliński P. et al. Evaluation of a fatigue detector using eye closure-associated indicators acquired from truck drivers in a simulator study // Sensors. 2021. V. 21. № 19. P. 6449.
- Schleicher R., Galley N., Briest S., Galley L. Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? // Ergonomics. 2008. V. 51. № 7. P. 982.
- Hopstaken J.F., van der Linden D., Bakker A.B. et al. Shifts in attention during mental fatigue: Evidence from subjective, behavioral, physiological, and eye-tracking data // J. Exp. Psychol. Hum. Percept. Perform. 2016. V. 42. № 6. P. 878.
- Pomplun M., Sunkara S. Pupil dilation as an indicator of cognitive workload in human-computer interaction / Human-Centered Computing. Cognitive, Social, and Ergonomic Aspects // Eds. Harris D., Duffy V., Smith M., Stephanidis C. Boca Raton: CRC Press, 2019. V. 3. P. 542.
- Morad Y., Barkana Y., Zadok D. et al. Ocular parameters as an objective tool for the assessment of truck drivers’ fatigue // Accid. Anal. Prev. 2009. V. 41. № 4. P. 856.
- Di Stasi L.L., Marchitto M., Antolí A., Cañas J.J. Saccadic peak velocity as an alternative index of operator attention: A short review // Eur. Rev. Appl. Psychol. 2013. V. 63. № 6. P. 335.
- Diaz-Piedra C., Rieiro H., Suárez J. et al. Fatigue in the military: towards a fatigue detection test based on the saccadic velocity // Physiol. Meas. 2016. V. 37. № 9. P. N62.
- Ito J., Yamane Y., Suzuki M. et al. Switch from ambient to focal processing mode explains the dynamics of free viewing eye movements // Sci. Rep. 2017. V. 7. № 1. P. 1082.
- Pannasch S., Velichkovsky B.M. Distractor effect and saccade amplitudes: Further evidence on different modes of processing in free exploration of visual images // Vis. Cogn. 2009. V. 17. № 6-7. P. 1109.
- Величковский Б.М., Коростелева А.Н., Паннаш С. и др. Две системы зрения и их Движения глаз: эксперимент с фиксациями как событиями и сверхбыстрой фМРТ примиряет соперничающие взгляды // Современные технологии в медицине. 2019. Т. 11. № 4. С. 7.
- Шошина И.И., Шелепин Ю.Е. Механизмы глобального и локального анализа зрительной информации при шизофрении. СПб.: Изд-во ВВМ, 2016. 300 с.
- Шошина И.И., Мухитова Ю.В., Трегубенко И.А. и др. Контрастная чувствительность зрительной системы и когнитивные функции при шизофрении и депрессии // Физиология человека. 2021. Т. 47. № 5. С. 48.
- Milner A.D. How do the two visual streams interact with each other? // Exp. Brain Res. 2017. V. 235. № 5. P. 1297.
- Kunasegaran K., Ismail A.M.H., Ramasamy S. et al. Understanding mental fatigue and its detection: a comparative analysis of assessments and tools // Peer J. 2023. V. 11. P. e15744
- Tran Y., Craig A., Craig R. et al. The influence of mental fatigue on brain activity: Evidence from a systematic review with meta-analyses // Psychophysiology. 2020. V. 57. № 5. P. e13554.
- Hsu T.-Y., Hsu Y.-F., Wang H.-Y., Wang C.-A. Role of the frontal eye field in human pupil and saccade orienting responses // Eur. J. Neurosci. 2021. V. 54. P. 4283.
- Bafna T., Hansen J.P. Mental fatigue measurement using eye metrics: A systematic literature review // Psychophysiology. 2021. V. 58. № 6. P. e13828.
- Ansari M.F., Kasprowski P., Obetkal M. Gaze tracking using an unmodified web camera and convolutional neural network // Appl. Sci. 2021. V. 11. № 19. P. 9068.
- Naeeri S., Kang Z., Mandal S., Kim K. Multimodal analysis of eye movements and fatigue in a simulated glass cockpit environment // Aerospace. 2021. V. 8. № 10. P. 283.
- Mengtao L., Fan L., Gangyan X., Su H. Leveraging eye-tracking technologies to promote aviation safety – A review of key aspects, challenges, and future perspectives // Saf. Sci. 2023. V. 168. P. 106295.
- Hu X., Lodewijks G. Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: The value of differentiation of sleepiness and mental fatigue // J. Safety Res. 2020. V. 72. P. 173.
- Zhimin L., Ruilin L., Liqiang Y. et al. A benchmarking framework for eye-tracking-based vigilance prediction of vessel traffic controllers // Eng. Appl. Artif. Intell. 2024. V. 129. P. e107660.
Дополнительные файлы
