Confusion prediction from eye-tracking data: Experiments with machine learning

Joni Salminen, Haewoon Kwak, Soon Gyo Jung, Mridul Nagpal, Jisun An, Bernard J. Jansen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70% when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90%. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.

Original languageEnglish
Title of host publicationProceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450362924
DOIs
Publication statusPublished - 24 Mar 2019
Event9th International Conference on Information Systems and Technologies, ICIST 2019 - Cairo, Egypt
Duration: 24 Mar 201926 Mar 2019

Publication series

NameACM International Conference Proceeding Series

Conference

Conference9th International Conference on Information Systems and Technologies, ICIST 2019
CountryEgypt
CityCairo
Period24/3/1926/3/19

Fingerprint

Learning systems
Sampling
Experiments
Application programs
Virtual reality
Interfaces (computer)
Websites

Keywords

  • Confusion detection
  • Eye tracking
  • Machine learning

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Salminen, J., Kwak, H., Jung, S. G., Nagpal, M., An, J., & Jansen, B. J. (2019). Confusion prediction from eye-tracking data: Experiments with machine learning. In Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019 [a5] (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3361570.3361577

Confusion prediction from eye-tracking data : Experiments with machine learning. / Salminen, Joni; Kwak, Haewoon; Jung, Soon Gyo; Nagpal, Mridul; An, Jisun; Jansen, Bernard J.

Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019. Association for Computing Machinery, 2019. a5 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Salminen, J, Kwak, H, Jung, SG, Nagpal, M, An, J & Jansen, BJ 2019, Confusion prediction from eye-tracking data: Experiments with machine learning. in Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019., a5, ACM International Conference Proceeding Series, Association for Computing Machinery, 9th International Conference on Information Systems and Technologies, ICIST 2019, Cairo, Egypt, 24/3/19. https://doi.org/10.1145/3361570.3361577
Salminen J, Kwak H, Jung SG, Nagpal M, An J, Jansen BJ. Confusion prediction from eye-tracking data: Experiments with machine learning. In Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019. Association for Computing Machinery. 2019. a5. (ACM International Conference Proceeding Series). https://doi.org/10.1145/3361570.3361577
Salminen, Joni ; Kwak, Haewoon ; Jung, Soon Gyo ; Nagpal, Mridul ; An, Jisun ; Jansen, Bernard J. / Confusion prediction from eye-tracking data : Experiments with machine learning. Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019. Association for Computing Machinery, 2019. (ACM International Conference Proceeding Series).
@inproceedings{6ce877a83677440ab66dce9743419cde,
title = "Confusion prediction from eye-tracking data: Experiments with machine learning",
abstract = "Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70{\%} when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90{\%}. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.",
keywords = "Confusion detection, Eye tracking, Machine learning",
author = "Joni Salminen and Haewoon Kwak and Jung, {Soon Gyo} and Mridul Nagpal and Jisun An and Jansen, {Bernard J.}",
year = "2019",
month = "3",
day = "24",
doi = "10.1145/3361570.3361577",
language = "English",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery",
booktitle = "Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019",

}

TY - GEN

T1 - Confusion prediction from eye-tracking data

T2 - Experiments with machine learning

AU - Salminen, Joni

AU - Kwak, Haewoon

AU - Jung, Soon Gyo

AU - Nagpal, Mridul

AU - An, Jisun

AU - Jansen, Bernard J.

PY - 2019/3/24

Y1 - 2019/3/24

N2 - Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70% when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90%. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.

AB - Predicting user confusion can help improve information presentation on websites, mobile apps, and virtual reality interfaces. One promising information source for such prediction is eye-tracking data about gaze movements on the screen. Coupled with think-aloud records, we explore if user's confusion is correlated with primarily fixation-level features. We find that random forest achieves an accuracy of more than 70% when prediction user confusion using only fixation features. In addition, adding user-level features (age and gender) improves the accuracy to more than 90%. We also find that balancing the classes before training improves performance. We test two balancing algorithms, Synthetic Minority Over Sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) finding that SMOTE provides a higher performance increase. Overall, this research contains implications for researchers interested in inferring users' cognitive states from eye-tracking data.

KW - Confusion detection

KW - Eye tracking

KW - Machine learning

UR - http://www.scopus.com/inward/record.url?scp=85076139447&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85076139447&partnerID=8YFLogxK

U2 - 10.1145/3361570.3361577

DO - 10.1145/3361570.3361577

M3 - Conference contribution

AN - SCOPUS:85076139447

T3 - ACM International Conference Proceeding Series

BT - Proceedings of the 9th International Conference on Information Systems and Technologies, ICIST 2019

PB - Association for Computing Machinery

ER -