Attention Assessment: Evaluation of Facial Expressions of Children with Autism Spectrum Disorder

Bilikis Banire, Dena Al Thani, Mustapha Makki, Marwa Qaraqe, Kruthika Anand, Olcay Connor, Kamran Khowaja, Bilal Mansoor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Technological interventions for teaching children with autism spectrum disorders (ASD) are becoming popular due to their potentials for sustaining the attention of children with rich multimedia and repetitive functionalities. The degree of attentiveness to these technological interventions differs from one child to another due to variability in the spectrum. Therefore, an objective approach, as opposed to the subjective type of attention assessment, becomes essential for automatically monitoring attention in order to design and develop adaptive learning tools, as well as to support caregivers to evaluate learning tools. The analysis of facial expressions recently emerged as an objective method of measuring attention and participation levels of typical learners. However, few studies have examined facial expressions of children with ASD during an attention task. Thus, this study aims to evaluate existing facial expression parameters developed by “affectiva”, a commercial engagement level measuring tool. We conducted fifteen experimental scenarios of 5 min each with 4 children with ASD and 4 typically developing children with an average age of 8.8 years, A desktop virtual reality-continuous performance task (VR-CPT) as attention stimuli and a webcam were used to stream real-time facial expressions. All the participants scored above average in the VR-CPT and the performance of the TD group was better than that of ASD. While 3 out of 10 facial expressions were prominent in the two groups, ASD group showed addition facial expression. Our findings showed that facial expression could serve as a biomarker for measuring attention differentiating the groups.

Original languageEnglish
Title of host publicationUniversal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings
EditorsMargherita Antona, Constantine Stephanidis
PublisherSpringer Verlag
Pages32-48
Number of pages17
ISBN (Print)9783030235628
DOIs
Publication statusPublished - 1 Jan 2019
Event13th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019 - Orlando, United States
Duration: 26 Jul 201931 Jul 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11573 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference13th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019
CountryUnited States
CityOrlando
Period26/7/1931/7/19

Fingerprint

Facial Expression
Disorder
Virtual reality
Evaluation
Biomarkers
Virtual Reality
Teaching
Monitoring
Adaptive Learning
Evaluate
Children
Multimedia
Real-time
Scenarios

Keywords

  • Adaptive learning
  • Affectiva
  • ASD
  • Attention
  • Facial expression
  • Virtual reality

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Banire, B., Al Thani, D., Makki, M., Qaraqe, M., Anand, K., Connor, O., ... Mansoor, B. (2019). Attention Assessment: Evaluation of Facial Expressions of Children with Autism Spectrum Disorder. In M. Antona, & C. Stephanidis (Eds.), Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings (pp. 32-48). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11573 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-23563-5_4

Attention Assessment : Evaluation of Facial Expressions of Children with Autism Spectrum Disorder. / Banire, Bilikis; Al Thani, Dena; Makki, Mustapha; Qaraqe, Marwa; Anand, Kruthika; Connor, Olcay; Khowaja, Kamran; Mansoor, Bilal.

Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. ed. / Margherita Antona; Constantine Stephanidis. Springer Verlag, 2019. p. 32-48 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11573 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Banire, B, Al Thani, D, Makki, M, Qaraqe, M, Anand, K, Connor, O, Khowaja, K & Mansoor, B 2019, Attention Assessment: Evaluation of Facial Expressions of Children with Autism Spectrum Disorder. in M Antona & C Stephanidis (eds), Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11573 LNCS, Springer Verlag, pp. 32-48, 13th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019, Orlando, United States, 26/7/19. https://doi.org/10.1007/978-3-030-23563-5_4
Banire B, Al Thani D, Makki M, Qaraqe M, Anand K, Connor O et al. Attention Assessment: Evaluation of Facial Expressions of Children with Autism Spectrum Disorder. In Antona M, Stephanidis C, editors, Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. Springer Verlag. 2019. p. 32-48. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-23563-5_4
Banire, Bilikis ; Al Thani, Dena ; Makki, Mustapha ; Qaraqe, Marwa ; Anand, Kruthika ; Connor, Olcay ; Khowaja, Kamran ; Mansoor, Bilal. / Attention Assessment : Evaluation of Facial Expressions of Children with Autism Spectrum Disorder. Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings. editor / Margherita Antona ; Constantine Stephanidis. Springer Verlag, 2019. pp. 32-48 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e8274af484704da5b51b69d0d8efa299,
title = "Attention Assessment: Evaluation of Facial Expressions of Children with Autism Spectrum Disorder",
abstract = "Technological interventions for teaching children with autism spectrum disorders (ASD) are becoming popular due to their potentials for sustaining the attention of children with rich multimedia and repetitive functionalities. The degree of attentiveness to these technological interventions differs from one child to another due to variability in the spectrum. Therefore, an objective approach, as opposed to the subjective type of attention assessment, becomes essential for automatically monitoring attention in order to design and develop adaptive learning tools, as well as to support caregivers to evaluate learning tools. The analysis of facial expressions recently emerged as an objective method of measuring attention and participation levels of typical learners. However, few studies have examined facial expressions of children with ASD during an attention task. Thus, this study aims to evaluate existing facial expression parameters developed by “affectiva”, a commercial engagement level measuring tool. We conducted fifteen experimental scenarios of 5{\^A} min each with 4 children with ASD and 4 typically developing children with an average age of 8.8 years, A desktop virtual reality-continuous performance task (VR-CPT) as attention stimuli and a webcam were used to stream real-time facial expressions. All the participants scored above average in the VR-CPT and the performance of the TD group was better than that of ASD. While 3 out of 10 facial expressions were prominent in the two groups, ASD group showed addition facial expression. Our findings showed that facial expression could serve as a biomarker for measuring attention differentiating the groups.",
keywords = "Adaptive learning, Affectiva, ASD, Attention, Facial expression, Virtual reality",
author = "Bilikis Banire and {Al Thani}, Dena and Mustapha Makki and Marwa Qaraqe and Kruthika Anand and Olcay Connor and Kamran Khowaja and Bilal Mansoor",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-23563-5_4",
language = "English",
isbn = "9783030235628",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "32--48",
editor = "Margherita Antona and Constantine Stephanidis",
booktitle = "Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings",

}

TY - GEN

T1 - Attention Assessment

T2 - Evaluation of Facial Expressions of Children with Autism Spectrum Disorder

AU - Banire, Bilikis

AU - Al Thani, Dena

AU - Makki, Mustapha

AU - Qaraqe, Marwa

AU - Anand, Kruthika

AU - Connor, Olcay

AU - Khowaja, Kamran

AU - Mansoor, Bilal

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Technological interventions for teaching children with autism spectrum disorders (ASD) are becoming popular due to their potentials for sustaining the attention of children with rich multimedia and repetitive functionalities. The degree of attentiveness to these technological interventions differs from one child to another due to variability in the spectrum. Therefore, an objective approach, as opposed to the subjective type of attention assessment, becomes essential for automatically monitoring attention in order to design and develop adaptive learning tools, as well as to support caregivers to evaluate learning tools. The analysis of facial expressions recently emerged as an objective method of measuring attention and participation levels of typical learners. However, few studies have examined facial expressions of children with ASD during an attention task. Thus, this study aims to evaluate existing facial expression parameters developed by “affectiva”, a commercial engagement level measuring tool. We conducted fifteen experimental scenarios of 5 min each with 4 children with ASD and 4 typically developing children with an average age of 8.8 years, A desktop virtual reality-continuous performance task (VR-CPT) as attention stimuli and a webcam were used to stream real-time facial expressions. All the participants scored above average in the VR-CPT and the performance of the TD group was better than that of ASD. While 3 out of 10 facial expressions were prominent in the two groups, ASD group showed addition facial expression. Our findings showed that facial expression could serve as a biomarker for measuring attention differentiating the groups.

AB - Technological interventions for teaching children with autism spectrum disorders (ASD) are becoming popular due to their potentials for sustaining the attention of children with rich multimedia and repetitive functionalities. The degree of attentiveness to these technological interventions differs from one child to another due to variability in the spectrum. Therefore, an objective approach, as opposed to the subjective type of attention assessment, becomes essential for automatically monitoring attention in order to design and develop adaptive learning tools, as well as to support caregivers to evaluate learning tools. The analysis of facial expressions recently emerged as an objective method of measuring attention and participation levels of typical learners. However, few studies have examined facial expressions of children with ASD during an attention task. Thus, this study aims to evaluate existing facial expression parameters developed by “affectiva”, a commercial engagement level measuring tool. We conducted fifteen experimental scenarios of 5 min each with 4 children with ASD and 4 typically developing children with an average age of 8.8 years, A desktop virtual reality-continuous performance task (VR-CPT) as attention stimuli and a webcam were used to stream real-time facial expressions. All the participants scored above average in the VR-CPT and the performance of the TD group was better than that of ASD. While 3 out of 10 facial expressions were prominent in the two groups, ASD group showed addition facial expression. Our findings showed that facial expression could serve as a biomarker for measuring attention differentiating the groups.

KW - Adaptive learning

KW - Affectiva

KW - ASD

KW - Attention

KW - Facial expression

KW - Virtual reality

UR - http://www.scopus.com/inward/record.url?scp=85069639093&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85069639093&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-23563-5_4

DO - 10.1007/978-3-030-23563-5_4

M3 - Conference contribution

AN - SCOPUS:85069639093

SN - 9783030235628

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 32

EP - 48

BT - Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments - 13th International Conference, UAHCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Proceedings

A2 - Antona, Margherita

A2 - Stephanidis, Constantine

PB - Springer Verlag

ER -