Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race

Soon Gyo Jung, Jisun An, Haewoon Kwak, Joni Salminen, Bernard Jansen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

In this research, we evaluate four widely used face detection tools, which are Face++, IBM Bluemix Visual Recognition, AWS Rekognition, and Microsoft Azure Face API, using multiple datasets to determine their accuracy in inferring user attributes, including gender, race, and age. Results show that the tools are generally proficient at determining gender, with accuracy rates greater than 90%, except for IBM Bluemix. Concerning race, only one of the four tools provides this capability, Face++, with an accuracy rate of greater than 90%, although the evaluation was performed on a high-quality dataset. Inferring age appears to be a challenging problem, as all four tools performed poorly. The findings of our quantitative evaluation are helpful for future computational social science research using these tools, as their accuracy needs to be taken into account when applied to classifying individuals on social media and other contexts. Triangulation and manual verification are suggested for researchers employing these tools.

Original languageEnglish
Title of host publication12th International AAAI Conference on Web and Social Media, ICWSM 2018
PublisherAAAI press
Pages624-627
Number of pages4
ISBN (Electronic)9781577357988
Publication statusPublished - 1 Jan 2018
Event12th International AAAI Conference on Web and Social Media, ICWSM 2018 - Palo Alto, United States
Duration: 25 Jun 201828 Jun 2018

Other

Other12th International AAAI Conference on Web and Social Media, ICWSM 2018
CountryUnited States
CityPalo Alto
Period25/6/1828/6/18

Fingerprint

Face recognition
Social sciences
Triangulation
Application programming interfaces (API)

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this

Jung, S. G., An, J., Kwak, H., Salminen, J., & Jansen, B. (2018). Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race. In 12th International AAAI Conference on Web and Social Media, ICWSM 2018 (pp. 624-627). AAAI press.

Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race. / Jung, Soon Gyo; An, Jisun; Kwak, Haewoon; Salminen, Joni; Jansen, Bernard.

12th International AAAI Conference on Web and Social Media, ICWSM 2018. AAAI press, 2018. p. 624-627.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Jung, SG, An, J, Kwak, H, Salminen, J & Jansen, B 2018, Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race. in 12th International AAAI Conference on Web and Social Media, ICWSM 2018. AAAI press, pp. 624-627, 12th International AAAI Conference on Web and Social Media, ICWSM 2018, Palo Alto, United States, 25/6/18.
Jung SG, An J, Kwak H, Salminen J, Jansen B. Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race. In 12th International AAAI Conference on Web and Social Media, ICWSM 2018. AAAI press. 2018. p. 624-627
Jung, Soon Gyo ; An, Jisun ; Kwak, Haewoon ; Salminen, Joni ; Jansen, Bernard. / Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race. 12th International AAAI Conference on Web and Social Media, ICWSM 2018. AAAI press, 2018. pp. 624-627
@inproceedings{0fac74ab67cd459cbb777057fb692cb3,
title = "Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race",
abstract = "In this research, we evaluate four widely used face detection tools, which are Face++, IBM Bluemix Visual Recognition, AWS Rekognition, and Microsoft Azure Face API, using multiple datasets to determine their accuracy in inferring user attributes, including gender, race, and age. Results show that the tools are generally proficient at determining gender, with accuracy rates greater than 90{\%}, except for IBM Bluemix. Concerning race, only one of the four tools provides this capability, Face++, with an accuracy rate of greater than 90{\%}, although the evaluation was performed on a high-quality dataset. Inferring age appears to be a challenging problem, as all four tools performed poorly. The findings of our quantitative evaluation are helpful for future computational social science research using these tools, as their accuracy needs to be taken into account when applied to classifying individuals on social media and other contexts. Triangulation and manual verification are suggested for researchers employing these tools.",
author = "Jung, {Soon Gyo} and Jisun An and Haewoon Kwak and Joni Salminen and Bernard Jansen",
year = "2018",
month = "1",
day = "1",
language = "English",
pages = "624--627",
booktitle = "12th International AAAI Conference on Web and Social Media, ICWSM 2018",
publisher = "AAAI press",

}

TY - GEN

T1 - Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race

AU - Jung, Soon Gyo

AU - An, Jisun

AU - Kwak, Haewoon

AU - Salminen, Joni

AU - Jansen, Bernard

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In this research, we evaluate four widely used face detection tools, which are Face++, IBM Bluemix Visual Recognition, AWS Rekognition, and Microsoft Azure Face API, using multiple datasets to determine their accuracy in inferring user attributes, including gender, race, and age. Results show that the tools are generally proficient at determining gender, with accuracy rates greater than 90%, except for IBM Bluemix. Concerning race, only one of the four tools provides this capability, Face++, with an accuracy rate of greater than 90%, although the evaluation was performed on a high-quality dataset. Inferring age appears to be a challenging problem, as all four tools performed poorly. The findings of our quantitative evaluation are helpful for future computational social science research using these tools, as their accuracy needs to be taken into account when applied to classifying individuals on social media and other contexts. Triangulation and manual verification are suggested for researchers employing these tools.

AB - In this research, we evaluate four widely used face detection tools, which are Face++, IBM Bluemix Visual Recognition, AWS Rekognition, and Microsoft Azure Face API, using multiple datasets to determine their accuracy in inferring user attributes, including gender, race, and age. Results show that the tools are generally proficient at determining gender, with accuracy rates greater than 90%, except for IBM Bluemix. Concerning race, only one of the four tools provides this capability, Face++, with an accuracy rate of greater than 90%, although the evaluation was performed on a high-quality dataset. Inferring age appears to be a challenging problem, as all four tools performed poorly. The findings of our quantitative evaluation are helpful for future computational social science research using these tools, as their accuracy needs to be taken into account when applied to classifying individuals on social media and other contexts. Triangulation and manual verification are suggested for researchers employing these tools.

UR - http://www.scopus.com/inward/record.url?scp=85050617087&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050617087&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85050617087

SP - 624

EP - 627

BT - 12th International AAAI Conference on Web and Social Media, ICWSM 2018

PB - AAAI press

ER -