Abstract
Hate is prevalent in online social media. This has resulted in a considerable amount of research in detecting and scoring it. Most computational efforts involve machine learning with crowdsourced ratings as training data. A prominent example of this is the Perspective API., a tool by Google to score toxicity of online comments. However., a major issue in the existing approaches is the lack of consideration for the subjective nature of online hate. While there is research that shows the intensity of hate varies and the hate depends on the context., there is no research that systematically investigates how hate interpretation varies by country or individual. In this exploratory research, we undertake this challenge. We sample crowd workers from 50 countries, have them score the same social media comments for toxicity and then evaluate the differences in the scores., altogether 18.,125 ratings. We find that the interpretation score differences among countries are highly significant. However., the hate interpretations vary more by the individual raters than by countries. These findings suggest that hate scoring systems should consider user-level features when scoring and automating the processing of online hate.
| Original language | English |
|---|---|
| Title of host publication | 2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 88-94 |
| Number of pages | 7 |
| ISBN (Electronic) | 9781538695883 |
| DOIs | |
| Publication status | Published - 30 Nov 2018 |
| Event | 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018 - Valencia, Spain Duration: 15 Oct 2018 → 18 Oct 2018 |
Other
| Other | 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018 |
|---|---|
| Country | Spain |
| City | Valencia |
| Period | 15/10/18 → 18/10/18 |
Fingerprint
Keywords
- hateinterpretation
- Online hate
- social media
ASJC Scopus subject areas
- Computer Networks and Communications
- Information Systems and Management
- Safety, Risk, Reliability and Quality
- Communication
Cite this
Online Hate Interpretation Varies by Country, but More by Individual : A Statistical Analysis Using Crowdsourced Ratings. / Salminen, Joni; Veronesi, Fabio; Almerekhi, Hind; Jung, Soon Gvo; Jansen, Bernard.
2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 88-94 8554954.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Online Hate Interpretation Varies by Country, but More by Individual
T2 - A Statistical Analysis Using Crowdsourced Ratings
AU - Salminen, Joni
AU - Veronesi, Fabio
AU - Almerekhi, Hind
AU - Jung, Soon Gvo
AU - Jansen, Bernard
PY - 2018/11/30
Y1 - 2018/11/30
N2 - Hate is prevalent in online social media. This has resulted in a considerable amount of research in detecting and scoring it. Most computational efforts involve machine learning with crowdsourced ratings as training data. A prominent example of this is the Perspective API., a tool by Google to score toxicity of online comments. However., a major issue in the existing approaches is the lack of consideration for the subjective nature of online hate. While there is research that shows the intensity of hate varies and the hate depends on the context., there is no research that systematically investigates how hate interpretation varies by country or individual. In this exploratory research, we undertake this challenge. We sample crowd workers from 50 countries, have them score the same social media comments for toxicity and then evaluate the differences in the scores., altogether 18.,125 ratings. We find that the interpretation score differences among countries are highly significant. However., the hate interpretations vary more by the individual raters than by countries. These findings suggest that hate scoring systems should consider user-level features when scoring and automating the processing of online hate.
AB - Hate is prevalent in online social media. This has resulted in a considerable amount of research in detecting and scoring it. Most computational efforts involve machine learning with crowdsourced ratings as training data. A prominent example of this is the Perspective API., a tool by Google to score toxicity of online comments. However., a major issue in the existing approaches is the lack of consideration for the subjective nature of online hate. While there is research that shows the intensity of hate varies and the hate depends on the context., there is no research that systematically investigates how hate interpretation varies by country or individual. In this exploratory research, we undertake this challenge. We sample crowd workers from 50 countries, have them score the same social media comments for toxicity and then evaluate the differences in the scores., altogether 18.,125 ratings. We find that the interpretation score differences among countries are highly significant. However., the hate interpretations vary more by the individual raters than by countries. These findings suggest that hate scoring systems should consider user-level features when scoring and automating the processing of online hate.
KW - hateinterpretation
KW - Online hate
KW - social media
UR - http://www.scopus.com/inward/record.url?scp=85060014671&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85060014671&partnerID=8YFLogxK
U2 - 10.1109/SNAMS.2018.8554954
DO - 10.1109/SNAMS.2018.8554954
M3 - Conference contribution
AN - SCOPUS:85060014671
SP - 88
EP - 94
BT - 2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018
PB - Institute of Electrical and Electronics Engineers Inc.
ER -