STFU NOOB! Predicting crowdsourced decisions on toxic behavior in online games

Jeremy Blackburn, Haewoon Kwak

Research output: Chapter in Book/Report/Conference proceedingConference contribution

46 Citations (Scopus)

Abstract

One problem facing players of competitive games is negative, or toxic, behavior. League of Legends, the largest eSport game, uses a crowdsourcing platform called the Tribunal to judge whether a reported toxic player should be punished or not. The Tribunal is a two stage system requiring reports from those players that directly observe toxic behavior, and human experts that review aggregated reports. While this system has successfully dealt with the vague nature of toxic behavior by majority rules based on many votes, it naturally requires tremendous cost, time, and human efforts. In this paper, we propose a supervised learning approach for predicting crowdsourced decisions on toxic behavior with largescale labeled data collections; over 10 million user reports involved in 1.46 million toxic players and corresponding crowdsourced decisions. Our result shows good performance in detecting overwhelmingly majority cases and predicting crowdsourced decisions on them. We demonstrate good portability of our classifier across regions. Finally, we estimate the practical implications of our approach, potential cost savings and victim protection. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

Original languageEnglish
Title of host publicationWWW 2014 - Proceedings of the 23rd International Conference on World Wide Web
PublisherAssociation for Computing Machinery, Inc
Pages877-887
Number of pages11
ISBN (Print)9781450327442
DOIs
Publication statusPublished - 1 Jan 2014
Externally publishedYes
Event23rd International Conference on World Wide Web, WWW 2014 - Seoul, Korea, Republic of
Duration: 7 Apr 201411 Apr 2014

Other

Other23rd International Conference on World Wide Web, WWW 2014
CountryKorea, Republic of
CitySeoul
Period7/4/1411/4/14

    Fingerprint

Keywords

  • Crowdsourcing
  • League of legends
  • Machine learning
  • Online video games
  • Toxic behavior

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Cite this

Blackburn, J., & Kwak, H. (2014). STFU NOOB! Predicting crowdsourced decisions on toxic behavior in online games. In WWW 2014 - Proceedings of the 23rd International Conference on World Wide Web (pp. 877-887). Association for Computing Machinery, Inc. https://doi.org/10.1145/2566486.2567987