Whom should we sense in “social sensing” - analyzing which users work best for social media now-casting

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Given the ever increasing amount of publicly available social media data, there is growing interest in using online data to study and quantify phenomena in the offline “real” world. As social media data can be obtained in near real-time and at low cost, it is often used for “now-casting” indices such as levels of flu activity or unemployment. The term “social sensing” is often used in this context to describe the idea that users act as “sensors”, publicly reporting their health status or job losses. Sensor activity during a time period is then typically aggregated in a “one tweet, one vote” fashion by simply counting. At the same time, researchers readily admit that social media users are not a perfect representation of the actual population. Additionally, users differ in the amount of details of their personal lives that they reveal. Intuitively, it should be possible to improve now-casting by assigning different weights to different user groups. In this paper, we ask “How does social sensing actually work?” or, more precisely, “Whom should we sense-and whom not-for optimal results?”. We investigate how different sampling strategies affect the performance of now-casting of two common offline indices: flu activity and unemployment rate. We show that now-casting can be improved by (1) applying user filtering techniques and (2) selecting users with complete profiles. We also find that, using the right type of user groups, now-casting performance does not degrade, even when drastically reducing the size of the dataset. More fundamentally, we describe which type of users contribute most to the accuracy by asking if “babblers are better”. We conclude the paper by providing guidance on how to select better user groups for more accurate now-casting.

Original languageEnglish
Article number22
Pages (from-to)1-22
Number of pages22
JournalEPJ Data Science
Volume4
Issue number1
DOIs
Publication statusPublished - 1 Dec 2015

Fingerprint

Social Media
Casting
Sensing
Unemployment
Sensors
Sensor
Sampling Strategy
Health
Vote
Sampling
Guidance
Counting
Quantify
Filtering
Real-time
Costs

Keywords

  • flu
  • nowcasting
  • prediction
  • sampling
  • social media
  • Twitter
  • unemployment rate

ASJC Scopus subject areas

  • Computer Science Applications
  • Computational Mathematics
  • Modelling and Simulation

Cite this

Whom should we sense in “social sensing” - analyzing which users work best for social media now-casting. / An, Jisun; Weber, Ingmar.

In: EPJ Data Science, Vol. 4, No. 1, 22, 01.12.2015, p. 1-22.

Research output: Contribution to journalArticle

@article{d4a67f0b7e2b4dac9e52ec44eeb15308,
title = "Whom should we sense in “social sensing” - analyzing which users work best for social media now-casting",
abstract = "Given the ever increasing amount of publicly available social media data, there is growing interest in using online data to study and quantify phenomena in the offline “real” world. As social media data can be obtained in near real-time and at low cost, it is often used for “now-casting” indices such as levels of flu activity or unemployment. The term “social sensing” is often used in this context to describe the idea that users act as “sensors”, publicly reporting their health status or job losses. Sensor activity during a time period is then typically aggregated in a “one tweet, one vote” fashion by simply counting. At the same time, researchers readily admit that social media users are not a perfect representation of the actual population. Additionally, users differ in the amount of details of their personal lives that they reveal. Intuitively, it should be possible to improve now-casting by assigning different weights to different user groups. In this paper, we ask “How does social sensing actually work?” or, more precisely, “Whom should we sense-and whom not-for optimal results?”. We investigate how different sampling strategies affect the performance of now-casting of two common offline indices: flu activity and unemployment rate. We show that now-casting can be improved by (1) applying user filtering techniques and (2) selecting users with complete profiles. We also find that, using the right type of user groups, now-casting performance does not degrade, even when drastically reducing the size of the dataset. More fundamentally, we describe which type of users contribute most to the accuracy by asking if “babblers are better”. We conclude the paper by providing guidance on how to select better user groups for more accurate now-casting.",
keywords = "flu, nowcasting, prediction, sampling, social media, Twitter, unemployment rate",
author = "Jisun An and Ingmar Weber",
year = "2015",
month = "12",
day = "1",
doi = "10.1140/epjds/s13688-015-0058-9",
language = "English",
volume = "4",
pages = "1--22",
journal = "EPJ Data Science",
issn = "2193-1127",
publisher = "Springer Science + Business Media",
number = "1",

}

TY - JOUR

T1 - Whom should we sense in “social sensing” - analyzing which users work best for social media now-casting

AU - An, Jisun

AU - Weber, Ingmar

PY - 2015/12/1

Y1 - 2015/12/1

N2 - Given the ever increasing amount of publicly available social media data, there is growing interest in using online data to study and quantify phenomena in the offline “real” world. As social media data can be obtained in near real-time and at low cost, it is often used for “now-casting” indices such as levels of flu activity or unemployment. The term “social sensing” is often used in this context to describe the idea that users act as “sensors”, publicly reporting their health status or job losses. Sensor activity during a time period is then typically aggregated in a “one tweet, one vote” fashion by simply counting. At the same time, researchers readily admit that social media users are not a perfect representation of the actual population. Additionally, users differ in the amount of details of their personal lives that they reveal. Intuitively, it should be possible to improve now-casting by assigning different weights to different user groups. In this paper, we ask “How does social sensing actually work?” or, more precisely, “Whom should we sense-and whom not-for optimal results?”. We investigate how different sampling strategies affect the performance of now-casting of two common offline indices: flu activity and unemployment rate. We show that now-casting can be improved by (1) applying user filtering techniques and (2) selecting users with complete profiles. We also find that, using the right type of user groups, now-casting performance does not degrade, even when drastically reducing the size of the dataset. More fundamentally, we describe which type of users contribute most to the accuracy by asking if “babblers are better”. We conclude the paper by providing guidance on how to select better user groups for more accurate now-casting.

AB - Given the ever increasing amount of publicly available social media data, there is growing interest in using online data to study and quantify phenomena in the offline “real” world. As social media data can be obtained in near real-time and at low cost, it is often used for “now-casting” indices such as levels of flu activity or unemployment. The term “social sensing” is often used in this context to describe the idea that users act as “sensors”, publicly reporting their health status or job losses. Sensor activity during a time period is then typically aggregated in a “one tweet, one vote” fashion by simply counting. At the same time, researchers readily admit that social media users are not a perfect representation of the actual population. Additionally, users differ in the amount of details of their personal lives that they reveal. Intuitively, it should be possible to improve now-casting by assigning different weights to different user groups. In this paper, we ask “How does social sensing actually work?” or, more precisely, “Whom should we sense-and whom not-for optimal results?”. We investigate how different sampling strategies affect the performance of now-casting of two common offline indices: flu activity and unemployment rate. We show that now-casting can be improved by (1) applying user filtering techniques and (2) selecting users with complete profiles. We also find that, using the right type of user groups, now-casting performance does not degrade, even when drastically reducing the size of the dataset. More fundamentally, we describe which type of users contribute most to the accuracy by asking if “babblers are better”. We conclude the paper by providing guidance on how to select better user groups for more accurate now-casting.

KW - flu

KW - nowcasting

KW - prediction

KW - sampling

KW - social media

KW - Twitter

KW - unemployment rate

UR - http://www.scopus.com/inward/record.url?scp=84958754340&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84958754340&partnerID=8YFLogxK

U2 - 10.1140/epjds/s13688-015-0058-9

DO - 10.1140/epjds/s13688-015-0058-9

M3 - Article

AN - SCOPUS:84958754340

VL - 4

SP - 1

EP - 22

JO - EPJ Data Science

JF - EPJ Data Science

SN - 2193-1127

IS - 1

M1 - 22

ER -