Crowd-guided ensembles

How can we choreograph crowd workers for video segmentation?

Alexandre Kaspar, Geneviève Patterson, Changil Kim, Yaǧiz Aksoy, Wojciech Matusik, Mohamed Elgharib

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In this work, we propose two ensemble methods leveraging a crowd workforce to improve video annotation, with a focus on video object segmentation. Their shared principle is that while individual candidate results may likely be insufficient, they often complement each other so that they can be combined into something better than any of the individual results-The very spirit of collaborative working. For one, we extend a standard polygon-drawing interface to allow workers to annotate negative space, and combine the work of multiple workers instead of relying on a single best one as commonly done in crowdsourced image segmentation. For the other, we present a method to combine multiple automatic propagation algorithms with the help of the crowd. Such combination requires an understanding of where the algorithms fail, which we gather using a novel coarse scribble video annotation task. We evaluate our ensemble methods, discuss our design choices for them, and make our web-based crowdsourcing tools and results publicly available.

Original languageEnglish
Title of host publicationCHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
Subtitle of host publicationEngage with CHI
PublisherAssociation for Computing Machinery
Volume2018-April
ISBN (Electronic)9781450356206, 9781450356213
DOIs
Publication statusPublished - 20 Apr 2018
Event2018 CHI Conference on Human Factors in Computing Systems, CHI 2018 - Montreal, Canada
Duration: 21 Apr 201826 Apr 2018

Other

Other2018 CHI Conference on Human Factors in Computing Systems, CHI 2018
CountryCanada
CityMontreal
Period21/4/1826/4/18

Fingerprint

Image segmentation

Keywords

  • Crowdsourcing
  • Ensemble methods
  • Keyframe segmentation
  • Segmentation propagation
  • Video object segmentation

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Kaspar, A., Patterson, G., Kim, C., Aksoy, Y., Matusik, W., & Elgharib, M. (2018). Crowd-guided ensembles: How can we choreograph crowd workers for video segmentation? In CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI (Vol. 2018-April). Association for Computing Machinery. https://doi.org/10.1145/3173574.3173685

Crowd-guided ensembles : How can we choreograph crowd workers for video segmentation? / Kaspar, Alexandre; Patterson, Geneviève; Kim, Changil; Aksoy, Yaǧiz; Matusik, Wojciech; Elgharib, Mohamed.

CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI. Vol. 2018-April Association for Computing Machinery, 2018.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kaspar, A, Patterson, G, Kim, C, Aksoy, Y, Matusik, W & Elgharib, M 2018, Crowd-guided ensembles: How can we choreograph crowd workers for video segmentation? in CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI. vol. 2018-April, Association for Computing Machinery, 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, Montreal, Canada, 21/4/18. https://doi.org/10.1145/3173574.3173685
Kaspar A, Patterson G, Kim C, Aksoy Y, Matusik W, Elgharib M. Crowd-guided ensembles: How can we choreograph crowd workers for video segmentation? In CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI. Vol. 2018-April. Association for Computing Machinery. 2018 https://doi.org/10.1145/3173574.3173685
Kaspar, Alexandre ; Patterson, Geneviève ; Kim, Changil ; Aksoy, Yaǧiz ; Matusik, Wojciech ; Elgharib, Mohamed. / Crowd-guided ensembles : How can we choreograph crowd workers for video segmentation?. CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems: Engage with CHI. Vol. 2018-April Association for Computing Machinery, 2018.
@inproceedings{be10b10cf0764abe8eba174408a0427e,
title = "Crowd-guided ensembles: How can we choreograph crowd workers for video segmentation?",
abstract = "In this work, we propose two ensemble methods leveraging a crowd workforce to improve video annotation, with a focus on video object segmentation. Their shared principle is that while individual candidate results may likely be insufficient, they often complement each other so that they can be combined into something better than any of the individual results-The very spirit of collaborative working. For one, we extend a standard polygon-drawing interface to allow workers to annotate negative space, and combine the work of multiple workers instead of relying on a single best one as commonly done in crowdsourced image segmentation. For the other, we present a method to combine multiple automatic propagation algorithms with the help of the crowd. Such combination requires an understanding of where the algorithms fail, which we gather using a novel coarse scribble video annotation task. We evaluate our ensemble methods, discuss our design choices for them, and make our web-based crowdsourcing tools and results publicly available.",
keywords = "Crowdsourcing, Ensemble methods, Keyframe segmentation, Segmentation propagation, Video object segmentation",
author = "Alexandre Kaspar and Genevi{\`e}ve Patterson and Changil Kim and Yaǧiz Aksoy and Wojciech Matusik and Mohamed Elgharib",
year = "2018",
month = "4",
day = "20",
doi = "10.1145/3173574.3173685",
language = "English",
volume = "2018-April",
booktitle = "CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Crowd-guided ensembles

T2 - How can we choreograph crowd workers for video segmentation?

AU - Kaspar, Alexandre

AU - Patterson, Geneviève

AU - Kim, Changil

AU - Aksoy, Yaǧiz

AU - Matusik, Wojciech

AU - Elgharib, Mohamed

PY - 2018/4/20

Y1 - 2018/4/20

N2 - In this work, we propose two ensemble methods leveraging a crowd workforce to improve video annotation, with a focus on video object segmentation. Their shared principle is that while individual candidate results may likely be insufficient, they often complement each other so that they can be combined into something better than any of the individual results-The very spirit of collaborative working. For one, we extend a standard polygon-drawing interface to allow workers to annotate negative space, and combine the work of multiple workers instead of relying on a single best one as commonly done in crowdsourced image segmentation. For the other, we present a method to combine multiple automatic propagation algorithms with the help of the crowd. Such combination requires an understanding of where the algorithms fail, which we gather using a novel coarse scribble video annotation task. We evaluate our ensemble methods, discuss our design choices for them, and make our web-based crowdsourcing tools and results publicly available.

AB - In this work, we propose two ensemble methods leveraging a crowd workforce to improve video annotation, with a focus on video object segmentation. Their shared principle is that while individual candidate results may likely be insufficient, they often complement each other so that they can be combined into something better than any of the individual results-The very spirit of collaborative working. For one, we extend a standard polygon-drawing interface to allow workers to annotate negative space, and combine the work of multiple workers instead of relying on a single best one as commonly done in crowdsourced image segmentation. For the other, we present a method to combine multiple automatic propagation algorithms with the help of the crowd. Such combination requires an understanding of where the algorithms fail, which we gather using a novel coarse scribble video annotation task. We evaluate our ensemble methods, discuss our design choices for them, and make our web-based crowdsourcing tools and results publicly available.

KW - Crowdsourcing

KW - Ensemble methods

KW - Keyframe segmentation

KW - Segmentation propagation

KW - Video object segmentation

UR - http://www.scopus.com/inward/record.url?scp=85046967600&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046967600&partnerID=8YFLogxK

U2 - 10.1145/3173574.3173685

DO - 10.1145/3173574.3173685

M3 - Conference contribution

VL - 2018-April

BT - CHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

ER -