Task assignment optimization in collaborative crowdsourcing

Habibur Rahman, Senjuti Basu Roy, Saravanan Thirumuruganathan, Sihem Amer-Yahia, Gautam Das

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Citations (Scopus)

Abstract

A number of emerging applications, such as, collaborative document editing, sentence translation, and citizen journalism require workers with complementary skills and expertise to form groups and collaborate on complex tasks. While existing research has investigated task assignment for knowledge intensive crowdsourcing, they often ignore the aspect of collaboration among workers, that is central to the success of such tasks. Research in behavioral psychology has indicated that large groups hinder successful collaboration. Taking that into consideration, our work is one of the first to investigate and formalize the notion of collaboration among workers and present theoretical analyses to understand the hardness of optimizing task assignment. We propose efficient approximation algorithms with provable theoretical guarantees and demonstrate the superiority of our algorithms through a comprehensive set of experiments using real-world and synthetic datasets. Finally, we conduct a real world collaborative sentence translation application using Amazon Mechanical Turk that we hope provides a template for evaluating collaborative crowdsourcing tasks in micro-task based crowdsourcing platforms.

Original languageEnglish
Title of host publicationProceedings - 15th IEEE International Conference on Data Mining, ICDM 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages949-954
Number of pages6
Volume2016-January
ISBN (Electronic)9781467395038
DOIs
Publication statusPublished - 5 Jan 2016
Externally publishedYes
Event15th IEEE International Conference on Data Mining, ICDM 2015 - Atlantic City, United States
Duration: 14 Nov 201517 Nov 2015

Other

Other15th IEEE International Conference on Data Mining, ICDM 2015
CountryUnited States
CityAtlantic City
Period14/11/1517/11/15

Fingerprint

Approximation algorithms
Hardness
Experiments

Keywords

  • Algorithms
  • Collaborative crowdsourcing
  • Crowdsourcing
  • Optimization

ASJC Scopus subject areas

  • Engineering(all)

Cite this

Rahman, H., Roy, S. B., Thirumuruganathan, S., Amer-Yahia, S., & Das, G. (2016). Task assignment optimization in collaborative crowdsourcing. In Proceedings - 15th IEEE International Conference on Data Mining, ICDM 2015 (Vol. 2016-January, pp. 949-954). [7373417] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2015.119

Task assignment optimization in collaborative crowdsourcing. / Rahman, Habibur; Roy, Senjuti Basu; Thirumuruganathan, Saravanan; Amer-Yahia, Sihem; Das, Gautam.

Proceedings - 15th IEEE International Conference on Data Mining, ICDM 2015. Vol. 2016-January Institute of Electrical and Electronics Engineers Inc., 2016. p. 949-954 7373417.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rahman, H, Roy, SB, Thirumuruganathan, S, Amer-Yahia, S & Das, G 2016, Task assignment optimization in collaborative crowdsourcing. in Proceedings - 15th IEEE International Conference on Data Mining, ICDM 2015. vol. 2016-January, 7373417, Institute of Electrical and Electronics Engineers Inc., pp. 949-954, 15th IEEE International Conference on Data Mining, ICDM 2015, Atlantic City, United States, 14/11/15. https://doi.org/10.1109/ICDM.2015.119
Rahman H, Roy SB, Thirumuruganathan S, Amer-Yahia S, Das G. Task assignment optimization in collaborative crowdsourcing. In Proceedings - 15th IEEE International Conference on Data Mining, ICDM 2015. Vol. 2016-January. Institute of Electrical and Electronics Engineers Inc. 2016. p. 949-954. 7373417 https://doi.org/10.1109/ICDM.2015.119
Rahman, Habibur ; Roy, Senjuti Basu ; Thirumuruganathan, Saravanan ; Amer-Yahia, Sihem ; Das, Gautam. / Task assignment optimization in collaborative crowdsourcing. Proceedings - 15th IEEE International Conference on Data Mining, ICDM 2015. Vol. 2016-January Institute of Electrical and Electronics Engineers Inc., 2016. pp. 949-954
@inproceedings{c9c1be142e374132af3ae3be62723445,
title = "Task assignment optimization in collaborative crowdsourcing",
abstract = "A number of emerging applications, such as, collaborative document editing, sentence translation, and citizen journalism require workers with complementary skills and expertise to form groups and collaborate on complex tasks. While existing research has investigated task assignment for knowledge intensive crowdsourcing, they often ignore the aspect of collaboration among workers, that is central to the success of such tasks. Research in behavioral psychology has indicated that large groups hinder successful collaboration. Taking that into consideration, our work is one of the first to investigate and formalize the notion of collaboration among workers and present theoretical analyses to understand the hardness of optimizing task assignment. We propose efficient approximation algorithms with provable theoretical guarantees and demonstrate the superiority of our algorithms through a comprehensive set of experiments using real-world and synthetic datasets. Finally, we conduct a real world collaborative sentence translation application using Amazon Mechanical Turk that we hope provides a template for evaluating collaborative crowdsourcing tasks in micro-task based crowdsourcing platforms.",
keywords = "Algorithms, Collaborative crowdsourcing, Crowdsourcing, Optimization",
author = "Habibur Rahman and Roy, {Senjuti Basu} and Saravanan Thirumuruganathan and Sihem Amer-Yahia and Gautam Das",
year = "2016",
month = "1",
day = "5",
doi = "10.1109/ICDM.2015.119",
language = "English",
volume = "2016-January",
pages = "949--954",
booktitle = "Proceedings - 15th IEEE International Conference on Data Mining, ICDM 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Task assignment optimization in collaborative crowdsourcing

AU - Rahman, Habibur

AU - Roy, Senjuti Basu

AU - Thirumuruganathan, Saravanan

AU - Amer-Yahia, Sihem

AU - Das, Gautam

PY - 2016/1/5

Y1 - 2016/1/5

N2 - A number of emerging applications, such as, collaborative document editing, sentence translation, and citizen journalism require workers with complementary skills and expertise to form groups and collaborate on complex tasks. While existing research has investigated task assignment for knowledge intensive crowdsourcing, they often ignore the aspect of collaboration among workers, that is central to the success of such tasks. Research in behavioral psychology has indicated that large groups hinder successful collaboration. Taking that into consideration, our work is one of the first to investigate and formalize the notion of collaboration among workers and present theoretical analyses to understand the hardness of optimizing task assignment. We propose efficient approximation algorithms with provable theoretical guarantees and demonstrate the superiority of our algorithms through a comprehensive set of experiments using real-world and synthetic datasets. Finally, we conduct a real world collaborative sentence translation application using Amazon Mechanical Turk that we hope provides a template for evaluating collaborative crowdsourcing tasks in micro-task based crowdsourcing platforms.

AB - A number of emerging applications, such as, collaborative document editing, sentence translation, and citizen journalism require workers with complementary skills and expertise to form groups and collaborate on complex tasks. While existing research has investigated task assignment for knowledge intensive crowdsourcing, they often ignore the aspect of collaboration among workers, that is central to the success of such tasks. Research in behavioral psychology has indicated that large groups hinder successful collaboration. Taking that into consideration, our work is one of the first to investigate and formalize the notion of collaboration among workers and present theoretical analyses to understand the hardness of optimizing task assignment. We propose efficient approximation algorithms with provable theoretical guarantees and demonstrate the superiority of our algorithms through a comprehensive set of experiments using real-world and synthetic datasets. Finally, we conduct a real world collaborative sentence translation application using Amazon Mechanical Turk that we hope provides a template for evaluating collaborative crowdsourcing tasks in micro-task based crowdsourcing platforms.

KW - Algorithms

KW - Collaborative crowdsourcing

KW - Crowdsourcing

KW - Optimization

UR - http://www.scopus.com/inward/record.url?scp=84963621661&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84963621661&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2015.119

DO - 10.1109/ICDM.2015.119

M3 - Conference contribution

VL - 2016-January

SP - 949

EP - 954

BT - Proceedings - 15th IEEE International Conference on Data Mining, ICDM 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -