Investigating the accessibility of crowdwork tasks on Mechanical Turk

Stephen Uzor; Jason T. Jacques; John J. Dudley; Per Ola Kristensson

doi:10.1145/3411764.3445291

Investigating the accessibility of crowdwork tasks on Mechanical Turk

Stephen Uzor, Jason T. Jacques, John J. Dudley, Per Ola Kristensson

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

21 Citations (Scopus)

8 Downloads (Pure)

Abstract

Crowdwork can enable invaluable opportunities for people with disabilities, not least the work fexibility and the ability to work from home, especially during the current Covid-19 pandemic. This paper investigates how engagement in crowdwork tasks is affected by individual disabilities and the resulting implications for HCI. We first surveyed 1,000 Amazon Mechanical Turk (AMT) workers to identify demographics of crowdworkers who identify as having various disabilities within the AMT ecosystem-including vision, hearing, cognition/mental, mobility, reading and motor impairments. Through a second focused survey and follow-up interviews, we provide insights into how respondents cope with crowdwork tasks. We found that standard task factors, such as task completion time and presentation, often do not account for the needs of users with disabilities, resulting in anxiety and a feeling of depression on occasion. We discuss how to alleviate barriers to enable effective interaction for crowdworkers with disabilities.

Original language	English
Title of host publication	CHI '21
Subtitle of host publication	proceedings of the 2021 CHI conference on human factors in computing systems
Editors	Pernille Bjørn, Steven Drucker
Publisher	ACM
Number of pages	14
ISBN (Print)	9781450380966
DOIs	https://doi.org/10.1145/3411764.3445291
Publication status	Published - 6 May 2021
Event	2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths, - Virtual, Online, Japan Duration: 8 May 2021 → 13 May 2021 https://chi2021.acm.org/

Publication series

Name	Conference on human factors in computing systems - proceedings
ISSN (Print)	1062-9432

Conference

Conference	2021 CHI Conference on Human Factors in Computing Systems
Abbreviated title	CHI 2021
Country/Territory	Japan
City	Virtual, Online
Period	8/05/21 → 13/05/21
Internet address	https://chi2021.acm.org/

Keywords

Accessibility
Amt
Crowdsourcing
Crowdwork
Disability
MTurk

Access to Document

10.1145/3411764.3445291Licence: CC BY

Uzor_2022_Investigating-accessibility_CC
Copyright © 2021 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License.
Final published version, 1.11 MBLicence: CC BY

Cite this

Uzor, S., Jacques, J. T., Dudley, J. J., & Kristensson, P. O. (2021). Investigating the accessibility of crowdwork tasks on Mechanical Turk. In P. Bjørn, & S. Drucker (Eds.), CHI '21: proceedings of the 2021 CHI conference on human factors in computing systems Article 381 (Conference on human factors in computing systems - proceedings). ACM. https://doi.org/10.1145/3411764.3445291

@inproceedings{dad4d1a2ca1e44728a6a2c33d844e2bb,

title = "Investigating the accessibility of crowdwork tasks on Mechanical Turk",

abstract = "Crowdwork can enable invaluable opportunities for people with disabilities, not least the work fexibility and the ability to work from home, especially during the current Covid-19 pandemic. This paper investigates how engagement in crowdwork tasks is affected by individual disabilities and the resulting implications for HCI. We first surveyed 1,000 Amazon Mechanical Turk (AMT) workers to identify demographics of crowdworkers who identify as having various disabilities within the AMT ecosystem-including vision, hearing, cognition/mental, mobility, reading and motor impairments. Through a second focused survey and follow-up interviews, we provide insights into how respondents cope with crowdwork tasks. We found that standard task factors, such as task completion time and presentation, often do not account for the needs of users with disabilities, resulting in anxiety and a feeling of depression on occasion. We discuss how to alleviate barriers to enable effective interaction for crowdworkers with disabilities.",

keywords = "Accessibility, Amt, Crowdsourcing, Crowdwork, Disability, MTurk",

author = "Stephen Uzor and Jacques, {Jason T.} and Dudley, {John J.} and Kristensson, {Per Ola}",

note = "Funding: This work was supported by the EPSRC (grants EP/R004471/1 and EP/S027432/1). Supporting data for this publication is available at https://doi.org/10.17863/CAM.62937.; 2021 CHI Conference on Human Factors in Computing Systems : Making Waves, Combining Strengths, , CHI 2021 ; Conference date: 08-05-2021 Through 13-05-2021",

year = "2021",

month = may,

day = "6",

doi = "10.1145/3411764.3445291",

language = "English",

isbn = "9781450380966",

series = "Conference on human factors in computing systems - proceedings",

publisher = "ACM",

editor = "Pernille Bj{\o}rn and Steven Drucker",

booktitle = "CHI '21",

address = "United States",

url = "https://chi2021.acm.org/",

}

Uzor, S, Jacques, JT, Dudley, JJ & Kristensson, PO 2021, Investigating the accessibility of crowdwork tasks on Mechanical Turk. in P Bjørn & S Drucker (eds), CHI '21: proceedings of the 2021 CHI conference on human factors in computing systems., 381, Conference on human factors in computing systems - proceedings, ACM, 2021 CHI Conference on Human Factors in Computing Systems, Virtual, Online, Japan, 8/05/21. https://doi.org/10.1145/3411764.3445291

Investigating the accessibility of crowdwork tasks on Mechanical Turk. / Uzor, Stephen; Jacques, Jason T.; Dudley, John J. et al.
CHI '21: proceedings of the 2021 CHI conference on human factors in computing systems. ed. / Pernille Bjørn; Steven Drucker. ACM, 2021. 381 (Conference on human factors in computing systems - proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Investigating the accessibility of crowdwork tasks on Mechanical Turk

AU - Uzor, Stephen

AU - Jacques, Jason T.

AU - Dudley, John J.

AU - Kristensson, Per Ola

N1 - Funding: This work was supported by the EPSRC (grants EP/R004471/1 and EP/S027432/1). Supporting data for this publication is available at https://doi.org/10.17863/CAM.62937.

PY - 2021/5/6

Y1 - 2021/5/6

N2 - Crowdwork can enable invaluable opportunities for people with disabilities, not least the work fexibility and the ability to work from home, especially during the current Covid-19 pandemic. This paper investigates how engagement in crowdwork tasks is affected by individual disabilities and the resulting implications for HCI. We first surveyed 1,000 Amazon Mechanical Turk (AMT) workers to identify demographics of crowdworkers who identify as having various disabilities within the AMT ecosystem-including vision, hearing, cognition/mental, mobility, reading and motor impairments. Through a second focused survey and follow-up interviews, we provide insights into how respondents cope with crowdwork tasks. We found that standard task factors, such as task completion time and presentation, often do not account for the needs of users with disabilities, resulting in anxiety and a feeling of depression on occasion. We discuss how to alleviate barriers to enable effective interaction for crowdworkers with disabilities.

AB - Crowdwork can enable invaluable opportunities for people with disabilities, not least the work fexibility and the ability to work from home, especially during the current Covid-19 pandemic. This paper investigates how engagement in crowdwork tasks is affected by individual disabilities and the resulting implications for HCI. We first surveyed 1,000 Amazon Mechanical Turk (AMT) workers to identify demographics of crowdworkers who identify as having various disabilities within the AMT ecosystem-including vision, hearing, cognition/mental, mobility, reading and motor impairments. Through a second focused survey and follow-up interviews, we provide insights into how respondents cope with crowdwork tasks. We found that standard task factors, such as task completion time and presentation, often do not account for the needs of users with disabilities, resulting in anxiety and a feeling of depression on occasion. We discuss how to alleviate barriers to enable effective interaction for crowdworkers with disabilities.

KW - Accessibility

KW - Amt

KW - Crowdsourcing

KW - Crowdwork

KW - Disability

KW - MTurk

UR - https://dl.acm.org/doi/proceedings/10.1145/3411764

U2 - 10.1145/3411764.3445291

DO - 10.1145/3411764.3445291

M3 - Conference contribution

AN - SCOPUS:85106748823

SN - 9781450380966

T3 - Conference on human factors in computing systems - proceedings

BT - CHI '21

A2 - Bjørn, Pernille

A2 - Drucker, Steven

PB - ACM

T2 - 2021 CHI Conference on Human Factors in Computing Systems

Y2 - 8 May 2021 through 13 May 2021

ER -

Investigating the accessibility of crowdwork tasks on Mechanical Turk

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Mechanical Turk Accessibility (CHI 2021) Dataset

Cite this

Investigating the accessibility of crowdwork tasks on Mechanical Turk

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Datasets

Mechanical Turk Accessibility (CHI 2021) Dataset

Cite this