Crowdsourcing the Perception of Machine Teaching

Jonggi Hong, Kyungjun Lee, June Xu, Hernisa Kacorri

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

27 Scopus citations

Abstract

Teachable interfaces can empower end-users to attune machine learning systems to their idiosyncratic characteristics and environment by explicitly providing pertinent training examples. While facilitating control, their effectiveness can be hindered by the lack of expertise or misconceptions. We investigate how users may conceptualize, experience, and reflect on their engagement in machine teaching by deploying a mobile teachable testbed in Amazon Mechanical Turk. Using a performance-based payment scheme, Mechanical Turkers (N=100) are called to train, test, and re-train a robust recognition model in real-time with a few snapshots taken in their environment. We find that participants incorporate diversity in their examples drawing from parallels to how humans recognize objects independent of size, viewpoint, location, and illumination. Many of their misconceptions relate to consistency and model capabilities for reasoning. With limited variation and edge cases in testing, the majority of them do not change strategies on a second training attempt.

Original languageEnglish
Title of host publicationCHI 2020 - Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
ISBN (Electronic)9781450367080
DOIs
StatePublished - 21 Apr 2020
Event2020 ACM CHI Conference on Human Factors in Computing Systems, CHI 2020 - Honolulu, United States
Duration: 25 Apr 202030 Apr 2020

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2020 ACM CHI Conference on Human Factors in Computing Systems, CHI 2020
Country/TerritoryUnited States
CityHonolulu
Period25/04/2030/04/20

Keywords

  • crowdsourcing
  • interactive machine learning
  • object recognition
  • personalization
  • teachable interfaces

Fingerprint

Dive into the research topics of 'Crowdsourcing the Perception of Machine Teaching'. Together they form a unique fingerprint.

Cite this