Multimodality sensing for eating recognition

Christopher Merck, Christina Maher, Mark Mirtchouk, Min Zheng, Yuxiao Huang, Samantha Kleinberg

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

26 Scopus citations

Abstract

While many sensors can monitor physical activity, there is no device that can unobtrusively measure eating at the same level of detail. Yet, tracking and reacting to food consumption is key to managing many chronic diseases such as obesity and diabetes. Eating recognition has primarily used a single sensor at a time in a constrained environment but sensors may fail and each may pick up different types of eating. We present a multi-modality study of eating recognition, which combines head and wrist motion (Google Glass, smartwatches on each wrist), with audio (custom earbud microphone). We collect 72 hours of data from 6 participants wearing all sensors and eating an unrestricted set of foods, and annotate video recordings to obtain ground truth. Using our noise cancellation method, audio sensing alone achieved 92% precision and 89% recall in finding meals, while motion sensing was needed to find individual intakes.

Original languageEnglish
Title of host publicationPervasiveHealth 2016 - 10th EAI International Conference on Pervasive Computing Technologies for Healthcare
EditorsJesus Favela, Aleksander Matic, Geraldine Fitzpatrick, Nadir Weibel, Jesse Hoey
ISBN (Electronic)9781631900501
DOIs
StatePublished - 16 Jun 2016
Event10th EAI International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2016 - Cancun, Mexico
Duration: 16 May 201619 May 2016

Publication series

NamePervasiveHealth: Pervasive Computing Technologies for Healthcare
Volume2016-May
ISSN (Print)2153-1633
ISSN (Electronic)2153-1641

Conference

Conference10th EAI International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2016
Country/TerritoryMexico
CityCancun
Period16/05/1619/05/16

Keywords

  • Acoustic and motion sensing
  • Eating recognition

Fingerprint

Dive into the research topics of 'Multimodality sensing for eating recognition'. Together they form a unique fingerprint.

Cite this