Detecting Granular Eating Behaviors from Body-worn Audio and Motion Sensors

Wearable sensor technology has made it possible to gain insight into dietary activity, learning not only when people are eating, but identifying fine-grained behaviors such as chews per minute, and causes of food choices. This may enable interventions to maintain health and assist individuals with chronic diseases such as diabetes (e.g. by providing insulin dosing assistance). However, existing work on dietary monitoring has focused on identifying meal times, rather than fine grained behavior such as chewing. A key barrier is the difficulty of obtaining granular ground truth. In free-living environments it is difficult to obtain the high-quality video needed, and annotating large datasets is labor intensive and does not scale well. To address this, we introduce a new multi-stage initialization approach for Stochastic Variational Deep Kernel Learning (SVDKL) that enables learning from data with a mix of coarse labels (meal times) and granular ones (chews, intakes). Our approach outperforms the state of the art on both free-living and laboratory datasets, with 84% recall and 67% precision for detecting chews compared to prior results of 73% precision and 34% recall on the same data. Ultimately, our work may enable more types of human activity recognition from real-world environments at a lower cost.