Ag Data Commons
Browse

SNAPMe: A Benchmark Dataset of Food Photos with Food Records for Evaluation of Computer Vision Algorithms in the Context of Dietary Assessment

Download (1.89 GB)
dataset
posted on 2024-02-21, 16:56 authored by Elizabeth L. Chin, Jules A. Larke, Yasmine Y. Bouzid, Tu Nguyen, Yael Vainberg, Jennifer. T. Smilowitz, Danielle LemayDanielle Lemay

Photo-based dietary assessment methods are becoming more feasible as artificial intelligence methods improve. However, advancement of these methods to the level usable in nutrition studies has been hindered by the lack of a dataset against which to benchmark algorithm performance. We conducted the Surveying Nutrient Assessment with Photographs of Meals (SNAPMe) Study (ClinicalTrials ID: NCT05008653 ) to develop a benchmark dataset of food photographs paired with traditional food records. Participants were recruited nationally and completed enrollment meetings via web-based video conferencing. By the end of the study, 90 participants had completed all three days of data collection; 95 participants completed at least one study day. Participants uploaded and annotated their meal photos using a mobile phone app called Bitesnap and completed food records using the Automated Self-Administered 24-hour Dietary Assessment Tool (ASA24®) on the same day. A sizing marker with black and white boxes of known size were included in meal photos. Participants included photos “before” and “after” eating non-packaged and multi-serving packaged meals, as well as photos of the “front” package label and “ingredient” label for single-serving packaged foods. In total, the SNAPMe Database (DB) contains 3,311 unique food photos linked with 275 ASA24 food records from 95 participants who photographed all foods consumed and recorded food records in parallel for up to 3 study days each for a total of 275 diet days. The SNAPMe DB includes 1,475 “before” photos of non-packaged foods, 1,436 “after” photos of non-packaged foods, 203 “front” photos of packaged foods, and 196 “ingredient” labels of packaged foods. Each line item of each ASA24 food record is linked to the relevant photo. These data will be transformative for the improvement of artificial intelligence algorithms for the adoption of photo-based dietary assessment in nutrition research.


Resources in this dataset:

  • Resource Title: snapme_db_09Dec2022.tar.gz.

    File Name: snapme_db_09Dec2022.tar.gz

    Resource Description: This is a gzipped tarball. To expand and read the README, gunzip snapme_db_09Dec2022.tar.gz; tar -xvf snapme_db_09Dec2022.tar cd snapme_db_09Dec2022 less README.txt

Funding

USDA-NIFA: 2020-67021-32855/project accession no. 1024262

History

Data contact name

Lemay, Danielle

Data contact email

Danielle.Lemay@usda.gov

Publisher

Ag Data Commons

Intended use

This data set is intended to be used as a benchmark to evaluate models for food prediction, ingredient prediction, and/or nutrient prediction from photos of meals, snacks, and/or beverages.

Use limitations

This data set should not be used to train models as it is a small curated data set. Models trained with this small set of images would not be generalizable.

Temporal Extent Start Date

2021-09-01

Temporal Extent End Date

2022-01-29

Theme

  • Not specified

Geographic Coverage

{"type":"FeatureCollection","features":[{"geometry":{"type":"Polygon","coordinates":[[[232.91826725006,48.88023962065],[240.9441447258,30.271030292887],[284.02366161346,24.469142785384],[295.03037452698,45.509798058907],[263.05728435516,49.161101815285],[232.91826725006,48.88023962065]]]},"type":"Feature","properties":{}}]}

Geographic location - description

United States

ISO Topic Category

  • health

National Agricultural Library Thesaurus terms

data collection; food records; computer vision; algorithms; nutrition assessment; artificial intelligence; dietary surveys; photographs; traditional foods; Internet; mobile telephones; automation; ingestion; databases; nutrition research; models; prediction; ingredients; snacks; beverages; image analysis; meals (menu); portion size; test meals; United States

ARS National Program Number

  • 107

Pending citation

  • No

Public Access Level

  • Public

Preferred dataset citation

Chin, Elizabeth L.; Larke, Jules A.; Bouzid, Yasmine Y.; Nguyen, Tu; Vainberg, Yael; Smilowitz, Jennifer. T.; Lemay, Danielle G. (2022). SNAPMe: A Benchmark Dataset of Food Photos with Food Records for Evaluation of Computer Vision Algorithms in the Context of Dietary Assessment. Ag Data Commons. https://doi.org/10.15482/USDA.ADC/1528346