IP Library Granted Patent US 11,048,939
Granted Patent B2
US 11,048,939 · App. 16/124,439 · Granted Jun 29, 2021

Health trends identification

Inventors: Fang Lu (Billerica, MA); Italo Buleje (Orlando, FL); Jingwei Yang (Cambridge, MA); Piyush Madan (Boston, MA); Rachita Chandra (Cambridge, MA); Shilpa N. Mahatma (Chappaqua, NY); Sundar Saranathan (Framingham, MA)
Assignee: International Business Machines Corporation
G06K9/00664G06K9/6201G08B21/182G16H20/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,048,939
App. No.
16/124,439
Granted
Jun 29, 2021
Kind
B2
Abstract

Methods and apparatus for identifying health trends via the identification of ambiguous food items via container identification may be provided by: receiving an image captured from a user device that includes a food item; identifying a food container present in the image; identifying a utensil present in the image; determining a cluster of food items from a food recognition database corresponding to the food container and to the utensil; selecting a candidate food item from the cluster based on a confidence score of the candidate food item matching the food item; and adding the candidate food item to a dietary log associated with the user device.

Claims (84)

1. A method, comprising:

receiving an image captured from a user device that includes a first food item;

identifying a food container for a second food item that is present in the image;

identifying a utensil present in the image;

determining, from a food recognition database, a cluster of food items corresponding to both the food container for the second food item and the utensil;

selecting a candidate food item from the cluster based on a confidence score of the candidate food item matching the first food item; and

adding the candidate food item to a dietary log associated with the user device.

2. The method of claim 1 , wherein determining the cluster of food items from the food recognition database is done in response to the food recognition database returning a no-match result for an identity of the first food item via direct image recognition.

3. The method of claim 1 , wherein the food container is identified based on a size and shape of the food container and at least one of:

a color of the food container; and

a logo present on the food container.

4. The method of claim 1 , wherein the utensil is identified by distinguishing the utensil from between at least two of:

a null utensil;

a chopstick;

a fork;

a knife;

a spoon; and

a straw.

5. The method of claim 1 , further comprising determining whether the candidate food item triggers an alert threshold based on at least one of:

a medication included in the dietary log noted as conflicting with the candidate food item,

a diet plan included in the dietary log; and

in response to determining that the candidate food item triggers the alert threshold, transmitting an alert to the user device.

6. The method of claim 1 , further comprising determining whether the first food item triggers an alert threshold based on a nutritional value taken in aggregate with earlier-consumed food items in a predefined time period exceeding a trend threshold; and

in response to determining that the first food item triggers the alert threshold, transmitting an alert to the user device.

7. The method of claim 6 , wherein the alert is transmitted to a third party device associated with one of:

a doctor;

a parent; and

a nutritionist.

8. A system for health trend tracking, comprising:

a processor; and

a memory, including instructions that when executed by the processor enable the processor to:

receive an image captured from a user device that includes a first food item;

identify a food container for a second food item present in the image;

identify a utensil present in the image;

determine, from a food recognition database, a cluster of food items corresponding to both the food container for the second food item and the utensil;

select a candidate food item from the cluster based on a confidence score of the candidate food item matching the first food item; and

add the candidate food item to a dietary log associated with the user device.

9. The system of claim 8 , wherein determining a cluster of food items from the food recognition database is done in response to the food recognition database returning a no-match result for an identity of the first food item via direct image recognition.

10. The system of claim 8 , wherein the food container is identified based on a size and shape of the food container and at least one of:

a color of the food container; and

a logo present on the food container.

11. The system of claim 8 , wherein the utensil is identified by distinguishing the utensil from between at least two of: as one of:

a null utensil;

a chopstick;

a fork;

a knife;

a spoon; and

a straw.

12. The system of claim 8 , further comprising determining whether the candidate food item triggers an alert threshold based on at least one of:

a medication included in the dietary log noted as conflicting with the candidate food item,

a diet plan included in the dietary log; and

in response to determining that the candidate food item triggers the alert threshold, transmitting an alert to the user device.

13. The system of claim 8 , further comprising

determining whether the first food item triggers an alert threshold based a

nutritional value taken in aggregate with earlier-consumed food items in a

predefined time period exceeding a trend threshold; and

in response to determining that the first food item triggers the alert threshold, transmitting an alert to the user device.

14. The system of claim 13 , wherein the alert is transmitted to a third party device associated with one of:

a doctor;

a parent; and

a nutritionist.

15. A computer program product for health trend tracking, the computer program product comprising:

a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to:

receive an image captured from a user device that includes a food item, a secondary food item, and a utensil;

identify the secondary food item based on a food container for the secondary food item present in the image;

identify the utensil present in the image;

determine, from a food recognition database, a cluster of food items corresponding to both the secondary food item and the utensil;

select a candidate food item from the cluster based on a confidence score of the candidate food item matching the food item; and

add the candidate food item to a dietary log associated with the user device.

16. The computer program product of claim 15 , wherein determining a cluster of food items from the food recognition database is done in response to the food recognition database returning a no-match result for an identity of the food item via direct image recognition.

17. The computer program product of claim 15 , wherein the food container is identified based on a size and shape of the food container and at least one of:

a color of the food container; and

a logo present on the food container.

18. The computer program product of claim 15 , further comprising determining whether the candidate food item triggers an alert threshold based on at least one of:

a medication included in the dietary log noted as conflicting with the candidate food item,

a diet plan included in the dietary log; and

in response to determining that the candidate food item triggers the alert threshold, transmitting an alert to the user device.

19. The computer program product of claim 15 , further comprising determining whether the food item triggers an alert threshold based on

a nutritional value taken in aggregate with earlier-consumed food items in a predefined time period exceeding a trend threshold; and

in response to determining that the food item triggers the alert threshold, transmitting an alert to the user device.

20. The computer program product of claim 19 , wherein the alert is transmitted to a third party device associated with one of:

a doctor;

a parent; and

a nutritionist.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 7, 2018
From: LU, FANG; BULEJE, ITALO; YANG, JINGWEI; MADAN, PIYUSH; CHANDRA, RACHITA; MAHATMA, SHILPA; SARANTHAN, SUNDAR
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 046812/0072 →
Continuity (1)
Related Publication 20200082169A1 · Mar 12, 2020
Cited By (1)
US 12,332,873