IP Library › Granted Patent US 11,842,543
Granted Patent B1
US 11,842,543 · App. 18/097,759 · Granted Dec 12, 2023

Camera system for providing an in-store experience to a remote user

Inventors: Natarajan Venkatakrishnan (Bellevue, WA); Kathryn K. Young (Seattle, WA)
Assignee: Starbucks Corporation
G06V20/52G06T3/0075G06V20/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,842,543
App. No.
18/097,759
Granted
Dec 12, 2023
Kind
B1
Abstract

A camera system can provide an in-store experience to a remote user according to some aspects described herein. In one example, a system can receive images from a camera that is oriented toward a display case at a physical location. The system can analyze the images to identify different types of objects in the display case. The system can then provide information about the different types of objects identified in the display case to a user device of a user that is remote from the physical location. The user device can receive the information and generate a graphical user interface that displays the information to the user.

Claims (61)

1. A method comprising:

receiving, by a server system, images from a camera that is oriented toward a display case at a physical location;

analyzing, by the server system, the images to identify a plurality of different types of objects in the display case;

providing, by the server system, information about the plurality of different types of objects identified in the display case to a user device of a user that is remote from the physical location, the user device being configured to generate a graphical user interface that displays the information to the user; and

providing, by the server system, at least one of the images to a drive-through terminal at a drive-through of the physical location for display to a driver of a vehicle at the drive-through.

2. The method of claim 1 , further comprising:

analyzing the images to identify at least one type of object that is absent from the display case; and

generating the information to indicate that the at least one type of object is absent from the display case.

3. The method of claim 1 , further comprising:

for each type of object identified in the display case, analyzing the images to determine a number of units of the type of object in the display case; and

generating the information to indicate the number of units of each type of object in the display case.

4. The method of claim 3 , further comprising:

updating a database to indicate the number of units of each type of object in the display case to track amounts of the plurality of different types of objects across a plurality of physical locations.

5. The method of claim 1 , wherein analyzing the images to identify the plurality of different types of objects in the display case involves:

identifying a plurality of pixel regions corresponding to a plurality of different types of food items in an image among the images, wherein the plurality of pixel regions are identified in the image using an image processing technique; and

analyzing the plurality of pixel regions to determine which types of food items are present in and absent from the display case.

6. The method of claim 1 , wherein the user device is configured to:

generate the graphical user interface to include a first interface page, the first interface page including a plurality of store locations that have corresponding graphical input elements;

detect a selection, by the user, of a graphical input element corresponding to a store location among the plurality of store locations, the store location corresponding to the physical location;

based on detecting the selection, receive the information from the server system; and

after receiving the information from the server system, generate a second interface page that includes display case data, wherein the display case data is generated based on the information.

7. The method of claim 6 , wherein the selection is a first selection, and wherein the user device is further configured to:

detect a second selection, by the user, associated with the display case data; and

based on detecting the second selection, output additional information about the display case in the graphical user interface, the additional information being different than the display case data.

8. The method of claim 7 , wherein the display case data is textual data and the additional information is an image of the display case.

9. The method of claim 6 , wherein the second interface page further includes seating availability data and queue length data associated with the store location.

10. The method of claim 1 , wherein the drive-through terminal is configured to concurrently display a menu of items for sale at the physical location and an image of the display case.

11. A system comprising:

one or more processors; and

one or more memories storing instructions that are executable by the one or more processors for causing the one or more processors to:

receive images from a camera that is oriented toward a display case at a physical location;

analyze the images to identify a plurality of different types of objects in the display case;

provide information about the plurality of different types of objects identified in the display case to a user device of a user that is remote from the physical location, the user device being configured to generate a graphical user interface that displays the information to the user; and

provide at least one of the images to a drive-through terminal at a drive-through of the physical location for display to a driver of a vehicle at the drive-through.

12. The system of claim 11 , wherein the one or more memories further include instructions that are executable by the one or more processors for causing the one or more processors to:

analyze the images to identify at least one type of object that is absent from the display case; and

generate the information to indicate that the at least one type of object is absent from the display case.

13. The system of claim 11 , wherein the one or more memories further include instructions that are executable by the one or more processors for causing the one or more processors to:

for each type of object identified in the display case, analyze the images to determine a number of units of the type of object in the display case; and

generate the information to indicate the number of units of each type of object in the display case.

14. The system of claim 11 , wherein the one or more memories further include instructions that are executable by the one or more processors for causing the one or more processors to:

update a database to indicate the number of units of each type of object in the display case to track amounts of the plurality of different types of objects across a plurality of physical locations.

15. The system of claim 11 , wherein the one or more memories further include instructions that are executable by the one or more processors for causing the one or more processors to:

identify a plurality of pixel regions corresponding to a plurality of different types of food items in an image among the images, wherein the plurality of pixel regions are identified in the image using an image processing technique; and

analyze the plurality of pixel regions to determine which types of food items are present in and absent from the display case.

16. A non-transitory computer-readable medium comprising program code that is executable by one or more processors for causing the one or more processors to:

receive images from a camera that is oriented toward a display case at a physical location;

analyze the images to identify a plurality of different types of objects in the display case;

provide information about the plurality of different types of objects identified in the display case to a user device of a user that is remote from the physical location, the user device being configured to generate a graphical user interface that displays the information to the user; and

provide at least one of the images to a drive-through terminal at a drive-through of the physical location for display to a driver of a vehicle at the drive-through.

17. The non-transitory computer-readable medium of claim 16 , further comprising program code that is executable by the one or more processors for causing the one or more processors to:

analyze the images to identify at least one type of object that is absent from the display case; and

generate the information to indicate that the at least one type of object is absent from the display case.

18. The non-transitory computer-readable medium of claim 16 , further comprising program code that is executable by the one or more processors for causing the one or more processors to:

for each type of object identified in the display case, analyze the images to determine a number of units of the type of object in the display case; and

generate the information to indicate the number of units of each type of object in the display case.

19. The non-transitory computer-readable medium of claim 16 , further comprising program code that is executable by the one or more processors for causing the one or more processors to:

update a database to indicate the number of units of each type of object in the display case to track amounts of the plurality of different types of objects across a plurality of physical locations.

20. The non-transitory computer-readable medium of claim 16 , further comprising program code that is executable by the one or more processors for causing the one or more processors to analyze the images to identify the plurality of different types of objects in the display case by:

identifying a plurality of pixel regions corresponding to a plurality of different types of food items in an image among the images, wherein the plurality of pixel regions are identified in the image using an image processing technique; and

analyzing the plurality of pixel regions to determine which types of food items are present in and absent from the display case.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2023
From: VENKATAKRISHNAN, VENKAT; YOUNG, KATHRYN
To: STARBUCKS CORPORATION
Reel/Frame 062408/0075 →