IP Library Granted Patent US 11,594,049
Granted Patent B2
US 11,594,049 · App. 17/542,251 · Granted Feb 28, 2023

Image-based drive-thru management system

Inventors: Evan DeSantola (Pittsburgh, PA); Alex Litzenberger (Pittsburgh, PA); Rassoul Mesbah (Aukland, NZ); Prashasti Sar (Pittsburgh, PA)
Assignee: Agot Co.
G06V20/68G01B11/22G01S17/89G06F3/0482G06K9/6256G06N20/00G06Q10/063112G06Q20/20G06Q50/12G06T7/11G06T7/20G06T7/70G06V10/70G06V10/765G06V20/20G06V20/36G06V20/52G06V20/64H04N5/247H04N7/18G06T2200/24G06T2207/10028G06T2207/20081G06T2207/30232G06V2201/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,594,049
App. No.
17/542,251
Granted
Feb 28, 2023
Kind
B2
Abstract

The subject matter of this specification can be implemented in, among other things, methods, systems, computer-readable storage medium. A method can include receiving, by a processing device, image data including one or more image frames indicative of a current state of a drive-thru area. The processing device determines a vehicle disposed within the drive-thru area based on the image data. The processing device receives order data with a pending meal order. The processing device determines a first association between the vehicle and the pending meal order based on the image data. The processing devices determine a meal delivery procedure associated with the based on the association between the vehicle and the pending meal order. The processing device performs may perform the meal delivery procedure. The processing device may provide the meal delivery procedure for display on a graphical user interface (GUI).

Claims (126)

1. A method, comprising:

receiving, by a processing device, image data comprising one or more image frames indicative of a state of a drive-thru area;

determining, by the processing device, a vehicle disposed within the drive-thru area based on the image data;

receiving, by the processing device, order data indicative of a pending meal order;

determining, by the processing device, a first association between the vehicle and the pending meal order based on the image data;

receiving second image data comprising one or more image frames indicative of a state of a meal preparation area;

determining a meal preparation status of the pending meal order based on the second image data;

determining, by the processing device, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association and the meal preparation status of the pending meal order; and

performing, by the processing device, at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area.

2. The method of claim 1 , further comprising:

determining a predicted completion time of the pending meal order based on the second image data, wherein the meal preparation status of the pending meal order is further determined based on the predicted completion time.

3. The method of claim 2 , further comprises:

determining an expected arrival time of the vehicle at the first meal delivery area based on the image data; and

directing the vehicle to a second meal delivery area of the plurality of meal delivery areas responsive to determining the predicted completion time is later than the expected arrival time, wherein the second meal delivery area corresponds to a waiting bay.

4. The method of claim 1 , further comprising:

determining a second association between a first portion of the one or more image frames and a first lane of the drive-thru area; and

determining the vehicle is disposed within the first lane of the drive-thru area based on the image data and the second association, wherein the first association is determined responsive to determining the vehicle is disposed within the first lane.

5. The method of claim 4 , further comprising:

determining a third association between a second portion of the one or more image frames and a second lane of the drive-thru area;

determining the vehicle is disposed within the second lane of the drive-thru area based on the image data and the second association; and

determining the vehicle moved from the second lane to the first lane based on the image data, the second association, and the third association.

6. The method of claim 5 , wherein the first lane is a lane created by merging the second lane and a third lane within the drive-thru area.

7. The method of claim 1 , wherein determining the vehicle disposed within the drive-thru area further comprises:

using the image data as input to a machine learning model; and

obtaining one or more outputs of the machine learning model, the one or more outputs indicative of the vehicle disposed within the drive-thru area.

8. A method, comprising:

receiving, by a processing device, image data comprising one or more image frames indicative of a state of a drive-thru area;

determining, by the processing device, a vehicle disposed within the drive-thru area based on the image data;

receiving, by the processing device, order data indicative of a pending meal order;

determining, by the processing device, a first association between the vehicle and the pending meal order based on the image data;

determining, by the processing device, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association;

performing, by the processing device, at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area;

determining a visual vehicle identifier corresponding to the vehicle disposed within the drive-thru area based on the image data;

determining a menu display update based on the visual vehicle identifier; and

providing the menu display update for presentation on an order display device.

9. A method, comprising:

receiving, by a processing device, image data comprising one or more image frames indicative of a state of a drive-thru area;

determining, by the processing device, a vehicle disposed within the drive-thru area based on the image data;

receiving, by the processing device, order data indicative of a pending meal order;

determining, by the processing device, a first association between the vehicle and the pending meal order based on the image data;

determining, by the processing device, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association;

performing, by the processing device, at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area;

determining a visual vehicle identifier corresponding to the vehicle disposed within the drive-thru area based on the image data;

determining a second association between the pending meal order and a user account based on the visual vehicle identifier; and

determining an order transaction procedure based on the second association.

10. A system comprising:

a first camera to capture first image data comprising one or more image frames of a first field of view of a drive-thru area, the one or more image frames indicative of a current state of the drive-thru area;

a memory; and

a processing device, coupled to the memory; to:

receive, from the first camera, the first image data;

determine a vehicle disposed within the drive-thru area based on the first image data;

receive, by the processing device, order data indicative of a pending meal order;

determine a first association between the vehicle and the pending meal order based on the first image data;

receive second image data from a second camera comprising one or more image frames indicative of a state of a meal preparation area;

determine a meal preparation status of the pending meal order based on the second image data;

determine, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association and the meal preparation status of the pending meal order; and

perform at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area.

11. The system of claim 10 , wherein the processing device is further to:

determine a predicted completion time of the pending meal order based on the second image data, wherein the meal preparation status of the pending meal order is further determined using the predicted completion time.

12. The system of claim 10 , wherein the processing device is further to:

determine a second association between a first portion of the one or more image frames and a first lane of the drive-thru area; and

determine the vehicle is disposed within the first lane of the drive-thru area based on the first image data and the second association, wherein the first association is determined responsive to determining the vehicle is disposed within the first lane.

13. The system of claim 12 , wherein the processing device is further to:

determine a third association between a second portion of the one or more image frames and a second lane of the drive-thru area;

determine the vehicle is disposed within the second lane of the drive-thru area based on the first image data and the second association; and

determine the vehicle moved from the second lane to the first lane based on the first image data, the second association, and the third association.

14. The system of claim 10 , further comprising a second camera to capture second image data comprising one or more image frames of a second field of view of the drive-thru area, wherein the processing device is further to:

determine a reference object disposed within the drive-thru area based on the first image data and the second image data;

determine a second association between the reference object from the first image data and the reference object from the second image data; and

determine a relative position between the first camera and the second camera based on the second association.

15. A system comprising:

a first camera to capture first image data comprising one or more image frames of a first field of view of a drive-thru area, the one or more image frames indicative of a current state of the drive-thru area;

an order display device;

a memory; and

a processing device, coupled to the memory; to:

receive, from the first camera, the first image data;

determine a vehicle disposed within the drive-thru area based on the first image data;

receive, by the processing device, order data indicative of a pending meal order;

determine a first association between the vehicle and the pending meal order based on the first image data;

determine, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association; and

perform at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area;

determine a visual vehicle identifier corresponding to the vehicle disposed within the drive-thru area based on the first image data;

determine a menu display update based on the visual vehicle identifier; and

provide the menu display update for presentation on the order display device.

16. A system comprising:

a first camera to capture first image data comprising one or more image frames of a first field of view of a drive-thru area, the one or more image frames indicative of a current state of the drive-thru area;

a memory; and

a processing device, coupled to the memory; to:

receive, from the first camera, the first image data;

determine a vehicle disposed within the drive-thru area based on the first image data;

receive, by the processing device, order data indicative of a pending meal order;

determine a first association between the vehicle and the pending meal order based on the first image data;

determine, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association; and

perform at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area;

determine a visual vehicle identifier corresponding to the vehicle disposed within the drive-thru area based on the first image data;

determine a second association between the pending meal order and a user account based on the visual vehicle identifier; and

determine an order transaction procedure based on the second association.

17. A non-transitory machine-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

receiving image data comprising one or more image frames indicative of a state of a drive-thru area;

determining a vehicle disposed within the drive-thru area based on the image data;

receiving order data indicative of a pending meal order;

determining a first association between the vehicle and the pending meal order based on the image data;

receiving second image data comprising one or more image frames indicative of a state of a meal preparation area;

determining a meal preparation status of the pending meal order based on the second image data;

determining, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association and the meal preparation status of the pending meal order; and

performing at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area.

18. A non-transitory machine-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

receiving image data comprising one or more image frames indicative of a state of a drive-thru area;

determining a vehicle disposed within the drive-thru area based on the image data;

receiving order data indicative of a pending meal order;

determining a first association between the vehicle and the pending meal order based on the image data;

determining, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association;

performing at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area;

determining a visual vehicle identifier corresponding to the vehicle disposed within the drive-thru area a based on the image data;

determining a menu display update based on the visual vehicle identifier; and

providing the menu display update for presentation on an order display device.

19. A non-transitory machine-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

receiving image data comprising one or more image frames indicative of a state of a drive-thru area;

determining a vehicle disposed within the drive-thru area based on the image data;

receiving order data indicative of a pending meal order;

determining a first association between the vehicle and the pending meal order based on the image data;

determining, from a plurality of meal delivery areas, a first meal delivery area associated with the vehicle based on the first association;

performing at least one of a) causing performance of a meal delivery procedure using the first meal delivery area or b) communicating a notification associated with the first meal delivery area;

determining a visual vehicle identifier corresponding to the vehicle disposed within the drive-thru area a based on the image data;

determining a second association between the pending meal order and a user account based on the visual vehicle identifier; and

determining an order transaction procedure based on the second association.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 31, 2024
From: AGOT CO.
To: HME HOSPITALITY & SPECIALTY COMMUNICATIONS, INC.
Reel/Frame 067583/0916 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 26, 2022
From: DESANTOLA, EVAN; LITZENBERGER, ALEX; MESBAH, RASSOUL; SAR, PRASHASTI
To: AGOT CO.
Reel/Frame 060629/0010 →
Continuity (2)
Provisional Application 63160668 · Mar 12, 2021
Related Publication 20220292618A1 · Sep 15, 2022
Cited By (5)
US 12,327,230 US 12,381,673 US 12,400,279 US 12,579,528 US 12,700,961