Computerized image analysis for automatically determining wait times for a queue area
A computer-implemented method allows a wait time to be determined automatically for a queue area. A series of images showing the environment are received over time. A wait time associated with the queue area is determined by detecting a location of a object corresponding to a person in a first one of the images; associating the object with an identifier uniquely identifying the object in the first one of the images matching objects in later images; and determining the wait time based on times associated with an image in which an object associated with the identifier enters the queue area through the defined entrance area and later one of the images in which an object associated with the identifier exits the queue area through the defined exit area. An indication of the wait time is output. Machine learning may be used to extract features of an object from an image.
1 . A computer system comprising:
at least one processor; and
a memory module coupled to the at least one processor and storing instructions that, when executed by the at least one processor, cause the computer system to:
receive respective images from a plurality of cameras;
synchronize the plurality of cameras based on respective timestamps associated with the respective images;
pass respective outputs of a plurality of object detection models trained using a data set corresponding to different vantage points of the plurality of cameras, to respective intra-feed identity models to track only within a queue area detected objects by the plurality of object detection models across sequential images of a plurality of series of images received from the synchronized cameras to maintain an identity of the detected objects as the detected objects move through time and space in the queue area;
pass outputs of the respective intra-feed identity models, to an inter-feed identity model that matches respective objects present in the plurality of series of images to identify new objects and identify previous objects in the plurality of series of images and to identify a common identifier across and within each series of the plurality of series of images; and
compare a last timestamp of a last detection of a first detected object of the detected objects when the first detected object has left the queue area, to a logged timestamp for the common identifier to determine, in real time and without requiring extensive computational resources, a wait time corresponding to how long the first detected object has been in the queue area.
2 . The computer system of claim 1 , wherein synchronizing the plurality of cameras is further based on frame rates of each of the plurality of cameras.
3 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
create, by object localization of the computer system, proposed regions that contain distinct objects; and
determine, by object classification of the computer system, detected objects among the distinct objects in the proposed regions.
4 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
log, by the computer system, a timestamp of the respective timestamps, for the detected objects with bounding boxes falling inside a bounding box of a queue area in association with a unique identifier of an object first detected in the queue area, and discarding, after a timeout period, those of the bounding boxes falling outside the bounding box of the queue area, and those of the detected objects remaining in the queue area for a prolonged period.
5 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
determine, by the computer system, respective outputs of the plurality of object detection models that identify bounding boxes for the detected objects.
6 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
delineate, by the respective intra-feed identity models executed by the computer system, bounding boxes that correspond to specific objects matching those objects detected in a previous frame and bounding boxes corresponding to a new object.
7 . The computer system of claim 6 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
reclassify, by the respective intra-feed identity models, bounding boxes provided by the plurality of object detection models, by:
associating the bounding boxes with a unique identifier;
assigning to the detected objects in the sequential images, the unique identifier as the first detected object so that identity of the detected objects is persisted across the sequential images of the series of the respective images;
assigning, upon first appearance of the new object, a new unique identifier to the new object and the matching objects in the sequential images of a plurality of series of images; and
correlating the bounding box of the first detected object with the bounding boxes corresponding to the objects matching the first detected object in earlier frames as the first detected object moves through the queue area and the subsequent images.
8 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to train the inter-feed identity model with a convolutional neural network on a labeled dataset of the plurality of cameras.
9 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
extract a fingerprint for each of the detected objects;
perform clustering on the extracted fingerprints; and
identify the clustered fingerprints as belonging to a same object.
10 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
correlate, using a terminal associated with a service position in the queue area, a specific time and account associated with the determined wait time of the first detected object, based on transactions performed in association with the queue area during a time window defined by times associated with the first detected object entering and leaving the queue area.
11 . The computer system of claim 1 , wherein the inter-feed identity model is a machine learning model used to extract features of an object from an image.
12 . The computer system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the computer system to:
determine that a transaction associated with the wait time is also associated with providing, by the computer system, a service that could have been accessed through a different channel that does not require waiting in the queue area; and
provide, to a device, an indication that the service provided by the computer system is accessible through the device.
13 . A computer implemented method comprising:
receiving, by a computer system, respective images from a plurality of cameras;
synchronizing, by the computer system, the plurality of cameras based on respective timestamps associated with the respective images;
passing, by the computer system, respective outputs of a plurality of object detection models trained using a data set corresponding to different vantage points of the plurality of cameras, to respective intra-feed identity models to track only within a queue area detected objects by the plurality of object detection models across sequential images of a plurality of series of images received from the synchronized cameras to maintain an identity of the detected objects as the detected objects move through time and space in the queue area;
passing, by the computer system, outputs of the respective intra-feed identity models, to an inter-feed identity model that matches respective objects present in the plurality of series of images to identify new objects and identify previous objects in the plurality of series of images and to identify a common identifier across and within each series of the plurality of series of images; and
comparing, by the computer system, a last timestamp of a last detection of a first detected object of the detected objects when the first detected object has left the queue area, to a timestamp for the common identifier to determine, in real time and without requiring extensive computational resources, a wait time corresponding to how long the first detected object has been in the queue area.
14 . The method of claim 11 , wherein synchronizing the plurality of cameras is further based on frame rates of each of the plurality of cameras.
15 . The method of claim 13 , further comprising:
creating, by object localization of the computer system, proposed regions that contain distinct objects; and
determining, by object classification of the computer system, detected objects among the distinct objects in the proposed regions.
16 . The method of claim 13 , further comprising:
logging, by the computer system, a timestamp of the respective timestamps, for the detected objects with bounding boxes falling inside a bounding box of a queue area in association with a unique identifier of an object first detected in the queue area, and discarding, after a timeout period, those of the bounding boxes falling outside the bounding box of the queue area, and those of the detected objects remaining in the queue area for a prolonged period.
17 . The method of claim 13 , further comprising:
determining, by the computer system, respective outputs of the plurality of object detection models that identify bounding boxes for the detected objects.
18 . The method of claim 13 , further comprising:
delineating, by the respective intra-feed identity models executed by the computer system, bounding boxes that correspond to specific objects matching those objects detected in a previous frame and bounding boxes corresponding to a new object.
19 . The method of claim 18 , further comprising:
reclassifying, by the respective intra-feed identity models executed by the computer system, bounding boxes provided by the plurality of object detection models, by:
associating the bounding boxes with a unique identifier;
assigning to the detected objects in the sequential images, the unique identifier as the first detected object so that identity of the detected objects is persisted across the sequential images of a plurality of series of images;
assigning, upon first appearance of the new object, a new unique identifier to the new object and the matching objects in the subsequent images of the series of the respective images; and
correlating the bounding box of the first detected object with the bounding boxes corresponding to the objects matching the first detected object in earlier frames as the first detected object moves through the queue area and the subsequent images.
20 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor cause the at least one processor to:
receive respective images from a plurality of cameras;
synchronize the plurality of cameras based on respective timestamps associated with the respective images;
pass respective outputs of a plurality of object detection models trained using a data set corresponding to different vantage points of the plurality of cameras, to respective intra-feed identity models to track only within a queue area detected objects by the plurality of object detection models across sequential images of a plurality of series of mages received from the synchronized cameras to maintain an identity of the detected objects as the detected objects move through time and space in the queue area;
pass outputs of the respective intra-feed identity models, to an inter-feed identity model that matches respective objects present in the plurality of series of images to identify new objects and identify previous objects in the plurality of series of images and to identify a common identifier across and within each series of the plurality of series of images; and
compare a last timestamp of a last detection of a first detected object of the detected objects when the first detected object has left the queue area, to a logged timestamp for the common identifier to determine, in real time and without requiring extensive computational resources, a wait time corresponding to how long the first detected object has been in the queue area.