IP Library › Granted Patent US 10,509,969
Granted Patent B2
US 10,509,969 · App. 15/702,061 · Granted Dec 17, 2019

Dynamic person queue analytics

Inventors: Victor Tsekay To (Sunnyvale, CA); Feng Jiang (San Jose, CA); Nham Van Le (San Jose, CA); Hugo Latapie (Long Beach, CA); Enzo Fenoglio (Issy-les-Moulineaux, FR)
Assignee: Cisco Technology, Inc.
G06K9/00778G06K9/00751G06K9/00771G06K3/02G06K9/3233
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,509,969
App. No.
15/702,061
Granted
Dec 17, 2019
Kind
B2
Abstract

In one embodiment, a device identifies, from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location. The device forms a set of nodes whereby a given node represents one or more of the identified people located within the physical location. The device represents a person queue as an ordered list of nodes from the set of nodes and adds a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest. The device adds one or more nodes to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list. The device provides an indication of the person queue to an interface.

Claims (62)

1. A method comprising:

identifying, by a device and from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location;

forming, by the device, a set of nodes, wherein a given node represents one or more of the identified people located within the physical location;

representing, by the device, a person queue as an ordered list of nodes from the set of nodes;

adding, by the device, a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest;

adding, by the device, one or more nodes to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list, wherein the angle and the distance range are determined using a multimodal recurrent neural network;

regressing, by the device, the angle and distance range by using temporal information as input to the multimodal recurrent neural network to identify a queue pattern of the person queue comprising at least one of: a fan out pattern for the person queue, a fan in pattern for the person queue, a person queue that is parallel to the person queue, or a zig zag pattern of the person queue; and

providing, by the device, an indication of the person queue and the queue pattern to an interface.

2. The method as in claim 1 , wherein forming the set of nodes comprises:

performing, by the device, person re-identification across the image data from non-overlapping cameras to identify a subset of the people that traveled together at a point in time; and

grouping, by the device, the subset of the people as a single node based on the subset of the people traveling together at a point in time.

3. The method as in claim 1 , wherein identifying the focal point of interest and people located within the physical location from the captured image data comprises:

locating, by the device, the people within the image data based in part on wireless network signals.

4. The method as in claim 1 , further comprising:

identifying, by the device, a second focal point of interest within the physical location; and

determining, by the device, a second person queue that begins at the second focal point of interest.

5. The method as in claim 1 , further comprising:

determining, by the device, the forward direction associated with the at least one node in the list by:

identifying shoulders of the one or more people represented by the at least one node; and

forming a two dimensional (2D) representation of the one or more people based on the identified shoulders of the one or more people; and

tracking the 2D representation of the one or more people over time.

6. The method as in claim 1 , further comprising:

tracking, by the device, movement of the nodes over time; and

estimating, by the device, a wait time metric for the person queue based on the tracked movement of the nodes over time, wherein the indication of the person queue provided to the interface comprises the estimated wait time.

7. The method as in claim 1 , wherein at least one of the nodes added to the list represents a non-human object.

8. An apparatus, comprising:

one or more network interfaces to communicate with a network;

a processor coupled to the network interfaces and configured to execute one or more processes; and

a memory configured to store a process executable by the processor, the process when executed configured to:

identify, from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location;

form a set of nodes, wherein a given node represents one or more of the identified people located within the physical location;

represent a person queue as an ordered list of nodes from the set of nodes;

add a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest;

add one or more nodes from the set to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list, wherein the angle and the distance range are determined using a multimodal recurrent neural network;

regress the angle and distance range by using temporal information as input to the multimodal recurrent neural network to identify a queue pattern of the person queue comprising at least one of: a fan out pattern for the person queue, a fan in pattern for the person queue, a person queue that is parallel to the person queue, or a zig zag pattern of the person queue; and

provide an indication of the person queue and the queue pattern to an interface.

9. The apparatus as in claim 8 , wherein the apparatus forms the set of nodes by:

performing person re-identification across the image data from non-overlapping cameras to identify a subset of the people that traveled together at a point in time; and

grouping the subset of the people as a single node based on the subset of the people traveling together at a point in time.

10. The apparatus as in claim 8 , wherein the apparatus identifies the focal point of interest and people located within the physical location from the captured image data by:

locating, by the device, the people within the image data based in part on wireless network signals.

11. The apparatus as in claim 8 , wherein the process when executed is further configured to:

identify a second focal point of interest within the physical location; and

determine a second person queue that begins at the second focal point of interest.

12. The apparatus as in claim 8 , wherein the process when executed is further configured to:

determine the forward direction associated with the at least one node in the list by:

identifying shoulders of the one or more people represented by the at least one node; and

forming a two dimensional (2D) representation of the one or more people based on the identified shoulders of the one or more people; and

tracking the 2D representation of the one or more people over time.

13. The apparatus as in claim 8 , wherein the process when executed is further configured to:

tracking, by the device, movement of the nodes over time; and

estimating, by the device, a wait time metric for the person queue based on the tracked movement of the nodes over time, wherein the indication of the person queue provided to the interface comprises the estimated wait time.

14. The apparatus as in claim 8 , wherein at least one of the nodes added to the list represents a non-human object.

15. The apparatus as in claim 8 , wherein the apparatus comprises a fog computing node in communication with the one or more cameras via the network.

16. A tangible, non-transitory, computer-readable media having software encoded thereon, the software when executed by a device cause the device to perform a process comprising:

identifying, by the device and from image data captured by one or more cameras of a physical location, a focal point of interest and people located within the physical location;

forming, by the device, a set of nodes, wherein a given node represents one or more of the identified people located within the physical location;

representing, by the device, a person queue as an ordered list of nodes from the set of nodes;

adding, by the device, a particular one of the set of nodes to the list based on the particular node being within a predefined distance to the focal point of interest;

adding, by the device, one or more nodes to the list based on the added node being within an angle and distance range trailing a forward direction associated with at least one node in the list wherein the angle and the distance range are determined using a multimodal recurrent neural network;

regressing, by the device, the angle and distance range by using temporal information as input to the multimodal recurrent neural network to identify a queue pattern of the person queue comprising at least one of: a fan out pattern for the person queue, a fan in pattern for the person queue, a person queue that is parallel to the person queue, or a zig zag pattern of the person queue; and

providing, by the device, an indication of the person queue and the queue pattern to an interface.

Continuity (1)
Related Publication 20190080178A1 · Mar 14, 2019