Granted Patent B2

US 12,579,874 · App. 17/991,166 · Granted Mar 17, 2026

Inventors: Rajiv Ramanasankaran (San Jose, CA); Santle Camilus Kulandai Samy (Sunnyvale, CA); Ambuj Shatdal (Madison, WI); Erik S. Paulson (Madison, WI); Lipphei Adam (Milwaukee, WI)

Assignee: TYCO FIRE & SECURITY GMBH

G08B13/19645G05B15/02G08B13/19604

Abstract

Claims (83)

1 . A building system of a building comprising one or more memory devices storing instructions thereon that, when executed by one or more processors, cause the one or more processors to:

receive a request to generate a video tracking movement of an entity throughout the building;

search, based on a graph database for the building, a video database for a set of images or videos of a plurality of cameras of the building that track the entity throughout the building, the graph database for the building comprising a plurality of nodes indicating a plurality of spaces of the building and the plurality of cameras of the building, the graph database for the building comprising a plurality of edges between the plurality of nodes representing semantic relationships between the plurality of spaces and the plurality of cameras;

identify a set of cameras of the plurality of cameras associated with the set of images or videos;

identify a set of nodes of the graph database for the building representing the set of cameras;

identify camera characteristics of the set of cameras by identifying edges of the plurality of edges linking the set of nodes to nodes representing the camera characteristics;

normalize the set of images or videos based on the camera characteristics of the set of cameras; and

join the set of images or videos together to create the video.

2 . The building system of claim 1 , wherein at least some of the plurality of nodes indicate equipment, people, or events of the building;

wherein at least some of the plurality of edges represent relationships between the equipment, the people, or the events of the building.

3 . The building system of claim 1 , wherein the instructions cause the one or more processors to:

generate a trajectory graph based on the graph database for the building and a plurality of images or videos of the video database, the trajectory graph indicating a path of the entity through the building, the trajectory graph including a plurality of nodes and a plurality of edges, wherein the plurality of nodes represent the plurality of cameras that captured pictures of the entity or filmed the entity, the plurality of edges indicating an order in which the plurality of cameras captured the pictures of the entity or filmed the entity; and

search the video database for the set of images or videos based on the trajectory graph.

4 . The building system of claim 1 , wherein the request to generate the video tracking movement of the entity throughout the building is at least one of:

a user request of a user device requesting the video tracking movement of the entity throughout the building be generated; or

a request generated responsive to a security event involving the entity occurring within the building.

5 . The building system of claim 1 , wherein the instructions cause the one or more processors to:

search the video database for the set of images or videos of the plurality of cameras of the building by:

identifying one or more first videos or images of a first camera of the plurality of cameras stored in the video database, the one or more first videos or images including the entity, the first camera surveilling a first space of the building;

identifying a second space of the building connected to the first space based on the graph database for the building;

identifying a second camera of the plurality of cameras surveilling the second space of the building based on the graph database for the building;

selecting one or more second videos or images of the second camera stored in the video database responsive to identifying the second camera; and

identifying the entity within the one or more second videos or images of the second camera indicating that the entity moved from the first space to the second space; and

join the one or more first videos or images and the one or more second videos or images to create the video.

6 . The building system of claim 5 , wherein the instructions cause the one or more processors to:

generate a trajectory graph by:

causing the trajectory graph to include a first node representing the first camera of the plurality of cameras in response to identifying the one or more first videos or images of the first camera of the plurality of cameras including the entity; and

causing the trajectory graph to include a second node and an edge between the first node and the second node indicating that the entity has moved from a field of view of the first camera to a field of view of the second camera in response identifying the entity is within the one or more second videos or images of the second camera.

7 . The building system of claim 6 , wherein the instructions cause the one or more processors to:

identify, based on the graph database for the building, a camera not represented in the trajectory graph that includes a field of view that captures a portion of a path the entity traveled on;

update the trajectory graph to include a node representing the camera; and

join the set of images or videos together with images or videos of the camera to create the video.

8 . The building system of claim 7 , wherein the instructions cause the one or more processors to:

determine at least one of a starting time or an ending time for the images or videos by:

identifying a speed of the entity based on a length of time the entity takes to move across at least a portion of the field of view of the first camera; and

determining at least one of the starting time or the ending time based on the speed of the entity.

9 . A method, comprising:

receiving, by one or more processing circuits, a request to generate a video tracking movement of an entity throughout a building;

searching, by the one or more processing circuits, based on a graph database for the building, a video database for a set of images or videos of a plurality of cameras of the building that track the entity throughout the building, the graph database for the building comprising a plurality of nodes indicating a plurality of spaces of the building and the plurality of cameras of the building, the graph database for the building comprising a plurality of edges between the plurality of nodes representing semantic relationships between the plurality of spaces and the plurality of cameras;

identifying, by the one or more processing circuits, a set of cameras of the plurality of cameras associated with the set of images or videos;

identifying, by the one or more processing circuits, a set of nodes of the graph database for the building representing the set of cameras;

identifying, by the one or more processing circuits, camera characteristics of the set of cameras by identifying edges of the plurality of edges linking the set of nodes to nodes representing the camera characteristics;

normalizing, by the one or more processing circuits, the set of images or videos based on the camera characteristics of the set of cameras; and

joining, by the one or more processing circuits, the set of images or videos together to create the video.

10 . The method of claim 9 , wherein at least some of the plurality of nodes indicate equipment, people, or events of the building;

wherein at least some of the plurality of edges represent relationships between the equipment, the people, or the events of the building.

11 . The method of claim 9 , comprising:

generating, by the one or more processing circuits, a trajectory graph based on the graph database for the building and a plurality of images or videos of the video database, the trajectory graph indicating a path of the entity through the building, the trajectory graph including a plurality of nodes and a plurality of edges, wherein the plurality of nodes represent the plurality of cameras that captured pictures of the entity or filmed the entity, the plurality of edges indicating an order in which the plurality of cameras captured the pictures of the entity or filmed the entity; and

searching, by the one or more processing circuits, the video database for the set of images or videos based on the trajectory graph.

12 . The method of claim 9 , wherein the request to generate the video tracking movement of the entity throughout the building is at least one of:

a user request from a user device requesting the video tracking movement of the entity throughout the building be generated; or

a request generated responsive to a security event involving the entity occurring within the building.

13 . The method of claim 9 , comprising:

searching, by the one or more processing circuits, the video database for the set of images or videos of the plurality of images or videos of the plurality of cameras of the building by:

identifying one or more first videos or images of a first camera of the plurality of cameras stored in the video database, the one or more first videos or images including the entity, the first camera surveilling a first space of the building;

identifying a second space of the building connected to the first space based on the graph database for the building;

identifying a second camera of the plurality of cameras surveilling the second space of the building based on the graph database for the building;

selecting one or more second videos or images of the second camera stored in the video database responsive to identifying the second camera; and

identifying the entity within the one or more second videos or images of the second camera indicating that the entity moved from the first space to the second space; and

joining, by the one or more processing circuits, the one or more first videos or images and the one or more second videos or images to create the video.

14 . The method of claim 13 , comprising:

generating, by the one or more processing circuits, a trajectory graph by:

causing the trajectory graph to include a first node representing the first camera of the plurality of cameras in response to identifying the one or more first videos or images of the first camera of the plurality of cameras including the entity; and

causing the trajectory graph to include a second node and an edge between the first node and the second node indicating that the entity has moved from a field of view of the first camera to a field of view of the second camera in response identifying the entity is within the one or more second videos or images of the second camera.

15 . A system comprising one or more memory devices storing instructions thereon that, when executed by one or more processors, cause the one or more processors to:

receive a request to generate a video tracking movement of an entity throughout an environment;

search, based on a digital twin of the environment comprising a graph database, a video database for a set of images or videos of a plurality of cameras of the environment that track the entity throughout the environment, the graph database of the digital twin representing areas of the environment and the plurality of cameras of the environment, the graph database of the digital twin representing semantic relationships between the areas and the plurality of cameras;

identify a set of cameras of the plurality of cameras associated with the set of images or videos;

identify a set of nodes of the graph database of the digital twin representing the set of cameras;

identify camera characteristics of the set of cameras by identifying one or more relationships of the graph database of the digital twin linking the set of nodes to nodes representing the camera characteristics;

normalizing the set of images or videos based on the camera characteristics of the set of cameras; and

join the set of images or videos together to create the video.

16 . The system of claim 15 , wherein the instructions cause the one or more processors to:

search the video database for the set of images or videos of the plurality of images or videos of the plurality of cameras of the environment by:

identifying one or more first videos or images of a first camera of the plurality of cameras stored in the video database, the one or more first videos or images including the entity, the first camera surveilling a first area of the environment;

identifying a second area of the environment connected to the first area based on the graph database of the digital twin;

identifying a second camera of the plurality of cameras surveilling the second area of the environment based on the graph database of the digital twin;

selecting one or more second videos or images of the second camera stored in the video database responsive to identifying the second camera;

identifying the entity within the one or more second videos or images of the second camera indicating that the entity moved from the first area to the second area; and

join the one or more first videos or images and the one or more second videos or images to create the video.

17 . The system of claim 15 , wherein the instructions cause the one or more processors to:

generate a trajectory graph based on the graph database of the digital twin and a plurality of images or videos of the video database, the trajectory graph indicating a path of the entity through the environment, the trajectory graph including a plurality of nodes and a plurality of edges, wherein the plurality of nodes represent the plurality of cameras that captured pictures of the entity or filmed the entity, the plurality of edges indicating an order in which the plurality of cameras captured the pictures of the entity or filmed the entity; and

search the video database for the set of images or videos based on the trajectory graph.

Assignments (2)

Continuity (2)

References Cited (400)