Granted Patent B1

US 12,657,924 · App. 19/376,410 · Granted Jun 16, 2026

Inventors: Amal Ayadh Alsulamy (Riyadh, SA); Munirah Nasser AlYahya (Riyadh, SA)

Assignee: Saudi Technology and Security Comprehensive Control Co. Ltd.

G06V20/52G06T7/292G06T7/70G06V10/25G06V10/764G06V40/10G06T2207/10016G06T2207/30196G06T2207/30232

Abstract

Claims (139)

1 . One or more systems for tracking data objects, determining statuses of the data objects, and responsively controlling alert communications, the one or more systems comprising:

one or more computing databases;

one or more computing servers comprising one or more computing device processors and a memory storing instructions, the instructions being executable by the one or more computing device processors to:

receive a substantially real-time video stream captured by a first camera in a first area or a second camera in the first area or a second area;

execute one or more multi-camera multi-object tracking (MCMOT) computing operations on the substantially real-time video stream;

identify and track, based on the one or more MCMOT computing operations, a first data object in the substantially real-time video stream;

identify and track, based on the one or more MCMOT computing operations, a second data object in the substantially real-time video stream;

identify and track, based on the one or more MCMOT computing operations, a third data object in the substantially real-time video stream;

identify and track, based on the one or more MCMOT computing operations, a fourth data object in the substantially real-time video stream;

determine, based on the one or more MCMOT computing operations, that the first data object is of a first data object type;

determine, based on the one or more MCMOT computing operations, that the second data object is of a second data object type;

determine, based on the one or more MCMOT computing operations, that the third data object is of the first data object type;

determine, based on the one or more MCMOT computing operations, that the fourth data object is of the second data object type;

execute one or more human-object interaction recognition (HOIR) computing operations on at least one of: the first data object, the second data object, the third data object, and the fourth data object, or

the substantially real-time video stream;

determine, based on the one or more HOIR computing operations, a first interaction associated with the first data object and the second data object;

initiate storage of, first interaction data associated with the first interaction, in the one or more computing databases;

determine, based on the one or more HOIR computing operations, a second interaction associated with the third data object and the fourth data object;

initiate storage of, second interaction data associated with the second interaction, in the one or more computing databases;

execute one or more first relationship computing operations on the first interaction data;

execute one or more second relationship computing operations on the second interaction data;

determine, based on the first interaction data, a first duration of the first interaction;

determine, based on the first interaction data, the first interaction is of a first interaction type;

determine, based on the second interaction data, a second duration of the second interaction;

determine, based on the second interaction data, the second interaction is of a second interaction type;

determine, based on the one or more first relationship computing operations, the first duration, and the first interaction type, that a first relationship between the first data object and the second data object is of a first relationship type;

determine, based on the one or more second relationship computing operations, the second duration, and the second interaction type, that a second relationship between the third data object and the fourth data object is of a second relationship type;

update, based on the first relationship type, a first status of the second data object, wherein the first status does not cause first initiation of an alert communication for transmission to one or more first computing devices;

update, based on the second relationship type, a second status of the fourth data object, wherein the second status causes the first initiation of the alert communication for transmission to the one or more first computing devices or one or more second computing devices; and

access recovery information associated with the fourth data object, the recovery information comprising at least one of:

location information associated with a sixth data object linked to the fourth data object,

appearance information associated with the sixth data object linked to the fourth data object, or

facial information associated with the sixth data object linked to the fourth data object.

2 . The one or more systems of claim 1 , wherein:

one or more of:

the first data object is associated with a first person,

the second data object is associated with a second person,

the third data object is associated with a third person, or

the fourth data object is associated with a fourth person, and

the substantially real-time video stream comprises:

a fifth data object that is associated with a pet, and

a sixth data object that is associated with an article.

3 . The one or more systems of claim 1 , wherein the substantially real-time video stream captured by the first camera in the first area, or the second camera in the first area or the second area, is captured by one or more of:

one or more first cameras configured to provide one or more first views of the first data object, the second data object, the third data object, and the fourth data object, and

one or more second cameras configured to provide one or more second views of the first data object, the second data object, the third data object, and the fourth data object, the one or more second cameras being positioned to provide one or more top-down views of the first data object, the second data object, the third data object, and the fourth data object.

4 . The one or more systems of claim 1 , wherein:

the first area or the second area is divided into a first number of regions comprising at least a first region and a second region,

at least a first video stream of at least a first portion of the first region is captured by the first camera,

at least a second video stream of a second portion of the second region is captured by the second camera, and

the instructions are executable by the one or more computing device processors to:

enable tracking a fifth data object moving from the first region to the second region,

synchronize the first video stream and the second video stream to a coordinate system for enabling the tracking, or

calibrate the first video stream and the second video stream to the coordinate system for enabling the tracking.

5 . The one or more systems of claim 1 , wherein the instructions are executable by the one or more computing device processors to:

assign, based on the one or more MCMOT computing operations, the first data object type to the first data object,

assign, based on the one or more MCMOT computing operations a first bounding box to the first data object,

assign, based on the one or more MCMOT computing operations a first set of location coordinates to the first data object, and

assign, based on the one or more MCMOT computing operations, a first identifier to the first data object.

6 . The one or more systems of claim 5 , wherein the assign, based on the one or more MCMOT computing operations, the first identifier to the first data object further comprises assign, based on the first bounding box, and based on the first data object type, the first identifier to the first data object.

7 . The one or more systems of claim 1 , wherein the instructions are executable by the one or more computing device processors to:

assign an uninitialized status to the fourth data object, and

update the second status of the fourth data object based on first duration information associated with the second relationship and first distance information associated with the second relationship.

8 . The one or more systems of claim 1 , wherein the instructions are executable by the one or more computing device processors to:

track, based on the second status, the fourth data object,

determine a third relationship between the fourth data object and a fifth data object, and

update the second status of the fourth data object based on whether an identifier of the fifth data object is comprised in a data record associated with the fourth data object, the data record being stored in the one or more computing databases.

9 . The one or more systems of claim 1 , wherein the instructions are executable by the one or more computing device processors to include, in the alert communication:

location information associated with the fourth data object,

appearance information associated with the fourth data object, and

the second data object type associated with the fourth data object.

10 . The one or more systems of claim 1 , wherein the sixth data object linked to the fourth data object has an identifier that is comprised in a data record associated with the fourth data object.

11 . A method for tracking data objects, determining statuses of the data objects, and responsively controlling alert communications, the method comprising:

receiving, using one or more computing device processors, a substantially real-time video stream captured by a first camera in a first area or a second camera in the first area or a second area;

executing, using the one or more computing device processors, one or more multi-camera multi-object tracking (MCMOT) computing operations on the substantially real-time video stream;

identifying and tracking, using the one or more computing device processors, based on the one or more MCMOT computing operations, a first data object in the substantially real-time video stream;

identifying and tracking, using the one or more computing device processors, based on the one or more MCMOT computing operations, a second data object in the substantially real-time video stream;

identifying and tracking, using the one or more computing device processors, based on the one or more MCMOT computing operations, a third data object in the substantially real-time video stream;

identifying and tracking, using the one or more computing device processors, based on the one or more MCMOT computing operations, a fourth data object in the substantially real-time video stream;

determining, using the one or more computing device processors, based on the one or more MCMOT computing operations, that the first data object is of a first data object type;

determining, using the one or more computing device processors, based on the one or more MCMOT computing operations, that the second data object is of a second data object type;

determining, using the one or more computing device processors, based on the one or more MCMOT computing operations, that the third data object is of the first data object type;

determining, using the one or more computing device processors, based on the one or more MCMOT computing operations, that the fourth data object is of the second data object type;

executing, using the one or more computing device processors, one or more human-object interaction recognition (HOIR) computing operations on at least one of the first data object, the second data object, the third data object, and the fourth data object, or

the substantially real-time video stream;

determining, using the one or more computing device processors, based on the one or more HOIR computing operations, a first interaction associated with the first data object and the second data object;

initiating storage of, using the one or more computing device processors, first interaction data associated with the first interaction, in one or more computing databases;

determining, using the one or more computing device processors, based on the one or more HOIR computing operations, a second interaction associated with the third data object and the fourth data object;

initiating storage of, using the one or more computing device processors, second interaction data associated with the second interaction, in the one or more computing databases;

executing, using the one or more computing device processors, one or more first relationship computing operations on the first interaction data;

executing, using the one or more computing device processors, one or more second relationship computing operations on the second interaction data;

determining, using the one or more computing device processors, based on the first interaction data, a first duration of the first interaction;

determining, using the one or more computing device processors, based on the first interaction data, the first interaction is of a first interaction type;

determining, using the one or more computing device processors, based on the second interaction data, a second duration of the second interaction;

determining, using the one or more computing device processors, based on the second interaction data, the second interaction is of a second interaction type;

determining, using the one or more computing device processors, based on the one or more first relationship computing operations, the first duration, and the first interaction type, that a first relationship between the first data object and the second data object is of a first relationship type;

determining, using the one or more computing device processors, based on the one or more second relationship computing operations, the second duration, and the second interaction type, that a second relationship between the third data object and the fourth data object is of a second relationship type;

updating, using the one or more computing device processors, based on the first relationship type, a first status of the second data object, wherein the first status does not cause first initiation of an alert communication for transmission to one or more first computing devices;

updating, using the one or more computing device processors, based on the second relationship type, a second status of the fourth data object, wherein the second status causes the first initiation of the alert communication for transmission to the one or more first computing devices or one or more second computing devices; and

accessing, using the one or more computing device processors, recovery information associated with the fourth data object, the recovery information comprising at least one of:

location information associated with a sixth data object linked to the fourth data object,

appearance information associated with the sixth data object linked to the fourth data object, or

facial information associated with the sixth data object linked to the fourth data object.

12 . The method of claim 11 , wherein

one or more of:

the first data object is associated with a first person,

the second data object is associated with a second person,

the third data object is associated with a third person, or

the fourth data object is associated with a fourth person, and

the substantially real-time video stream comprises:

a fifth data object that is associated with a pet, and

a sixth data object that is associated with an article.

13 . The method of claim 11 , wherein the substantially real-time video stream captured by the first camera in the first area, or the second camera in the first area or the second area, is captured by one or more of:

one or more first cameras configured to provide one or more first views of the first data object, the second data object, the third data object, and the fourth data object, and

one or more second cameras configured to provide one or more second views of the first data object, the second data object, the third data object, and the fourth data object, the one or more second cameras being positioned to provide one or more top-down views of the first data object, the second data object, the third data object, and the fourth data object.

14 . The method of claim 11 , wherein:

the first area or the second area is divided into a first number of regions comprising at least a first region and a second region,

at least a first video stream of at least a first portion of the first region is captured by the first camera,

at least a second video stream of a second portion of the second region is captured by the second camera, and

the method further comprises:

enabling tracking a fifth data object moving from the first region to the second region, and

synchronizing the first video stream and the second video stream to a coordinate system for enabling the tracking, or

calibrating the first video stream and the second video stream to the coordinate system for enabling the tracking.

15 . The method of claim 11 , further comprising:

assigning, based on the one or more MCMOT computing operations, the first data object type to the first data object,

assigning, based on the one or more MCMOT computing operations a first bounding box to the first data object,

assigning, based on the one or more MCMOT computing operations a first set of location coordinates to the first data object, and

assigning, based on the one or more MCMOT computing operations, a first identifier to the first data object.

16 . The method of claim 15 , wherein assigning, based on the one or more MCMOT computing operations, the first identifier to the first data object further comprises assigning, based on the first bounding box, and based on the first data object type, the first identifier to the first data object.

17 . The method of claim 11 , further comprising:

assigning an uninitialized status to the fourth data object, and

updating the second status of the fourth data object based on first duration information associated with the second relationship and first distance information associated with the second relationship.

18 . The method of claim 11 , further comprising:

tracking, based on the second status, the fourth data object;

determining a third relationship between the fourth data object and a fifth data object; and

updating the second status of the fourth data object based on whether an identifier of the fifth data object is comprised in a data record associated with the fourth data object, the data record being stored in the one or more computing databases.

19 . The method of claim 11 , further comprising, including, in the alert communication:

location information associated with the fourth data object,

appearance information associated with the fourth data object, and

the second data object type associated with the fourth data object.

Assignments (1)

References Cited (9)