IP Library Granted Patent US 11,558,539
Granted Patent B2
US 11,558,539 · App. 16/717,994 · Granted Jan 17, 2023

Systems and methods of detecting and identifying an object

Inventor: Anton Kramarov (Samara, RU)
Assignee: SMART SUPERVISION SYSTEM LLC
H04N5/23206G06Q10/087G06V20/20H04N5/23218H04N5/23299
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,558,539
App. No.
16/717,994
Granted
Jan 17, 2023
Kind
B2
Abstract

Systems and methods of detecting and identifying objects are provided. In one exemplary embodiment, a method performed by one of a plurality of network nodes, with each network node having an optical sensor and being operable to wirelessly communicate with at least one other network node, comprises sending, by a network node over a wireless communication channel, to another network node, an indication associated with an object that is detected and identified by the network node based on one more images of that object that are captured by the optical sensor of the network node. Further, the detection and identification of the object is contemporaneous with the capture of the one or more images of that object. Also, the network node is operable to control a spatial orientation of the sensor so that the sensor has a viewing angle towards the object.

Claims (64)

1. A method, comprising:

by a first network node of a plurality of network nodes, with each network node having an optical sensor intergrated with a processings circuit that is operable to perform object detection, controlling a spatial orientation of the optical sensor of the first network node to have a viewing angle towards an object positioned in a first area;

controlling a focal length of the optical sensor of the first network node so that the object is captured in a larger portion of an image;

capturing, through the optical sensor of the first network node, one or more images of the object; and

responsive to determining that a processing power of the processings circuit of the first network node is less than a processing power threshold associated with performing the object detection of the object on the one or more images of the object:

sending, by the first network node, to a second network node of the plurality of network nodes, at least one of the one or more images of the object, with the second network node being operable to perform the object detection on the at least one of the one or more images of the object contemporaneous with the first network node capturing the one or more images of the object;

receiving, by the first network node, from the second network node, a first indication associated with the detected object; and

determining that the detected object is removed from the first area based on the one or more images; and

sending, by the first network node, to a third network node of the plurality of network nodes, a second indication associated with the object being removed from the first area.

2. The method of claim 1 , further comprising:

identifying the object based on the one or more captured images responsive to the receiving the first indication associated with the detected object.

3. The method of claim 1 , wherein the controlling the spatial orientation of the optical sensor includes controlling a pan or tilt of the optical sensor.

4. The method of claim 1 , wherein the controlling the focal length of the optical sensor is responsive to the controlling the spatial orientation of the optical sensor.

5. The method of claim 1 , further comprising:

determining a distance and a direction of the object relative to the first network node based on the spatial orientation of the optical sensor and a height of the optical sensor relative to a predetermined reference point.

6. The method of claim 5 , further comprising:

sending, by the first network node, to the second network node, an indication associated with the distance and the direction of the object relative to the first network node.

7. The method of claim 5 , further comprising:

receiving, by the first network node, from a spatial orientation sensor of the first network node, the spatial orientation of the optical sensor.

8. The method of claim 7 , further comprising:

determining an angle of inclination of the optical sensor relative to the object based on the spatial orientation of the optical sensor; and

determining the distance of the object relative to the first network node based on the angle of inclination of the optical sensor.

9. The method of claim 5 , further comprising:

receiving, by the first network node, from a global positioning system (GPS) receiver of the first network node, a location of the first network node; and

determining a location of the object based on the location of the network node and the distance and the direction of the object relative to the network node; and

sending, by the first network node, to the second network node, an indication associated with the location of the object.

10. The method of claim 1 , further comprising:

determining that a parking space is full based on the detected and identified object, wherein the object is associated with the parking space; and

sending, by the first network node, to the second network node, an indication that the parking space is full.

11. The method of claim 1 , further comprising:

receiving, by the first network node, from the optical sensor, the one or more images of the object.

12. The method of claim 1 , wherein the plurality of network nodes is operable as a wireless mesh network.

13. The method of claim 1 , wherein the plurality of network nodes wirelessly communicate over a Wireless Fidelity (Wi-Fi) network.

14. The method of claim 1 , wherein the object is detected and identified within one second from the capture of the one or more images of the object.

15. The method of claim 1 , further comprising:

sending, by the first network node, to the second network node, the at least one of the one or more images of the object that are captured by the optical sensor of the first network node so that the second network node is enabled to time synchronize these images with one or more images of the object that are captured by the optical sensor of the second network node and to process the time synchronized images to identify the object, with the optical sensor of the first network node and the optical sensor of the second network node having different viewing angles towards the object.

16. The method of claim 1 , wherein the first network node is operable to distribute to the second network node at least a portion of processing to identify the object.

17. The method of claim 1 , wherein the optical sensor includes a camera, with the camera being positioned above and in front of the retail shelf so that the viewing angle of the camera is down and towards the front of the retail shelf.

18. The method of claim 1 , wherein the one or more captured images include one or more objects representing one or more retail items positioned in the first area.

19. A first network node of a plurality of network nodes, with each network node having an optical sensor and being operable to communicate with at least one other network node, comprising:

an optical sensor integrated with a processor that is operable to perform object detection, and

a memory, the memory containing instructions executable by the processor whereby the processor is configured to:

control a spatial orientation of the optical sensor of the first network node to have a viewing angle towards an object positioned in a first area;

control a focal length of the optical sensor so that the object is captured in a larger portion of an image;

capture, through the optical sensor, one or more images of the object; and

responsive to determining that a processing power of the first network node is less than a processing power threshold associated with performing object detection of the object on the one or more images of the object:

send, to a second network node of the plurality of network nodes, at least one of the one or more images of the object, with the second network node being operable to perform the object detection on the at least one of the one or more images of the object contemporaneous with the capturing of the one or more images of the object;

receive, from the second network node, a first indication associated with the detected object;

determine that detected object is removed from the first area based on the one or more images; and

send, to a third network node of the plurality of network nodes, a second indication associated with the object being removed from the first area, with each network node of the plurality of network nodes having an optical sensor integrated with a processor that is operable to perform object detection.

20. A method, comprising:

by a second network node of a plurality of network nodes, with each network node having an optical sensor intergrated with a processings circuit that is operable to perform object detection, receiving, by the second network node, from a first network node of the plurality of network nodes, one or more images of an object positioned in a first area that are captured by an optical sensor of the first network node, with the first network node being operable to send the one or more images of the object to the second network node responsive to determining that a processing power of the first network node is less than a processing power threshold associated with performing object detection of the object on the one or more images of the object;

performing object detection of the object on the one or more images of the object contemporaneous with the capturing of the one or more images of the object; and

sending, by the second network node, to the first network node, an indication associated with the detected object.

21. The method of claim 20 , further comprising:

identifying the detected object based on the one or more images; and

sending, by the second network node, to the first network node, an indication of the identified object.

22. The method of claim 20 , wherein the performing the object detection is responsive to determining that a processing power of the first network node is less than a processing power threshold associated with performing object detection of the object on the one or more images of the object.

23. A second network node of a plurality of network nodes, with each network node having an optical sensor and being operable to communicate with at least one other network node, comprising:

an optical sensor integrated with a processor that is operable to perform object detection, and

a memory, the memory containing instructions executable by the processor whereby the network node is configured to:

receive, from a first network node of the plurality of network nodes, one or more images of an object positioned in a first area that is captured by an optical sensor of the first network node, with the first network node being operable to send the one or more images of the object to the second network node responsive to determining that a processing power of the first network node is less than a processing power threshold associated with performing object detection of the object on the one or more images of the object;

perform object detection of the object on the one or more images of the object contemporaneous with the capturing of the one or more images of the object; and

send, to the first network node, an indication associated with the detected.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2023
From: SMART SUPERVISION SYSTEM LLC
To: ANTON KRAMAROV
Reel/Frame 063002/0432 →
LICENSE Recorded Oct 22, 2021
From: SMART SUPERVISION SYSTEMS, LLC; SMASS TECHNOLOGIES, INC.
To: TOSHIBA GLOBAL COMMERCE SOLUTIONS, INC.
Reel/Frame 057883/0835 →
LICENSE Recorded Oct 22, 2021
From: SMART SUPERVISION SYSTEMS, LLC; SMASS TECHNOLOGIES, INC.
To: TOSHIBA GLOBAL COMMERCE SOLUTIONS, INC.
Reel/Frame 058901/0446 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2021
From: KRAMAROV, ANTON
To: SMART SUPERVISION SYSTEM LLC
Reel/Frame 056654/0127 →
Continuity (2)
Provisional Application 62817714 · Mar 13, 2019
Related Publication 20200296278A1 · Sep 17, 2020
Cited By (1)
US 12,188,957