IP Library Granted Patent US 11,557,185
Granted Patent B2
US 11,557,185 · App. 15/807,477 · Granted Jan 17, 2023

Image processing method

Inventors: Ilya Romanenko (Cambridge, GB); Michael Tusch (Cambridge, GB)
Assignee: ARM Limited
G08B13/19602G06V10/25G06V20/41G06V20/52G06V40/10G06V40/161G08B13/19656G08B13/19667H04N7/18H04N7/183H04N19/115H04N19/117H04N19/124H04N19/137H04N19/142H04N19/167H04N19/172H04N19/503H04N19/87G06V20/44G06V40/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,557,185
App. No.
15/807,477
Filed
Nov 8, 2017
Granted
Jan 17, 2023
Kind
B2
Art Unit
2661
USPC
382/103
Abstract

An image processing method is provided. The method includes acquiring a video. The method includes using an object detection engine to detect a person in the video. The object detection engine is integrated with an image signal processing pipeline. The method includes transmitting the video over a network. The method includes determining that the detected person has moved less than a pre-set distance. The method includes, responsive to the determining, pausing transmission of the video. An embedded image processor including an object detection engine is also provided.

Claims (49)

1. An image processing method, comprising:

acquiring a video;

using an object detection engine to detect a person in the video, the object detection engine being integrated with an image signal processing pipeline;

using an encoder to compress the acquired video;

modifying the behaviour of the encoder based on an output of the object detection engine;

transmitting the compressed video over a network;

determining, by analysing the acquired video prior to or during transmission of the compressed video over the network, that the detected person has moved less than a pre-set distance; and

responsive to the determining, pausing the transmission of the compressed video over the network.

2. The method of claim 1 , wherein the object detection engine is integrated with the image signal processing pipeline in an embedded image processor.

3. The method of claim 1 , wherein the object detection engine receives raw digital video from an image sensor and analyses the raw digital video on a frame by frame basis.

4. The method of claim 1 , wherein the object detection engine outputs data for each video frame describing:

a pose of one or more detected persons;

a location or coordinates of the one or more detected persons; and

a location or coordinates of at least one body part of the one or more detected persons.

5. The method of claim 1 , comprising:

extracting a thumbnail image from the video, the thumbnail image comprising a face of the detected person; and

sending a text alert to an end user, the text alert comprising the thumbnail, wherein the end user can choose to access a live video feed.

6. The method of claim 5 , comprising sending the alert responsive to determining, based on pre-defined rules, that the detected person is not an allowed person.

7. The method of claim 1 , wherein modifying the behaviour of the encoder comprises modifying one or more of:

compression ratio;

quantization;

output resolution; and

frame rate.

8. An embedded image processor comprising an object detection engine, the embedded image processor configured to:

acquire a video;

use the object detection engine to detect a person in the video, the object detection engine being integrated with an image signal processing pipeline;

use an encoder to compress the acquired video;

modify the behaviour of the encoder based on an output of the object detection engine;

transmit the compressed video over a network;

determine, by analysing the acquired video prior to or during transmission of the compressed video over the network, that the detected person has moved less than a pre-set distance; and

responsive to the determining, pause the transmission of the compressed video over the network.

9. The embedded image processor of claim 8 , wherein the object detection engine is integrated with the image signal processing pipeline in the embedded image processor.

10. The embedded image processor of claim 8 , wherein the object detection engine receives raw digital video from an image sensor and analyses the raw digital video on a frame by frame basis.

11. The embedded image processor of claim 8 , wherein the object detection engine outputs data for each video frame describing:

a pose of one or more detected persons;

a location or coordinates of the one or more detected persons; and

a location or coordinates of at least one body part of the one or more detected persons.

12. The embedded image processor of claim 8 , further configured to:

extract a thumbnail image from the video, the thumbnail image comprising a face of the detected person; and

send a text alert to an end user, the text alert comprising the thumbnail, wherein the end user can choose to access a live video feed.

13. The embedded image processor of claim 12 , further configured to send the alert responsive to determining, based on pre-defined rules, that the detected person is not an allowed person.

14. The embedded image processor of claim 8 , wherein the object detection engine and the image signal processing pipeline:

are integrated with a graphics processing unit or a system on a chip; or

form part of the graphics processing unit or the system on the chip.

15. The embedded image processor of claim 8 , wherein modifying the behaviour of the encoder comprises modifying one or more of:

compression ratio;

quantization;

output resolution; and

frame rate.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2022
From: APICAL LIMITED
To: ARM LIMITED
Reel/Frame 060620/0954 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2018
From: ROMANENKO, ILYA; TUSCH, MICHAEL
To: APICAL LIMITED
Reel/Frame 046942/0693 →
Priority Claims (1)
GB 1508074 · May 12, 2015 · national
Continuity (2)
Continuation PCTGB2016051362 · May 12, 2016
Related Publication 20180068540A1 · Mar 8, 2018