IP Library Granted Patent US 10,867,390
Granted Patent B2
US 10,867,390 · App. 16/127,007 · Granted Dec 15, 2020

Computer vision processing

Inventors: Yuhao Zhu (Rochester, NY); Paul Nicholas Whatmough (Cambridge, MA)
Assignee: Arm Limited
G06T7/20G06K9/3233G06N3/04G06T1/0014G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,867,390
App. No.
16/127,007
Granted
Dec 15, 2020
Kind
B2
Abstract

A data processing apparatus detects motion between frames in a sequence of frames. The data processing apparatus then selects and/or tracks a region of interest in the sequence of frames based on the detected motion. An artificial neural network is then implemented to process image data for the selected region of interest in an attempt to classify an object within the region of interest. The data processing apparatus can provide an efficient way of performing computer vision processing.

Claims (46)

1. A method of performing computer vision processing, the method comprising:

detecting motion between a first frame and a second frame in a sequence of frames, wherein detecting motion between the first frame and the second frame in the sequence of frames comprises detecting amounts of motion for the second frame of the sequence of frames;

determining a region of interest in the second frame of the sequence of frames based on the detected motion between the first frame and the second frame, wherein determining the region of interest in the second frame of the sequence of frames based on the detected motion comprises selecting the region of interest in the second frame of the sequence of frames based on the detected amounts of motion for the second frame;

indicating the region of interest for processing by an artificial neural network; and

implementing the artificial neural network to process the image data for the region of interest in an attempt to classify an object within the region of interest.

2. A method as claimed in claim 1 , wherein:

detecting motion between the first frame and the second frame in the sequence of frames comprises performing motion estimation in respect of the frames in the sequence of frames.

3. A method as claimed in claim 1 , wherein:

detecting motion between the first frame and the second frame in the sequence of frames comprises detecting directions of motion;

the method further comprising tracking the position of the region of interest between frames of the sequence of frames based on a detected direction of motion for the region of interest.

4. A method as claimed in claim 1 , wherein:

implementing the artificial neural network to process the image data for the region of interest in an attempt to classify an object within the region of interest comprises the artificial neural network processing some but not all of the image data for the second frame.

5. A method as claimed in claim 1 , further comprising:

the artificial neural network receiving some but not all of the image data for the second frame for processing.

6. A method as claimed in claim 1 , wherein:

implementing the artificial neural network to process the image data for the region of interest in an attempt to classify an object within the region of interest comprises implementing plural different artificial neural networks to process the image data for the region of interest in an attempt to classify an object within the region of interest.

7. A method as claimed in claim 1 , further comprising:

indicating the classification of the object to a user of a data processing apparatus or a user of another device that is in communication with the data processing apparatus.

8. A method as claimed in claim 1 , further comprising:

using the classification of the object to control the operation of a data processing apparatus or another device that is in communication with the data processing apparatus.

9. A data processing apparatus for performing computer vision processing, the apparatus comprising:

motion detection circuitry configured to detect motion between a first and a second frame in a sequence of frames, wherein the motion detection circuitry is configured to, when detecting motion between the first frame and the second frame in the sequence of frames, detect amounts of motion for the second frame of the sequence of frames;

motion processing circuitry configured to determine a region of interest in the second frame of the sequence of frames based on the detected motion between the first frame and the second frame, wherein the motion processing circuitry is configured to, when determining the region of interest in the second frame of the sequence of frames based on the detected motion, select the region of interest in the second frame of the sequence of frames based on the detected amounts of motion for the second frame; and

indicate the region of interest for processing by an artificial neural network; and

artificial neural network circuitry configured to implement the artificial neural network to process the image data for the region of interest in an attempt to classify an object within the region of interest.

10. An apparatus as claimed in claim 9 , wherein:

the motion detection circuitry is configured to, when detecting motion between the first frame and the second frame in the sequence of frames, perform motion estimation in respect of the frames in the sequence of frames.

11. An apparatus as claimed in claim 9 , wherein:

the motion detection circuitry is configured to, when detecting motion between the first and the second frame in the sequence of frames, detect directions of motion;

the apparatus further comprising tracking circuitry configured to:

track the position of the region of interest between frames of the sequence of frames based on a detected direction of motion for the region of interest.

12. An apparatus as claimed in claim 9 , wherein:

the artificial neural network circuitry is configured to, when implementing the artificial neural network to process the image data for the region of interest in an attempt to classify an object within the region of interest, process some but not all of the image data for the second frame.

13. An apparatus as claimed in claim 9 , wherein:

the artificial neural network circuitry is configured to receive some but not all of the image data for the second frame for processing.

14. An apparatus as claimed in claim 9 , wherein:

the artificial neural network circuitry is configured to, when implementing the artificial neural network to process the image data for the region of interest in an attempt to classify an object within the region of interest, implement plural different artificial neural networks to process the image data for the region of interest in an attempt to classify an object within the region of interest.

15. An apparatus as claimed in claim 9 , further comprising:

output circuitry configured to indicate the classification of the object to a user of the data processing apparatus or a user of another device that is in communication with the data processing apparatus.

16. An apparatus as claimed in claim 9 , further comprising:

control circuitry configured to use the classification of the object to control the operation of the data processing apparatus or another device that is in communication with the data processing apparatus.

17. A non-transitory computer readable storage medium storing computer software code which, when executing on one or more processors of a data processing apparatus, performs a method of performing computer vision processing, the method comprising:

detecting motion between a first frame and a second frame in a sequence of frames, wherein detecting motion between the first frame and the second frame in the sequence of frames comprises detecting amounts of motion for the second frame of the sequence of frames;

determining a region of interest in the second frame of the sequence of frames based on the detected motion between the first frame and the second frame, wherein determining the region of interest in the second frame of the sequence of frames based on the detected motion comprises selecting the region of interest in the second frame of the sequence of frames based on the detected amounts of motion for the second frame;

indicating the region of interest for processing by an artificial neural network; and

implementing the artificial neural network to process the image data for the region of interest in an attempt to classify an object within the region of interest.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2018
From: ZHU, YUHAO; WHATMOUGH, PAUL NICHOLAS
To: ARM LIMITED
Reel/Frame 047055/0353 →
Continuity (1)
Related Publication 20200082544A1 · Mar 12, 2020