IP Library › Granted Patent US 11,062,419
Granted Patent B1
US 11,062,419 · App. 16/929,511 · Granted Jul 13, 2021

Image processing system

Inventor: Robin Paul Fell (Cambridge, GB)
Assignee: Arm Limited
G06T1/0021H04N1/32267H04N1/32352
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,062,419
App. No.
16/929,511
Granted
Jul 13, 2021
Kind
B1
Abstract

An image processing system comprises one or more image sensors configured to take images, one or more image processors configured to process images taken by the one or more image sensors to produce output images, and one or more circuits configured to use output images. The one or more image processors comprise a set of one or more image processing stages, and each stage is configured to process images using a respective image processing operation. Processing circuitry is configured to verify image processing performed by one or more image processing stages, by inserting one or more probes into one or more images prior to the one or more images being processed by the one or more image processing stages, and attempting to identify the one or more probes in an output of the one or more processing stages.

Claims (57)

1. An image processing system, comprising:

one or more image sensors configured to take images;

one or more image processors configured to process images taken by the one or more image sensors so as to produce output images, wherein the one or more image processors comprise a set of one or more image processing stages, each stage being configured to process images using a respective image processing operation;

one or more circuits configured to use output images produced by the one or more image processors; and

processing circuitry configured to verify image processing performed by one or more image processing stages of the set of one or more image processing stages, by:

inserting one or more probes into one or more images prior to the one or more images being processed by the one or more image processing stages of the set of one or more image processing stages; and

attempting to identify one or more probes in an output of the one or more processing stages.

2. The system of claim 1 , wherein the processing circuitry is configured to insert the one or more probes into the one or more images such that the one or more probes are processed by the one or more image processing stages of the set of one or more image processing stages together with the one or more images.

3. The system of claim 1 , wherein the processing circuitry is configured to insert the one or more probes into the one or more images by modifying the one or more images.

4. The system of claim 1 , wherein:

the one or more images comprises a sequence of images; and

the processing circuitry is configured to insert the one or more probes into the one or more images by inserting one or more test images into the sequence of images.

5. The system of claim 1 , wherein the image processing system is configured to remove or discard one or more probes prior to using the output images.

6. The system of claim 1 , wherein the one or more probes comprises a set of plural probes, and wherein the processing circuitry is configured to insert the set of plural probes into an image such that each probe is evenly distributed throughout the image.

7. The system of claim 1 , wherein the image processing system is configured to change the configuration of the one or more probes with time.

8. The system of claim 1 , wherein the processing circuitry is configured to attempt to identify one or more expected probes in the output, where each expected probe is a probe that, based on the image processing operations of the one or more image processing stages, is expected to appear in the output.

9. The system of claim 1 , wherein the image processing system is configured to determine that the one or more image processing stages have not accurately performed image processing when:

one or more expected probes cannot be identified in the output;

one or more expected probes is identified in an unexpected temporal and/or spatial position within the output; and/or

one or more unexpected probes are identified in the output.

10. The system of claim 1 , wherein the image processing system is configured to determine that the one or more image processing stages have accurately performed image processing when:

one or more expected probes can be identified in the output;

one or more expected probes can be identified at an expected temporal and/or spatial position in the output; and/or

one or more unexpected probes cannot be identified in the output.

11. The system of claim 1 , wherein the set of one or more image processing stages comprises one or more of: a scaler, a cropping stage, a flip stage, one or more image enhancement stages, a compositor, a filter, and a rotation stage.

12. An image processor comprising:

image processing circuitry configured to process images so as to produce output images, wherein the image processing circuitry comprise a set of one or more image processing stages, each stage being configured to process images using a respective image processing operation; and

processing circuitry configured to verify image processing performed by one or more image processing stages of the set of one or more image processing stages, by:

inserting one or more probes into one or more images prior to the one or more images being processed by the one or more processing stages of the set of one or more image processing stages; and

attempting to identify one or more probes in an output of the one or more processing stages.

13. A method of operating an image processor, the method comprising:

processing images using a set of one or more image processing stages so as to produce output images; and

verifying image processing performed by one or more image processing stages of the set of one or more image processing stages, by:

inserting one or more probes into one or more images prior to the one or more images being processed by the one or more processing stages of the set of one or more image processing stages; and

attempting to identify one or more probes in an output of the one or more processing stages.

14. The method of claim 13 , further comprising using and/or displaying the one or more output images.

15. The method of claim 13 , comprising:

inserting the one or more probes into the one or more images; and

the one or more image processing stages of the set of one or more image processing stages: processing the one or more probes together with the one or more images.

16. The method of claim 13 , comprising inserting the one or more probes into the one or more images by modifying the one or more images.

17. The method of claim 13 , wherein the one or more images comprises a sequence of images, and the method comprises:

inserting the one or more probes into the one or more images by inserting one or more test images into the sequence of images.

18. The method of claim 13 , further comprising changing the configuration of the one or more probes with time.

19. The method of claim 13 , comprising attempting to identify one or more expected probes in the output, where each expected probe is a probe that, based on the image processing operations of the one or more image processing stages, is expected to appear in the output.

20. The method of claim 13 , further comprising determining that the one or more image processing stages have not accurately performed image processing when:

one or more expected probes cannot be identified in the output;

one or more expected probes is identified in an unexpected temporal and/or spatial position within the output; and/or

one or more unexpected probes are identified in the output.

21. The method of claim 13 , further comprising determining that the one or more image processing stages have accurately performed image processing when:

one or more expected probes can be identified in the output;

one or more expected probes can be identified at an expected temporal and/or spatial position in the output; and/or

one or more unexpected probes cannot be identified in the output.

22. A non-transitory computer readable storage medium storing computer software code which when executing on a processor performs a method of operating an image processor, the method comprising:

processing images using a set of one or more image processing stages so as to produce output images; and

verifying image processing performed by one or more image processing stages of the set of one or more image processing stages, by:

inserting one or more probes into one or more images prior to the one or more images being processed by the one or more processing stages of the set of one or more image processing stages; and

attempting to identify one or more probes in an output of the one or more processing stages.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2020
From: FELL, ROBIN PAUL
To: ARM LIMITED
Reel/Frame 053217/0929 →
Cited By (2)
US 12,238,265 US 12,316,965