IP Library Granted Patent US 10,916,040
Granted Patent B2
US 10,916,040 · App. 16/044,070 · Granted Feb 9, 2021

Processing image data using different data reduction rates

Inventors: Daren Croxford (Swaffham Prior, GB); Roberto Lopez Mendez (Cambridge, GB); Sean Tristram LeGuay Ellis (Cambridge, GB); Jayavarapu Srinivasa Rao (Cambridge, GB)
Assignees: Apical Ltd.; Arm Limited
G06T9/001G02B27/017G06F3/013G06T1/20G06T15/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,916,040
App. No.
16/044,070
Granted
Feb 9, 2021
Kind
B2
Abstract

Examples of the present disclosure relate to methods for processing image data. In one such example, first data representing a rendered image is received. In some cases, second data useable to identify at least one target region of the rendered image is received, the at least one target region being associated with a gaze direction of a viewer. A first portion of the first data is processed in accordance with a first data reduction rate to derive first processed data, the first portion representing the at least one target region. A second portion of the first data is processed in accordance with a second data reduction rate, different from the first data reduction rate. The second portion represents a further region of the rendered image, different from the at least one target image. At least the first processed data is outputted.

Claims (56)

1. A method of processing image data, the method comprising:

receiving first data representing a rendered image;

receiving second data useable to identify at least one target region of the rendered image, the at least one target region being associated with a gaze direction of a viewer;

processing a first portion of the first data in accordance with a first data reduction rate to derive first processed data, the first portion representing the at least one target region;

processing a second portion of the first data in accordance with a second data reduction rate, different from the first data reduction rate, the second portion representing a further region of the rendered image, different from the at least one target region;

processing a third portion of the first data in accordance with a third data reduction rate, the third data reduction rate being intermediate the first data reduction rate and the second data reduction rate, the third portion representing a region of the image that is located between the at least one target region and the further region; and

outputting at least the first processed data.

2. A method according to claim 1 ,

wherein processing the first portion of the first data comprises compressing the first portion of the first data in accordance with a first compression ratio, and

wherein processing the second portion of the first data comprises compressing the second portion of the first data in accordance with a second, different, compression ratio.

3. A method according to claim 1 ,

wherein processing the first portion of the first data comprises performing lossless or visually lossless compression of the first portion of the first data; and

wherein processing the second portion of the first data comprises performing lossy compression of the second portion of the first data.

4. A method according to claim 1 ,

wherein processing the first portion of the first data comprises compressing the first portion of the first data in accordance with a first data rate target,

wherein processing the second portion of the first data comprises compressing the second portion of the first data in accordance with a second, different, data rate target, and

wherein the first data rate target and the second data rate target have been derived in dependence on an overall data rate target for compressing the first data.

5. A method according to claim 4 , wherein the first and second data rate targets comprise first and second bitrate targets, respectively.

6. A method according to claim 1 , the method comprising outputting the at least first processed data for transmission via a display serial interface to a display device.

7. A method according to claim 6 , the method comprising outputting the second data for transmission to the display device.

8. A method according to claim 6 , wherein the display device comprises a head-mounted display (HMD).

9. A method according to claim 1 , wherein processing a given portion of the first data comprises compressing the given portion of the first data using a Display Stream Compression (DSC) protocol.

10. A method according to claim 1 , wherein processing the second portion of the first data comprises deriving second processed data, the method comprising outputting the second processed data.

11. A method according to claim 1 , wherein, prior to receiving the first data representing the rendered image, the at least one target region of the rendered image is rendered at a first level of quality, and the further region of the rendered image is rendered at a second, different, level of quality.

12. A method according to claim 1 , wherein the second data comprises eye-tracking data generated by an eye-tracking sensor.

13. A method according to claim 1 ,

wherein the rendered image comprises a current frame of a video,

the method comprising obtaining third data representing a previous frame of the video,

wherein processing a given portion of the first data comprises:

deriving a measure of similarity between the given portion of the first data and a portion of the third data; and

in response to determining that the derived measure of similarity has a predetermined relationship with a similarity threshold associated with the given portion, outputting the given portion of the first data,

wherein the first data reduction rate comprises a first similarity threshold and the second data reduction rate comprises a second, different, similarity threshold.

14. A method according to claim 13 , wherein the first processed data is arranged to indicate whether or not the given portion of the first data is output.

15. A method according to claim 1 ,

wherein processing a given portion of the first data comprises discarding a predetermined number of least significant bits in the given portion of the first data, the predetermined number being associated with the given portion, and

wherein the first data reduction rate comprises a first number of least significant bits to be discarded and the second data reduction rate comprises a second, different, number of least significant bits to be discarded.

16. A method according to claim 1 , performed by a graphics processing unit (GPU), the method comprising outputting the at least first processed data to memory.

17. A method according to claim 1 , the method comprising:

obtaining tile quality metadata indicating a desired quality of compressed data corresponding to a given region of the rendered image; and

deriving the first and second data reduction rates based on the tile quality metadata.

18. Apparatus for processing image data, the apparatus comprising:

a receiver configured to:

receive first data representing a rendered image; and

receive second data useable to identify at least one target region of the rendered image, the at least one target region being associated with a gaze direction of a viewer; and

a processor configured to:

process a first portion of the first data in accordance with a first data reduction rate to derive first processed data, the first portion representing the at least one target region;

process a second portion of the first data in accordance with a second data reduction rate, different from the first data reduction rate, the second portion representing a further region of the rendered image, different from the at least one target region;

process a third portion of the first data in accordance with a third data reduction rate, the third data reduction rate being intermediate the first data reduction rate and the second data reduction rate, the third portion representing a region of the image that is located between the at least one target region and the further region; and

output at least the first processed data.

19. A non-transitory, computer-readable storage medium comprising a set of computer-readable instructions which, when executed by at least one processor, cause the at least one processor to:

receive first data representing a rendered image;

receive second data useable to identify at least one target region of the rendered image, the at least one target region being associated with a gaze direction of a viewer;

process a first portion of the first data in accordance with a first data reduction rate to derive first processed data, the first portion representing the at least one target region;

process a second portion of the first data in accordance with a second data reduction rate, different from the first data reduction rate, the second portion representing a further region of the rendered image, different from the at least one target region; and

process a third portion of the first data in accordance with a third data reduction rate, the third data reduction rate being intermediate the first data reduction rate and the second data reduction rate, the third portion representing a region of the image that is located between the at least one target region and the further region; and

output at least the first processed data.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2022
From: APICAL LIMITED
To: ARM LIMITED
Reel/Frame 061300/0471 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2018
From: CROXFORD, DAREN
To: APICAL LIMITED
Reel/Frame 047593/0252 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 16, 2018
From: MENDEZ, ROBERTO LOPEZ; RAO, JAYAVARAPU SRINIVASA; ELLIS, SEAN TRISTAM LEGUAY
To: ARM LIMITED
Reel/Frame 047593/0316 →
Continuity (1)
Related Publication 20200034993A1 · Jan 30, 2020