IP Library Granted Patent US 10,319,114
Granted Patent B2
US 10,319,114 · App. 15/649,047 · Granted Jun 11, 2019

Foveated compression of display streams

Inventors: Behnam Bastani (San Jose, CA); Carlin Vieri (Menlo Park, CA); Haomiao Jiang (Mountain View, CA)
Assignee: GOOGLE LLC
G06T9/00G09G5/397H04N19/172H04N19/174H04N19/182H04N19/423H04N19/44G09G2310/0297G09G2340/10G09G2350/00G09G2354/00G09G2360/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,319,114
App. No.
15/649,047
Granted
Jun 11, 2019
Kind
B2
Abstract

A processing unit is configured to render first pixels representative of a high-acuity region in the image and second pixels representative of a low-acuity region in the image. A shaper is configured to reorganize the first pixels based on at least one dimension of the low-acuity region. A multiplexer is configured to multiplex the reorganized first pixels and the second pixels to form a display stream. An encoder is configured to compress the display stream for transmission to a display. A decoder configured to decompress the display stream. A demultiplexer is configured to demultiplex the first pixels and the second pixels. Another processing unit is configured to blend the first pixels and the second pixels to form blended pixel values representative of the image for presentation on a screen.

Claims (57)

1. A method comprising:

accessing first pixels from a first buffer of an electronic device, wherein values of the first pixels represent a high-acuity region in an image and wherein the first pixels are organized as a set of first pixel lines in the first buffer;

accessing second pixels from a second buffer of the electronic device, wherein values of the second pixels represent a low-acuity region in the image and wherein the second pixels are organized as a set of second pixel lines in the second buffer;

reorganizing the first pixels based on at least one dimension of the low-acuity region, wherein reorganizing the first pixels comprises concatenating subsets of the set of first pixel lines to form modified first pixel lines including a number of first pixels that is equal to a number of second pixels in the second pixel lines;

multiplexing the reorganized first pixels and the second pixels to form a display stream; and

compressing the display stream for transmission to a display.

2. The method of claim 1 , wherein multiplexing the reorganized first pixels and the second pixels comprises combining the modified first pixel lines and the second pixel lines on a line-by-line basis.

3. The method of claim 2 , wherein combining the modified first pixel lines and the second pixel lines on the line-by-line basis comprises concatenating the modified first pixel lines and the second pixel lines.

4. The method of claim 3 , wherein concatenating the modified first pixel lines and the second pixel lines comprises appending the modified first pixel lines above, below, to the left, or to the right of the second pixel lines.

5. The method of claim 2 , wherein combining the modified first pixel lines and the second pixel lines on the line-by-line basis comprises interlacing the modified first pixel lines and the second pixel lines on a line-by-line basis.

6. The method of claim 5 , wherein interlacing the modified first pixel lines and the second pixel lines comprises partitioning the modified first pixel lines and the second pixel lines into slices having a height that corresponds to a slice height used to compress the display stream.

7. The method of claim 6 , wherein interlacing the modified first pixel lines and the second pixel lines comprises interlacing a number s 2 of slices of the modified first pixel lines with each slice of the second pixel lines, wherein s represents an upscaling factor that is applied to the low-acuity region.

8. The method of claim 5 , wherein interlacing the modified first pixel lines and the second pixel lines comprises statically interlacing the modified first pixel lines at a fixed position relative to the second pixel lines.

9. The method of claim 5 , wherein interlacing the modified first pixel lines and the second pixel lines comprises dynamically interlacing the modified first pixel lines at a variable position relative to the second pixel lines, wherein the variable position is determined based on relative positions of the high-acuity region and the low-acuity region in the image.

10. The method of claim 1 , further comprising:

decompressing the display stream;

demultiplexing the display stream to generate the reorganized first pixels and the second pixels; and

reconstructing an original configuration of the first pixels from the reorganized first pixels.

11. A method comprising:

accessing first pixels from a first buffer of an electronic device, wherein values of the first pixels represent a high-acuity region in an image;

accessing second pixels from a second buffer of the electronic device, wherein values of the second pixels represent a low-acuity region in the image;

reorganizing the first pixels based on at least one dimension of the low-acuity region;

multiplexing the reorganized first pixels and the second pixels to form a display stream; and

compressing the display stream for transmission to a display, wherein compressing the display stream comprises compressing slices of the display stream on a line-by-line basis, wherein each slice of the display stream includes a number of lines that is an integer fraction of a total number of the modified first pixel lines.

12. The method of claim 10 , further comprising:

blending the original configuration of the first pixels and the second pixels to form blended pixel values representative of the image; and

displaying the blended pixel values on a screen.

13. An apparatus comprising:

a processing unit configured to render first pixels representative of a high-acuity region in an image and second pixels representative of a low-acuity region in the image;

a first buffer configured to store values of the first pixels rendered by the processing unit, wherein the first pixels are organized as a set of first pixel lines in the first buffer;

a second buffer configured to store values of the second pixels rendered by the processing unit, wherein the second pixels are organized as a set of second pixel lines in the second buffer;

a shaper configured to reorganize the first pixels based on at least one dimension of the low-acuity region, wherein the shaper is configured to access the values of the first pixels from the first buffer and to access the values of the second pixels from the second buffer and configured to reorganized the first pixels by concatenating subsets of the set of first pixel lines to form modified first pixel lines including a number of first pixels that is equal to a number of second pixels in the second pixel lines;

a multiplexer configured to multiplex the reorganized first pixels and the second pixels to form a display stream; and

an encoder configured to compress the display stream for transmission to a display.

14. The apparatus of claim 13 , wherein the multiplexer is configured to combine the modified first pixel lines and the second pixel lines on a line-by-line basis.

15. The apparatus of claim 14 , wherein the multiplexer is configured to concatenate the modified first pixel lines and the second pixel lines.

16. The apparatus of claim 15 , wherein the multiplexer is configured to append the modified first pixel lines above, below, to the left, or to the right of the second pixel lines.

17. The apparatus of claim 14 , wherein the multiplexer is configured to interlace the modified first pixel lines and the second pixel lines on a line-by-line basis.

18. The apparatus of claim 17 , wherein the multiplexer is configured to interlace the modified first pixel lines and the second pixel lines into slices having a height that corresponds to a slice height used to compress the display stream.

19. The apparatus of claim 18 , wherein the multiplexer is configured to interlace a number s 2 of slices of the modified first pixel lines with each slice of the second pixel lines, wherein s represents an upscaling factor that is applied to the low-acuity region.

20. The apparatus of claim 17 , wherein the multiplexer is configured to statically interlace the modified first pixel lines at a fixed position relative to the second pixel lines.

21. The apparatus of claim 17 , wherein the multiplexer is configured to dynamically interlace the modified first pixel lines at a variable position relative to the second pixel lines, wherein the variable position is determined based on relative positions of the high-acuity region and the low-acuity region in the image.

22. An apparatus comprising:

a processing unit configured to render first pixels representative of a high-acuity region in an image and second pixels representative of a low-acuity region in the image;

a shaper configured to reorganize the first pixels based on at least one dimension of the low-acuity region;

a multiplexer configured to multiplex the reorganized first pixels and the second pixels to form a display stream; and

an encoder configured to compress the display stream for transmission to a display, wherein the encoder is configured to compress slices of the display stream on a line-by-line basis, wherein each slice of the display stream includes a number of lines that is an integer fraction of a total number of the modified first pixel lines.

23. An apparatus comprising:

a decoder configured to decompress a display stream formed of first pixels having values representative of a high-acuity region in an image and second pixels having values representative of a low-acuity region in the image, wherein the first pixels are reorganized from an array having a first dimension to an array having a corresponding second dimension different than the first dimension prior to being compressed based on at least one dimension of the low-acuity region;

a demultiplexer configured to demultiplex the first pixels and the second pixels; and

a processing unit configured to blend the first pixels and the second pixels to form blended pixel values representative of the image, wherein the processing unit is configured to reconstruct the first pixels from the array having the second dimension into the array having the first dimension.

24. The apparatus of claim 23 , wherein the processing unit is configured to blend the first pixels in the array having the first dimension and the second pixels to form the blended pixel values representative of the image.

25. The apparatus of claim 23 , further comprising:

a first buffer configured to store values of the first pixels received from the demultiplexer, wherein the processing unit is configured to access the first pixels from the first buffer; and

a second buffer configured to store values of the second pixels received from the demultiplexer, wherein the processing unit is configured to access the second pixels from the second buffer.

26. The apparatus of claim 23 , further comprising:

a screen configured to present the image based on the blended pixel values.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2017
From: BASTANI, BEHNAM; VIERI, CARLIN; JIANG, HAOMIAO
To: GOOGLE INC.
Reel/Frame 043001/0333 →
Continuity (1)
Related Publication 20190019315A1 · Jan 17, 2019
Cited By (3)
US 12,212,901 US 12,341,941 US 12,536,706