IP Library Granted Patent US 9,883,137
Granted Patent B2
US 9,883,137 · App. 14/931,358 · Granted Jan 30, 2018

Updating regions for display based on video decoding mode

Inventors: Dileep Marchya (San Diego, CA); Mastan Manoj Kumar Amara Venkata (San Diego, CA); Sudeep Ravi Kottilingal (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N5/915H04N5/147
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,883,137
App. No.
14/931,358
Granted
Jan 30, 2018
Kind
B2
Abstract

Techniques are described in which a video decoding mode is used to determine which regions of a picture need to be composed. If a region of a current picture is decoded in skip mode with the reference picture of the skip mode being a previous picture that is displayed immediately prior to the current picture, then pixel values for that region may not need to be retrieved from system memory.

Claims (75)

1. A method of composing images for display, the method comprising:

reconstructing at least a portion of a current picture based on pixel values of a previous picture that is displayed immediately prior to the current picture, wherein reconstructing comprises generating one or more first regions in the current picture by copying pixel values of one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

subsequent to reconstructing the current picture, identifying, with processing circuitry and based on decoding modes that were used for reconstructing the current picture, the one or more first regions in the current picture that were generated by copying the pixel values of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

subsequent to reconstructing the current picture, identifying, with the processing circuitry, one or more second regions of the current picture that changed relative to the previous picture based on the identified one or more first regions in the current picture; and

providing, with the processing circuitry, instructions to a display processor to retrieve pixel values of the identified one or more second regions of the current picture that changed relative to the previous picture from a system memory, and no pixel values for the identified one or more first regions from the system memory, for composing the current picture with one or more layers of image content to generate a composed image.

2. The method of claim 1 , further comprising:

providing instructions to the display processor to output information used to display the composed image.

3. The method of claim 1 , further comprising:

identifying an updating region based on a union of the identified one or more second regions of the current picture that changed relative to the previous picture and respective regions of the one or more layers of image content having image content that changed relative to a previous composed image that is displayed immediately prior to the display of the composed image; and

providing instructions to the display processor to compose the identified updating region.

4. The method of claim 1 , wherein the current picture comprises a first picture, the method further comprising:

determining that an entirety of a second picture was generated by copying pixel values of a third picture that is displayed immediately prior to the second picture;

in response to determining that the entirety of the second picture was generated by copying pixel values, providing instructions to the display processor to compose the second picture with the one or more layers of image content without retrieving any pixel values for the second picture from the system memory.

5. The method of claim 1 , wherein providing instructions to the display processor to retrieve pixel values comprises providing instructions to the display processor to retrieve pixel values of the identified one or more second regions of the current picture that changed from a layer buffer in the system memory, the method further comprising:

providing instructions to the display processor to retrieve pixel values of the identified one or more first regions from a local memory of the display processor.

6. The method of claim 1 , wherein identifying the one or more first regions in the current picture that were generated by copying pixel values of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture comprises:

determining that the one or more first regions in the current picture were decoded in a skip mode; and

determining that a reference picture for the skip mode was the previous picture that is displayed immediately prior to the current picture.

7. The method of claim 1 , wherein the processing circuitry includes the display processor.

8. The method of claim 1 , further comprising composing, with the display processor, the identified one or more second regions with the one or more layers of image content to generate the composed image by stitching the identified one or more second regions with the image content of the one or more layers of image content.

9. A device for composing images for display, the device comprising:

a display processor;

a system memory configured to store a previous picture that is displayed immediately prior to a current picture; and

processing circuitry configured to:

reconstruct at least a portion of a current picture based on pixel values of a previous picture that is displayed immediately prior to the current picture, wherein to reconstruct, the processing circuitry is configured to generate one or more first regions in the current picture by copying pixel values of one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

subsequent to reconstructing the current picture, identify, based on decoding modes that were used for reconstructing the current picture, the one or more first regions in the current picture that were generated by copying the pixel values stored in the system memory of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

subsequent to reconstructing the current picture, identify one or more second regions of the current picture that changed relative to the previous picture based on the identified one or more first regions in the current picture; and

provide instructions to the display processor to retrieve pixel values of the identified one or more second regions of the current picture that changed relative to the previous picture from the system memory, and no pixel values for the identified one or more first regions from the system memory, for composing the current picture with one or more layers of image content to generate a composed image.

10. The device of claim 9 , wherein the processing circuitry is configured to provide instructions to the display processor to output information used to display the composed image.

11. The device of claim 9 , wherein the processing circuitry is configured to:

identify an updating region based on a union of the identified one or more second regions of the current picture that changed relative to the previous picture and respective regions of the one or more layers of image content having image content that changed relative to a previous composed image that is displayed immediately prior to the display of the composed image; and

provide instructions to the display processor to compose the identified updating region.

12. The device of claim 9 , wherein the current picture comprises a first picture, and wherein the processing circuitry is configured to:

determine that an entirety of a second picture was generated by copying pixel values of a third picture that is displayed immediately prior to the second picture;

in response to determining that the entirety of the second picture was generated by copying pixel values, provide instructions to the display processor to compose the second picture with the one or more layers of image content without retrieving any pixel values for the second picture from the system memory.

13. The device of claim 9 , further comprising:

a layer buffer in the system memory,

wherein the processing circuitry is configured to:

provide instructions to the display processor to retrieve pixel values of the identified one or more second regions of the current picture that changed from the layer buffer in the system memory; and

provide instructions to the display processor to retrieve pixel values of the identified one or more first regions from a local memory of the display processor.

14. The device of claim 9 , wherein to identify the one or more first regions in the current picture that were generated by copying pixel values of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture, the processing circuitry is configured to:

determine that the one or more first regions in the current picture were decoded in a skip mode; and

determine that a reference picture for the skip mode was the previous picture that is displayed immediately prior to the current picture.

15. The device of claim 9 , wherein the processing circuitry includes the display processor.

16. The device of claim 9 , wherein the display processor is configured to compose the identified one or more second regions with one or more layers of image content to generate the composed image by stitching the identified one or more second regions with the image content of the one or more layers of image content.

17. The device of claim 9 , wherein the device comprises one of:

a wireless device comprising a display configured to display the composed image;

an integrated circuit; or

a microprocessor.

18. A device for composing images for display, the device comprising:

means for reconstructing at least a portion of a current picture based on pixel values of a previous picture that is displayed immediately prior to the current picture, wherein reconstructing comprises generating one or more first regions in the current picture by copying pixel values of one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

means for identifying, subsequent to reconstructing the current picture and based on decoding modes that were used for reconstructing the current picture, the one or more first regions in the current picture that were generated by copying the pixel values of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

means for identifying, subsequent to reconstructing the current picture, one or more second regions of the current picture that changed relative to the previous picture based on the identified one or more first regions in the current picture; and

means for providing instructions to a display processor to retrieve pixel values of the identified one or more second regions of the current picture that changed relative to the previous picture from a system memory, and no pixel values for the identified one or more first regions from the system memory, for composing the current picture with one or more layers of image content to generate a composed image.

19. The device of claim 18 , further comprising:

means for providing instructions to the display processor to output information used to display the composed image.

20. The device of claim 18 , further comprising:

means for identifying an updating region based on a union of the identified one or more second regions of the current picture that changed relative to the previous picture and respective regions of the one or more layers of image content having image content that changed relative to a previous composed image that is displayed immediately prior to the display of the composed image; and

means for providing instructions to the display processor to compose the identified updating region.

21. The device of claim 18 , wherein the means for identifying the one or more first regions in the current picture that were generated by copying pixel values of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture comprises:

means for determining that the one or more first regions in the current picture were decoded in a skip mode; and

means for determining that a reference picture for the skip mode was the previous picture that is displayed immediately prior to the current picture.

22. A computer-readable storage medium having instructions stored thereon that when executed cause one or more processors of a device for composing images for display to:

reconstruct at least a portion of a current picture based on pixel values of a previous picture that is displayed immediately prior to the current picture, wherein the instructions that cause the one or processors to reconstruct comprise instructions that cause the one or more processors to generate one or more first regions in the current picture by copying pixel values of one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

subsequent to reconstructing the current picture, identify, based on decoding modes that were used for reconstructing the current picture, the one or more first regions in the current picture that were generated by copying the pixel values of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture;

subsequent to reconstructing the current picture, identify one or more second regions of the current picture that changed relative to the previous picture based on the identified one or more first regions in the current picture; and

provide instructions to a display processor to retrieve pixel values of the identified one or more second regions of the current picture that changed relative to the previous picture from a system memory, and no pixel values for the identified one or more first regions from the system memory, for composing the current picture with one or more layers of image content to generate a composed image.

23. The computer-readable storage medium of claim 22 , further comprising instructions that cause the one or more processors to:

provide instructions to the display processor to output information used to display the composed image.

24. The computer-readable storage medium of claim 22 , further comprising instructions that cause the one or more processors to:

identify an updating region based on a union of the identified one or more second regions of the current picture that changed relative to the previous picture and respective regions of the one or more layers of image content having image content that changed relative to a previous composed image that is displayed immediately prior to the display of the composed image; and

provide instructions to the display processor to compose the identified updating region.

25. The computer-readable storage medium of claim 22 , wherein the instructions that cause the one or more processors to identify the one or more first regions in the current picture that were generated by copying pixel values of the one or more respective co-located regions in the previous picture that is displayed immediately prior to the current picture comprise instructions that cause the one or more processors to:

determine that the one or more first regions in the current picture were decoded in a skip mode; and

determine that a reference picture for the skip mode was the previous picture that is displayed immediately prior to the current picture.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2015
From: MARCHYA, DILEEP; AMARA VENKATA, MASTAN MANOJ KUMAR; KOTTILINGAL, SUDEEP RAVI
To: QUALCOMM INCORPORATED
Reel/Frame 036950/0298 →
Continuity (1)
Related Publication 20170127012A1 · May 4, 2017