Axis based compression for remote rendering
Disclosed herein are related to a system and a method of remotely rendering an image. In one approach, a console device generates an image according to a gaze direction of a user of a head mounted display (HMD). In one aspect, the image includes a first area and a second area disposed along an axis, where the second area is located farther away from a foveated area of the image than the first area. In one aspect, the foveated area corresponds to the gaze direction of the user of the HMD. In one aspect, the console device compresses the image according to the axis, where the second area is compressed at a higher level than the first area. In one aspect, the compressed image is transmitted to the HMD. The HMD may decompress the compressed image according to the axis, and render the decompressed image.
1. A device comprising:
at least one processor configured to:
receive, from another device through a communication link, a compressed image, the compressed image including a first set of areas allocated by a first set of non-uniformly spaced axes and a second set of non-uniformly spaced axes intersecting the first set of non-uniformly spaced axes, the first set of areas including a first area and a second area, along an axis of the first set of non-uniformly spaced axes, the second area located farther away from a foveated area of the compressed image than the first area,
modify the compressed image to obtain a modified image by decompressing the compressed image according to compression information indicating the first set of non-uniformly spaced axes and the second set of non-uniformly spaced axes, the modified image including a second set of areas allocated by a third set of uniformly spaced axes and a fourth set of uniformly spaced axes intersecting the third set of uniformly spaced axes, the second area decompressed at a higher level than the first area, and
render the modified image to a user of the device.
2. The device of claim 1 , wherein the at least one processor is configured to detect a gaze direction of the user, wherein the foveated area corresponds to the gaze direction of the user.
3. The device of claim 1 , wherein the axis is parallel to a horizontal side of the compressed image, or is parallel to a vertical side of the compressed image.
4. The device of claim 1 , wherein the second area has a smaller size than the first area, wherein the decompressed second area and the decompressed first area have a same size.
5. The device of claim 1 , wherein each of the third set of uniformly spaced axes corresponds to a respective one of the first set of non-uniformly spaced axes, and wherein each of the fourth set of uniformly spaced axes corresponds to a respective one of the second set of non-uniformly spaced axes.
6. The device of claim 1 , wherein the at least one processor is configured to apply a predistortion to the decompressed image to correct for a distortion due to a lens of the device.
7. The device of claim 1 , wherein the at least one processor is configured to receive the compression information with the compressed image.
8. The device of claim 1 , wherein the at least one processor is configured to store the compression information prior to receiving the compressed image.
9. The device of claim 1 , wherein the at least one processor is configured to:
compare a length of a portion of the axis with a length of a corresponding portion of another axis,
determine a ratio between the length of the portion of the axis and the length of the corresponding portion of the another axis, and
modify the compressed image by decompressing the compressed image according to the determined ratio.
10. A method comprising:
receiving, by a device from another device through a communication link, a compressed image, the compressed image including a first set of areas allocated by a first set of non-uniformly spaced axes and a second set of non-uniformly spaced axes intersecting the first set of non-uniformly spaced axes, the first set of areas including a first area and a second area, along an axis of the first set of non-uniformly spaced axes, the second area located farther away from a foveated area of the compressed image than the first area;
modifying, by the device, the compressed image to obtain a modified image by decompressing the compressed image according to compression information indicating the first set of non-uniformly spaced axes and the second set of non-uniformly spaced axes, the modified image including a second set of areas allocated by a third set of uniformly spaced axes and a fourth set of uniformly spaced axes intersecting the third set of uniformly spaced axes, the second area decompressed at a higher level than the first area; and
rendering the modified image to a user of the device.
11. The method of claim 10 , comprising:
detecting a gaze direction of the user, wherein the foveated area corresponds to the gaze direction of the user.
12. The method of claim 10 , wherein the axis is parallel to a horizontal side of the compressed image, or is parallel to a vertical side of the compressed image.
13. The method of claim 10 , wherein the second area has a smaller size than the first area, wherein the decompressed second area and the decompressed first area have a same size.
14. The method of claim 10 , wherein each of the third set of uniformly spaced axes corresponds to a respective one of the first set of non-uniformly spaced axes, and wherein each of the fourth set of uniformly spaced axes corresponds to a respective one of the second set of non-uniformly spaced axes.
15. The method of claim 10 , comprising:
applying a predistortion to the decompressed image to correct for a distortion due to a lens of the device.
16. The method of claim 10 , comprising:
receiving the compression information with the compressed image.
17. The method of claim 10 , comprising:
storing the compression information prior to receiving the compressed image.
18. The method of claim 10 , comprising:
comparing a length of a portion of the axis with a length of a corresponding portion of another axis;
determining a ratio between the length of the portion of the axis and the length of the corresponding portion of the another axis; and
modifying the compressed image by decompressing the compressed image according to the determined ratio.
19. A non-transitory computer readable medium storing program instructions for causing at least one processor of a device to implement operations of:
receiving, from another device through a communication link, a compressed image, the compressed image including a first set of areas allocated by a first set of non-uniformly spaced axes and a second set of non-uniformly spaced axes intersecting the first set of non-uniformly spaced axes, the first set of areas including a first area and a second area, along an axis of the first set of non-uniformly spaced axes, the second area located farther away from a foveated area of the compressed image than the first area,
modifying the compressed image to obtain a modified image by decompressing the compressed image according to compression information indicating the first set of non-uniformly spaced axes and the second set of non-uniformly spaced axes, the modified image including a second set of areas allocated by a third set of uniformly spaced axes and a fourth set of uniformly spaced axes intersecting the third set of uniformly spaced axes, the second area decompressed at a higher level than the first area, and
rendering the modified image to a user of the device.
20. The non-transitory computer readable medium of claim 19 , wherein the second area has a smaller size than the first area, wherein the decompressed second area and the decompressed first area have a same size.