Independent scene movement based on mask layers
Techniques and systems are provided for image generation. For instance, a process can include rendering an image of a scene, the scene including a first virtual object associated with a first physical object that moves independently of the scene, identifying a first set of pixels associated with the first virtual object in the image of the scene, generating a mask layer based on the first set of pixels, the mask layer indicating locations of the first set of pixels in the image, and transmitting the mask layer and the image to a display.
1 . An apparatus for image generation, comprising:
a memory comprising instructions; and
a processor coupled to the memory and configured to:
render an image of a scene, the scene including a first virtual object associated with a first physical object that moves independently of the scene;
identify a first set of pixels associated with the first virtual object in the image of the scene;
generate a mask layer based on the first set of pixels, the mask layer indicating locations of the first set of pixels in the image; and
transmit the mask layer and the image to a display.
2 . The apparatus of claim 1 , wherein the display is coupled to, but separate from the apparatus.
3 . The apparatus of claim 1 , wherein the mask layer includes an object identifier corresponding to pixels of the image.
4 . The apparatus of claim 3 , wherein, to generate the mask layer, the processor is configured to set locations in the mask layer corresponding with the identified first set of pixels with the object identifier associated with the first virtual object.
5 . The apparatus of claim 1 , wherein the processor is configured to:
generate depth information for the first virtual object; and
transmit the depth information along with the mask layer and the image to the display.
6 . The apparatus of claim 1 , wherein the processor is configured to determine that the first physical object can move independently of the scene.
7 . The apparatus of claim 6 , wherein, to determine that the first physical object can move independently of the scene, the processor is configured to:
receive an indication that the display has stored pose information associated with the first physical object; and
determine that the first physical object can move independently of the scene based on the received indication.
8 . The apparatus of claim 1 , wherein the scene includes a second virtual object associated with a second physical object that moves independently of the scene, and wherein the processor is configured to:
render the image of the scene including the second virtual object; identify a second set of pixels associated with the second virtual object in the image of the scene; and
generate the mask layer based on the first set of pixels and the second set of pixels.
9 . The apparatus of claim 1 , wherein the apparatus comprises a mobile phone.
10 . An apparatus for displaying images, comprising:
a memory comprising instructions; and
a processor coupled to the memory and configured to:
obtain a mask layer and an image associated with the mask layer, wherein the mask layer is associated with a first virtual object;
generate updated pose information for the apparatus, wherein the updated pose information indicates movement between a time when the image was generated and a current time;
obtain updated pose information for a first physical object, wherein the first physical object moves independently of the apparatus, and wherein the first physical object is associated with the first virtual object;
determine a first amount to warp the image based on the updated pose information for the apparatus;
determine a second amount to warp a first portion of the image based on the mask layer and the updated pose information for the first physical object;
warp the image, based on the first amount to warp the image, and the first portion of the image, based on the second amount to warp a first virtual object with the first physical object; and
display a final warped image based on the warping of the image and the warping of the first portion of the image.
11 . The apparatus of claim 10 , wherein the mask layer includes an object identifier corresponding to pixels of the image.
12 . The apparatus of claim 10 , wherein the processor is configured to:
receive depth information associated with the first virtual object along with the mask layer and image; and
determine the second amount to warp the first portion of the image further based on the depth information.
13 . The apparatus of claim 10 , wherein the processor is configured to:
obtain first pose information for the first physical object;
generate second pose information for the apparatus; and
transmit the first pose information and second pose information to a companion device for rendering images for the apparatus.
14 . The apparatus of claim 10 , wherein the mask layer is associated with a second virtual object associated with a second physical object, and wherein processor is configured to:
obtain updated pose information for the second physical object;
determine a third amount to warp a second portion of the image based on the mask layer and the updated pose information for the second physical object; and
warp the second portion of the image based on the third amount.
15 . A method for image generation, comprising:
rendering an image of a scene, the scene including a first virtual object associated with a first physical object that moves independently of the scene;
identifying a first set of pixels associated with the first virtual object in the image of the scene;
generating a mask layer based on the first set of pixels, the mask layer indicating locations of the first set of pixels in the image; and
transmitting the mask layer and the image to a display.
16 . The method of claim 15 , wherein the display is coupled to, but separate from a companion device.
17 . The method of claim 15 , wherein the mask layer includes an object identifier corresponding to pixels of the image.
18 . The method of claim 17 , wherein generating the mask layer comprises setting locations in the mask layer corresponding with the identified first set of pixels with the object identifier associated with the first virtual object.
19 . The method of claim 15 , further comprising:
generating depth information for the first virtual object; and
transmitting the depth information along with the mask layer and the image to the display.
20 . The method of claim 15 , further comprising determining that the first physical object can move independently of the scene.
21 . The method of claim 20 , wherein determining that the first physical object can move independently of the scene comprises:
receiving an indication that the display has stored pose information associated with the first physical object; and
determining that the first physical object can move independently of the scene based on the received indication.
22 . The method of claim 15 , wherein the scene includes a second virtual object associated with a second physical object that moves independently of the scene, and further comprising:
rendering the image of the scene including the second virtual object;
identifying a second set of pixels associated with the second virtual object in the image of the scene; and
generate the mask layer based on the first set of pixels and the second set of pixels.
23 . A method for displaying images, comprising:
obtaining a mask layer and an image associated with the mask layer, wherein the mask layer is associated with a first virtual object;
generating updated pose information for an apparatus, wherein the updated pose information indicates movement between a time when the image was generated and a current time;
obtaining updated pose information for a first physical object, wherein the first physical object moves independently of the apparatus, and wherein the first physical object is associated with the first virtual object;
determining a first amount to warp the image based on the updated pose information for the apparatus;
determining a second amount to warp a first portion of the image based on the mask layer and the updated pose information for the first physical object;
warping the image, based on the first amount to warp the image, and the first portion of the image, based on the second amount to warp a first virtual object with the first physical object; and
displaying a final warped image based on the warping of the image and the warping of the first portion of the image.
24 . The method of claim 23 , wherein the mask layer includes an object identifier corresponding to pixels of the image.
25 . The method of claim 23 , further comprising:
receiving depth information associated with the first virtual object along with the mask layer and image; and
determining the second amount to warp the first portion of the image further based on the depth information.
26 . The method of claim 23 , further comprising:
obtaining first pose information for the first physical object;
generating second pose information for the apparatus; and
transmitting the first pose information and second pose information to a companion device for rendering images.
27 . The method of claim 23 , wherein the mask layer is associated with a second physical object, and further comprising:
obtaining updated pose information for the second physical object;
determining a third amount to warp a second portion of the image based on the mask layer and the updated pose information for the second physical object; and
warping the second portion of the image based on the third amount.
28 . A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to:
render an image of a scene, the scene including a first virtual object associated with a first physical object that moves independently of the scene;
identify a first set of pixels associated with the first virtual object in the image of the scene;
generate a mask layer based on the first set of pixels, the mask layer indicating locations of the first set of pixels in the image; and
transmit the mask layer and the image to a display.
29 . A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to:
obtain, by a display device, a mask layer and an image associated with the mask layer, wherein the mask layer is associated with a first virtual object;
generate updated pose information for an apparatus, wherein the updated pose information indicates movement between a time when the image was generated and a current time;
obtain updated pose information for a first physical object, wherein the first physical object moves independently of the apparatus, and wherein the first physical object is associated with the first virtual object;
determine a first amount to warp the image based on the updated pose information for the apparatus;
determine a second amount to warp a first portion of the image based on the mask layer and the updated pose information for the first physical object;
warp the image, based on the first amount to warp the image based on the updated pose information, and the first portion of the image, based on the second amount to warp a first virtual object with the first physical object; and
display a final warped image based on the warping of the image and the warping of the first portion of the image.