Environment synthesis for lighting an object

1. A method of rendering a reflective surface of a computer-generated reality (“CGR”) object based on surface synthesis in a CGR environment, comprising:

at a device with one or more processors and a computer readable medium:

determining a first geometry and textural information of a CGR environment, wherein the CGR environment is based on one or more camera images of a physical scene;

determining that the CGR environment includes a surface having a captured surface portion and a missing portion, the surface in the CGR environment representing a surface in the physical scene and the missing portion of the surface in the CGR environment corresponding to an uncaptured portion of the surface in the physical scene;

generating an extension portion for the missing portion based on an assessment of whether the captured surface portion has been cutoff by a boundary defined by a captured geometry, wherein the extension portion is generated by extending a 3D geometry of the captured surface portion based on a characteristic of the captured surface portion;

determining a second geometry of the CGR environment representing the physical scene based on the extension portion;

determining second textural information, wherein the second textural information is synthesized for the extension portion by:

determining a characteristic of the captured surface portion that matches a stored reference characteristic of an object; or

identifying a repeating pattern or texture of the captured surface portion;

generating an environment map using the determined second geometry and the second textural information; and

rendering a CGR object in the CGR environment, wherein an appearance of a reflective surface of the CGR object is determined based on the environment map.

2. The method of claim 1 , further comprising:

determining a three-dimensional (“3D”) position of the CGR object in the CGR environment; and

determining a reflectance of the CGR object, wherein the appearance of the reflective surface of the CGR object is based at least in part on the 3D position and the reflectance of the CGR object.

3. The method of claim 2 , wherein a size of the extension portion is determined based at least in part on the 3D position, a geometry, and the reflectance of the CGR object.

4. The method of claim 1 , further comprising:

blending the extension portion and at least one adjacent portion.

5. The method of claim 1 , further comprising:

selecting a center of projection of the environment map based on a three-dimensional (“3D”) position of the CGR object, wherein generating the environment map is based at least in part on the center of projection of the environment map.

6. The method of claim 1 , wherein the second geometry is determined based at least in part on a plurality of extension portions.

7. The method of claim 1 , further comprising:

generating a plurality of environment maps; and

rendering multiple CGR objects in the CGR environment, wherein appearances of reflective surfaces of the multiple CGR objects are determined based on the plurality of environment maps.

8. The method of claim 1 , wherein the textural information comprises information about color, pattern, reflectance, or irradiance.

9. The method of claim 1 , further comprising:

identifying a darkest texture from the one or more camera images of the physical scene, wherein the textural information is based at least on part on the darkest texture.

10. The method of claim 1 , further comprising:

determining that the environment map includes pixels with no corresponding texture;

generating synthesized texture for the pixels with no corresponding texture; and

enhancing the environment map by replacing the pixels with no corresponding texture with the synthesized texture.

11. The method of claim 1 , wherein the CGR object in the CGR environment is rendered on a screen or on a semi-transparent display apparatus.

12. The method of claim 1 , wherein the extension portion is generated by extending the 3D geometry of the captured surface portion to extend a 3D model of the physical scene to include the extension portion.

13. The method of claim 1 , wherein the extension portion is sized based on the characteristic of the captured surface portion.

14. A system for rendering a reflective surface of a computer-generated reality (“CGR”) object based on environment map synthesis in a CGR environment, comprising:

a device with one or more processors; and

a computer readable medium including instructions that, when executed by the one or more processors, cause the system to:

determine a first geometry and textural information of a CGR environment, wherein the CGR environment is based on one or more camera images of a physical scene;

determine that the CGR environment includes a surface having a captured surface portion a missing portion, the surface in the CGR environment representing a surface in the physical scene and the missing portion of the surface in the CGR environment corresponding to an uncaptured portion of the surface in the physical scene;

generate an extension portion for the missing portion based on an assessment of whether the captured surface portion has been cutoff by a boundary defined by a captured geometry, wherein the extension portion is generated by extending a 3D geometry of the captured surface portion based on a characteristic of the captured surface portion;

determine a second geometry of the CGR environment representing the physical scene based on the extension portion;

determine second textural information, wherein the second textural information is synthesized for the extension portion based on identifying one or more patches of the captured surface portion;

generate an environment map using the determined second geometry and the second textural information; and

render a CGR object in the CGR environment, wherein an appearance of a reflective surface of the CGR object is determined based on the environment map.