Method and apparatus for improving visual coherency on rendering 3D real objects virtualized in extended reality
View Patent ↗Proposed is a rendering method and device for providing extended reality experiences. The rendering method includes obtaining image data in a first condition by an image data sensing part, the image data including a target real object to be virtualized, deriving a raw-to-sRGB conversion matrix on the basis of the obtained image data in the first condition by a raw-to-sRGB conversion estimation part, generating texture data of the target real object by using the raw-to-sRGB conversion matrix by an object texture data conversion generation part, virtualizing the target real object by using the object texture data and generating a virtualized real object by a real-time processing part, and rendering the virtualized real object by an object/background rendering part.
1 . A rendering method for extended reality, the rendering method comprising:
obtaining, by an image data sensing part, image data in a first condition, the image data including a target real object to be virtualized;
deriving, by a raw-to-sRGB conversion estimation part, a raw-to-sRGB conversion matrix on the basis of the obtained image data in the first condition;
generating, by an object texture data conversion generation part, texture data of the target real object by using the raw-to-sRGB conversion matrix;
virtualizing, by a real-time processing part, the target real object by using the object texture data and generating a virtualized real object;
rendering, by an object/background rendering part, the virtualized real object; and
providing, by an object/background rendering part, extended reality experiences.
2 . The rendering method of claim 1 , wherein the generating of the texture data of the target real object to be virtualized by using the raw-to-sRGB conversion matrix by the object texture data conversion generation part comprises:
deriving a raw-to-raw conversion matrix for estimating raw-to-raw conversion, on the basis of the image data in the first condition and image data in a second condition; and
performing conversion into the texture data of the target real object on the basis of the raw-to-raw conversion matrix, and generating the texture data of the target real object by using the raw-to-sRGB conversion matrix.
3 . The rendering method of claim 2 , wherein the first condition includes a characteristic of lighting or a characteristic of a camera in an environment in which extended reality experiences are provided or both, and
the second condition includes a characteristic of lighting or a characteristic of a camera in an environment in which a 3D modeling part operates offline or in a pre-test environment, or both.
4 . The rendering method of claim 1 , wherein the deriving of the raw-to-sRGB conversion matrix on the basis of the obtained image data by the raw-to-sRGB conversion estimation part comprises:
deriving the raw-to-sRGB conversion matrix on the basis of a reference color chart raw image data and a reference color chart sRGB image data included in the image data of the first condition.
5 . The rendering method of claim 1 , further comprising:
receiving user input for the virtualized real object; and
outputting a screen with updated interaction corresponding to the user input.
6 . A rendering device, comprising:
an image data sensing part configured to obtain image data in a first condition, the image data including a target real object to be virtualized;
a raw-to-sRGB conversion estimation part configured to derive a raw-to-sRGB conversion matrix on the basis of the obtained image data in the first condition;
an object texture data conversion generation part configured to generate texture data of the target real object by using the raw-to-sRGB conversion matrix;
a real-time processing part configured to virtualize the target real object by using the object texture data and generate a virtualized real object; and
an object/background rendering part configured to render the virtualized real object and provide extended reality experiences.
7 . The rendering device of claim 6 , wherein when generating the texture data of the target real object to be virtualized by using the raw-to-sRGB conversion matrix, the object texture data conversion generation part is configured to
derive a raw-to-raw conversion matrix for estimating raw-to-raw conversion, on the basis of the image data in the first condition and image data in a second condition, perform conversion into the texture data of the target real object on the basis of the raw-to-raw conversion matrix, and generate the texture data of the target real object by using the raw-to-sRGB conversion matrix.
8 . The rendering device of claim 7 , wherein the first condition includes a characteristic of lighting or a characteristic of a camera in an environment in which the extended reality experiences are provided or both, and
the second condition includes a characteristic of lighting or a characteristic of a camera in an environment in which a 3D modeling part operates offline or in a pre-test environment, or both.
9 . The rendering device of claim 6 , wherein when deriving the raw-to-sRGB conversion matrix on the basis of the obtained image data, the raw-to-sRGB conversion estimation part is configured to
derive the raw-to-sRGB conversion matrix on the basis of a reference color chart raw image data and a reference color chart sRGB image data included in the image data of the first condition.
10 . The rendering device of claim 6 , wherein user input for the virtualized real object is received, and a screen with updated interaction corresponding to the user input is output.