System for performing virtual image quality assessment of image data based on digital twin and operating method thereof

1 . A virtual image quality assessment system comprising:

an electronic device configured to:

receive simulation condition data based on a user input;

perform an optical simulation based on the simulation condition data;

perform a sensor simulation based on the simulation condition data and a result of the optical simulation;

generate, based on a result of the sensor simulation, raw image data representative of data output from an image sensor; and

perform an image quality assessment simulation based on the raw image data generated based on the result of the sensor simulation.

2 . The virtual image quality assessment system of claim 1 , wherein the electronic device comprises:

a random access memory (RAM) configured to store the simulation condition data;

at least one processor configured to:

perform the optical simulation and the sensor simulation; and

receive the simulation condition data from the RAM and to control at least one operation of the optical simulation or the sensor simulation based on the simulation condition data.

3 . The virtual image quality assessment system of claim 1 , wherein the optical simulation is performed based on a first subset of the simulation condition data, the first subset comprising first input data that corresponds to a virtual reality scene, and

wherein the optical simulation comprises modeling the virtual reality scene.

4 . The virtual image quality assessment system of claim 3 , wherein the first subset of the simulation condition data further comprises second input data that corresponds to a lens and third input data that corresponds to a virtual light source, and

wherein, when the first input data is three-dimensional data, the electronic device is further configured to operate in a first operation mode that comprises modeling the virtual reality scene in three dimensions based on the second input data, the third input data, and a lens design value.

5 . The virtual image quality assessment system of claim 4 , wherein, when the first input data is two-dimensional data, the electronic device is further configured to operate in a second operation mode that comprises modeling the virtual reality scene in two dimensions based on the second input data, the third input data, and lens effect data.

6 . The virtual image quality assessment system of claim 5 , wherein first photon data is generated as a result of the first operation mode and second photon data is generated as a result of the second operation mode, and

wherein the first photon data is the same as the second photon data.

7 . The virtual image quality assessment system of claim 5 , wherein the sensor simulation is performed based on the result of the optical simulation and a second subset of the simulation condition data, the second subset comprising fourth input data that corresponds to the image sensor.

8 . The virtual image quality assessment system of claim 7 , wherein the sensor simulation comprises generating the raw image data representative of data output from the image sensor by performing a pixel simulation, a noise simulation, and a digital signal output simulation based on the fourth input data and first photon data generated as a result of the first operation mode.

9 . The virtual image quality assessment system of claim 7 , wherein the sensor simulation comprises generating the raw image data representative of data output from the image sensor by performing a pixel simulation, a noise simulation, and a digital signal output simulation based on the fourth input data and second photon data generated as a result of the second operation mode.

10 . A method for simulating a virtual image quality assessment with respect to a virtual reality scene, the method comprising:

generating, by a metaverse implemented by at least one processor, simulation condition data based on a user input;

performing, by the metaverse implemented by the at least one processor, an optical simulation based on the simulation condition data;

generating, by the metaverse implemented by the at least one processor, by performing a sensor simulation based on the simulation condition data and a result of the optical simulation, raw image data representative of data output from an image sensor;

performing, by the metaverse implemented by the at least one processor, an image signal processing simulation based on the simulation condition data and the raw image data; and

performing, by the metaverse implemented by the at least one processor, an image quality assessment simulation based on the simulation condition data and image data generated as a result of the image signal processing simulation.

11 . The method of claim 10 , wherein the performing of the optical simulation is based on a first subset of the simulation condition data, the first subset comprising first input data corresponding to the virtual reality scene, and

wherein the performing of the optical simulation comprises modeling the virtual reality scene.

12 . The method of claim 11 , wherein the first subset of the simulation condition data further comprises second input data corresponding to a lens and third input data corresponding to a virtual light source, and

wherein, when the first input data is three-dimensional data, the modeling of the virtual reality scene comprises modeling, in a first operation mode, the virtual reality scene in three dimensions based on the second input data, the third input data, and a lens design value.

13 . The method of claim 12 , wherein, when the first input data is two-dimensional data, the modeling of the virtual reality scene comprises modeling, in a second operation mode, the virtual reality scene in two dimensions based on the second input data, the third input data, and lens effect data.

14 . The method of claim 13 , wherein the performing of the sensor simulation is based on the result of the optical simulation and a second subset of the simulation condition data, the second subset comprising fourth input data corresponding to the image sensor, and

wherein the generating of the raw image data representative of data output from the image sensor comprises performing a pixel simulation, a noise simulation, and a digital signal output simulation based on the fourth input data and first photon data generated as a result of the first operation mode.

15 . The method of claim 13 , wherein the performing of the sensor simulation is based on the result of the optical simulation and a second subset of the simulation condition data, the second subset comprising fourth input data corresponding to the image sensor, and

wherein the generating of the raw image data representative of data output from the image sensor comprises performing a pixel simulation, a noise simulation, and a digital signal output simulation based on the fourth input data and second photon data generated as a result of the second operation mode.

16 . The method of claim 10 , wherein the performing of the image signal processing simulation is based on the raw image data and a third subset of the simulation condition data, the third subset comprising fifth input data corresponding to the image signal processing simulation, and

wherein the performing of the image signal processing simulation comprises generating the image data based on the raw image data and the fifth input data.

17 . The method of claim 10 , wherein the performing of the image quality assessment simulation is based on the image data and a fourth subset of the simulation condition data, the fourth subset comprising sixth input data corresponding to the image quality assessment simulation, and

wherein the metaverse performs the image quality assessment simulation based on the image data and the sixth input data.

18 . A virtual image quality assessment system comprising:

a metaverse implemented by at least one processor, the metaverse configured to generate simulation condition data based on a user input, the metaverse configured to:

receive the simulation condition data based on the user input;

perform an optical simulation and a sensor simulation based on the simulation condition data; and

generate, based on a result of the optical simulation and the sensor simulation, raw image data representative of data output from an image sensor,

wherein the metaverse is further configured to perform an image signal processing simulation and an image quality assessment simulation with respect to the raw image data.

19 . The virtual image quality assessment system of claim 18 , wherein the simulation condition data comprises first input data that corresponds to a virtual reality scene, second input data that corresponds to a lens, and third input data that corresponds to a virtual light source, and

wherein the metaverse implemented by the at least one processor is further configured to:

operate in a first operation mode, when the virtual reality scene is three-dimensional charts, that comprises modeling the virtual reality scene in three dimensions based on the second input data, the third input data, and a lens design value; and

operate in a second operation mode, when the virtual reality scene is two-dimensional charts, that comprises modeling the virtual reality scene in two dimensions based on the second input data, the third input data, and lens effect data.

20 . The virtual image quality assessment system of claim 19 , wherein the simulation condition data further comprises fourth input data that corresponds to the image sensor,

wherein the sensor simulation comprises generating the raw image data representative of data output from the image sensor by performing a pixel simulation, a noise simulation, and a digital signal output simulation based on the fourth input data and one of first photon data generated as a result of the first operation mode or second photon data generated as a result of the second operation mode,

wherein the simulation condition data further comprises fifth input data that corresponds to the image signal processing simulation and sixth input data that corresponds to the image quality assessment simulation,

wherein the metaverse performs the image signal processing simulation based on the raw image data and the fifth input data, and

wherein the metaverse performs the image quality assessment simulation based on the sixth input data and image data generated as a result of the image signal processing simulation.