System and method for procedural reconstruction of synthesized images
Methods and systems for obtaining synthesized images are disclosed. To obtain synthesized images, an imaging system may obtain areas of interest associated with an image. The areas of interest may be keyed to various processes and/or services. A downstream consumer of the services may retrieve the areas of interest to perform further analysis. In order to more easily interpret the areas of interest, the areas of interest may be displayed to the downstream consumer as part of a synthesized image. The synthesized image may include the areas of interest and a synthesized environment. The synthesized environment may be intended to emulate an environment in which the downstream consumer may expect to find the areas of interest. By doing so, the downstream consumer may be able to understand and contextualize the areas of interest more easily than if viewing the areas of interest alone.
1. A method for managing generation of synthesized images, the method comprising:
obtaining an area of interest associated with an image, the area of interest indicating a portion of a scene depicted by the image being relevant to a process performed by a downstream consumer, and the portion of the scene being one of multiple portions of the scene depicted by the image;
obtaining a synthesized environment in which the portion of the scene naturally resides from a perspective of the downstream consumer, the synthesized environment depicting a type of scene similar in contextual information to a type of the scene depicted by other portions of the multiple portions of the scene depicted by the image, the contextual information providing the downstream consumer with an understanding of an original version the image without the downstream consumer actually needing to have seen the original version;
obtaining a synthesized image of the synthesized images using the area of interest and the synthesized environment, the synthesized image depicting the area of interest in the synthesized environment to facilitate presentation of the area of interest in a common environment in which the area of interest would be expected to be found by the downstream consumer; and
providing the synthesized image to the downstream consumer.
2. The method of claim 1 , wherein obtaining the synthesized environment comprises:
performing a procedural reconstruction process to obtain the synthesized environment using the area of interest, the synthesized environment comprising a second image about the area of interest.
3. The method of claim 2 , wherein portions of the area of interest adjacent to the second image about the area of interest are matched to disguise a boundary between the area of interest and the second image.
4. The method of claim 3 , wherein performing the procedural reconstruction process comprises:
ingesting the area of interest into a trained inference model to obtain the second image as output from the trained inference model.
5. The method of claim 4 , wherein performing the procedural reconstruction process further comprises:
ingesting at least a second area of the image into the trained inference model to obtain the output.
6. The method of claim 5 , wherein the trained inference model is a trained neural network.
7. The method of claim 1 , wherein the portion of the scene is relevant to a medical diagnosis through interpretation of the image, and the other portions of the multiple portions of the scene being irrelevant to the medical diagnosis made using the image.
8. The method of claim 7 , wherein the synthesized image presents the area of interest to a subsequent interpreter of the synthesized image in the common environment to facilitate interpretation by the subsequent interpreter without distraction due to a limited extent of the area of interest compared to an extent of the image.
9. The method of claim 1 , wherein obtaining the area of interest comprises:
obtaining image segments, at least one of image segments comprising a portion of the area of interest and a portion of the image outside the area of interest; and
isolating the portion of the area of interest from the portion of the image outside the area of interest to obtain a portion of the area of interest.
10. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing generation of synthesized images, the operations comprising:
obtaining an area of interest associated with an image, the area of interest indicating a portion of a scene depicted by the image being relevant to a process performed by a downstream consumer, and the portion of the scene being one of multiple portions of the scene depicted by the image;
obtaining a synthesized environment in which the portion of the scene naturally resides from a perspective of the downstream consumer, the synthesized environment depicting a type of scene similar in contextual information to a type of the scene depicted by other portions of the multiple portions of the scene depicted by the image, the contextual information providing the downstream consumer with an understanding of an original version the image without the downstream consumer actually needing to have seen the original version;
obtaining a synthesized image of the synthesized images using the area of interest and the synthesized environment, the synthesized image depicting the area of interest in the synthesized environment to facilitate presentation of the area of interest in a common environment in which the area of interest would be expected to be found by the downstream consumer; and
providing the synthesized image to the downstream consumer.
11. The non-transitory machine-readable medium of claim 10 , wherein obtaining the synthesized environment comprises:
performing a procedural reconstruction process to obtain the synthesized environment using the area of interest, the synthesized environment comprising a second image about the area of interest.
12. The non-transitory machine-readable medium of claim 11 , wherein portions of the area of interest adjacent to the second image about the area of interest are matched to disguise a boundary between the area of interest and the second image.
13. The non-transitory machine-readable medium of claim 12 , wherein performing the procedural reconstruction process comprises:
ingesting the area of interest into a trained inference model to obtain the second image as output from the trained inference model.
14. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing generation of synthesized images, the operations comprising:
obtaining an area of interest associated with an image, the area of interest indicating a portion of a scene depicted by the image being relevant to a process performed by a downstream consumer, and the portion of the scene being one of multiple portions of the scene depicted by the image;
obtaining a synthesized environment in which the portion of the scene naturally resides from a perspective of the downstream consumer, the synthesized environment depicting a type of scene similar in contextual information to a type of the scene depicted by other portions of the multiple portions of the scene depicted by the image, the contextual information providing the downstream consumer with an understanding of an original version the image without the downstream consumer actually needing to have seen the original version;
obtaining a synthesized image of the synthesized images using the area of interest and the synthesized environment, the synthesized image depicting the area of interest in the synthesized environment to facilitate presentation of the area of interest in a common environment in which the area of interest would be expected to be found by the downstream consumer; and
providing the synthesized image to the downstream consumer.
15. The data processing system of claim 14 , wherein obtaining the synthesized environment comprises:
performing a procedural reconstruction process to obtain the synthesized environment using the area of interest, the synthesized environment comprising a second image about the area of interest.
16. The data processing system of claim 15 , wherein portions of the area of interest adjacent to the second image about the area of interest are matched to disguise a boundary between the area of interest and the second image.
17. The data processing system of claim 16 , wherein performing the procedural reconstruction process comprises:
ingesting the area of interest into a trained inference model to obtain the second image as output from the trained inference model.
18. The method of claim 1 , wherein the synthesized environment comprises a copy of the area of interest in an original form of the area of interest within the image and at an original position of the area of interest within the image, the area of interest being superimposed onto the copy of the area of interest when the area of interest and the synthesized environment are used to obtain the synthesized image.
19. The method of claim 1 , wherein an area within the synthesized environment that corresponds to an area at which the area of interest is positioned in the original image is constructed as a placeholder area without an image object, the area of interest being situated within the placeholder area when the area of interest and the synthesized environment are used to obtain the synthesized image.
20. The method of claim 1 , wherein a first amount of computing resources, associated with a data processing system, required to generate the synthesized environment is smaller than a second amount of computing resources, associated with the data processing system, required to store an original form of the image in its entirety in the data processing system.