Methods and apparatuses for generating anatomical models using diagnostic images
A diagnostic imaging process and system may operate to generate a three-dimensional anatomical model based on monocular color endoscopic images. In one example, an apparatus may include a processor and a memory coupled to processor. The memory may include instructions that, when executed by the processor, may cause the processor to access a plurality of endoscopic training images comprising a plurality of synthetic images and a plurality of real images, access a plurality of depth ground truths associated with the plurality of synthetic images, perform supervised training of at least one computational model using the plurality of synthetic images and the plurality of depth ground truths to generate a synthetic encoder, and perform domain adversarial training on the synthetic encoder using the real images to generate a real image encoder for the at least one computational model. Other embodiments are described.
1 . An apparatus, comprising:
at least one processor;
a memory coupled to the at least one processor, the memory comprising instructions that, when executed by the at least one processor, cause the at least one processor to:
access a plurality of endoscopic training images comprising a plurality of synthetic images and a plurality of real images, access a plurality of depth ground truths associated with the plurality of synthetic images, perform supervised training of at least one computational model using the plurality of synthetic images and the plurality of depth ground truths to generate a synthetic encoder and synthetic decoder in a first training process, and
perform domain adversarial training on the synthetic encoder using the plurality of real images to generate a real image encoder for the at least one computational model in a second training phase subsequent to the first training phase, wherein the real image encoder is generated as a separate encoder distinct from the synthetic encoder.
2 . The apparatus of claim 1 , the instructions, when executed by the at least one processor, to cause the at least one processor to perform an inference process on the plurality of real images using the real image encoder and the synthetic decoder to generate depth images and confidence maps.
3 . The apparatus of claim 1 , the real image encoder comprising at least one coordinate convolution layer.
4 . The apparatus of claim 1 , the plurality of endoscopic training images comprising bronchoscopic images.
5 . The apparatus of claim 4 , the plurality of endoscopic training images comprising images generated via bronchoscope imaging of a phantom device.
6 . The apparatus of claim 1 , the instructions, when executed by the at least one processor, to cause the at least one processor to:
provide a patient image as input to the trained computational model, generate at least one anatomical model corresponding to the patient image.
7 . The apparatus of claim 6 , the instructions, when executed by the at least one processor, to cause the at least one processor to generate a depth image and a confidence map for the patient image.
8 . The apparatus of claim 6 , the instructions, when executed by the at least one processor, to cause the at least one processor to present the at least one anatomical model on a display device to facilitate navigation of an endoscopic device.
9 . A computer-implemented method, comprising, via at least one processor of a computing device:
accessing a plurality of endoscopic training images comprising a plurality of synthetic images and a plurality of real images;
accessing a plurality of depth ground truths associated with the plurality of synthetic images;
performing supervised training of at least one computational model using the plurality of synthetic images and the plurality of depth ground truths to generate a synthetic encoder and synthetic decoder in a first training process; and
performing domain adversarial training on the synthetic encoder using the plurality of real images to generate a real image encoder for the at least one computational model in a second training process subsequent to the first training process, wherein the real image encoder is generated as a separate encoder distinct from the synthetic encoder.
10 . The computer-implemented method of claim 9 , comprising performing an inference process on the plurality of real images using the real image encoder and the synthetic decoder to generate depth images and confidence maps.
11 . The computer-implemented method of claim 10 , the real image encoder comprising at least one coordinate convolution layer.
12 . The computer-implemented method of claim 9 , the plurality of endoscopic training images comprising bronchoscopic images.
13 . The computer-implemented method of claim 12 , the plurality of endoscopic training images comprising images generated via bronchoscope imaging of a phantom device.
14 . The computer-implemented method of claim 9 , comprising:
providing a patient image as input to the trained computational model, generating at least one anatomical model corresponding to the patient image.
15 . The computer-implemented method of claim 14 , comprising generating a depth image and a confidence map for the patient image.
16 . The computer-implemented method of claim 15 , comprising presenting the at least one anatomical model on a display device to facilitate navigation of an endoscopic device.
17 . The computer-implemented method of claim 16 , comprising performing an examination of a portion of a patient represented by the at least one anatomical model using the endoscopic device.
18 . A diagnostic imaging system, comprising:
an endoscope;
a computing device operatively coupled to the endoscope, the computing device comprising:
at least one processor;
a memory coupled to the at least one processor, the memory comprising instructions that, when executed by the at least one processor, cause the at least one processor to:
access a plurality of endoscopic training images comprising a plurality of synthetic images and a plurality of real images,
access a plurality of depth ground truths associated with the plurality of synthetic images, perform supervised training of at least one computational model using the plurality of synthetic images and the plurality of depth ground truths to generate a synthetic encoder and synthetic decoder in a first training process, perform domain adversarial training on the synthetic encoder using the plurality of real images to generate a real image encoder for the at least one computational model in a second training process subsequent to the first training process, wherein the real image encoder is generated as a separate encoder distinct from the synthetic encoder.
19 . The diagnostic imaging system of claim 18 , the instructions, when executed by the at least one processor, to cause the at least one processor to:
provide a patient image as input to the trained computational model, the patient image captured via the endoscope, generate at least one anatomical model corresponding to the patient image.
20 . The diagnostic imaging system of claim 19 , the instructions, when executed by the at least one processor, to cause the at least one processor to present the anatomical model on a display device to facilitate navigation of the endoscopic device within a portion of the patient represented by the at least one anatomical model.