Flap prediction system based on volumetric data and foundation models

1 . A method for an artificial intelligence (AI) based recommendation system associated with care or treatment of a patient, the method comprising:

receiving, via a chat-based user interface, a prompt requesting viable donor sites for flap harvesting;

capturing a plurality of images of the patient with a corresponding plurality of image capture devices, wherein each of the plurality of image capture devices is positioned at a different physical location with a different observation angle relative to the patient;

forming three-dimensional (3D) image data of the patient by fusion of at least a portion of the plurality of images in a 3D model, wherein the 3D image data of the patient includes volumetric data and the fusion aligns pixels of the portion of the plurality of images based on at least one of depth, color, or intensity;

detecting a plurality of anatomical treatment sites associated with the patient based on the 3D image data of the patient;

annotating the 3D image data of the patient with the plurality of anatomical treatment sites associated with the patient;

evaluating the annotated 3D image data of the patient with a foundation model to identify the viable donor sites, for the flap harvesting, amongst the plurality of anatomical treatment sites associated with the patient; and

displaying the recommendation based on the evaluating via the chat-based user interface, wherein the recommendation includes;

the annotated 3D image data of the patient marking one or more of the viable donor sites identified, by the foundation model, for the flap harvesting based on a treatment criteria associated with the patient; and

for each viable donor site of the one or more viable donor sites, a size of a flap to be harvested, a depth of the flap to be harvested, and a volume of the flap to be harvested.

2 . The method of claim 1 , further comprising receiving the treatment criteria as an input to the AI based recommendation system via the chat-based user interface, wherein the treatment criteria includes at least one of a functional objective, an aesthetic objective, or a risk factor objective.

3 . The method of claim 1 , wherein:

the prompt is at least one of text based, image based, or audio based; and

the recommendation is at least one of text based, image based, or audio based.

4 . The method of claim 3 , wherein receiving the prompt comprises receiving one or more of the treatment criteria for treatment of the patient, patient medical records, marked images of the patient, and context data associated with identification and recommendation of the viable donor sites.

5 . The method of claim 2 , further comprising identifying a tissue location of the patient that matches at least one of the functional objective, the aesthetic objective, or the risk factor objective.

6 . The method of claim 1 , further comprising identifying a tissue location of the patient with the foundation model based on at least one of the treatment criteria, the 3D image data, or context data associated with locations of the viable donor sites.

7 . The method of claim 1 , wherein annotating the 3D image data of the patient comprises adding preliminary recommendations to the 3D image data, where the preliminary recommendations include at least one of text, images, audio, or meta-data.

8 . The method of claim 1 , wherein the recommendation further includes tissue details.

9 . A computer-readable storage medium having computer-executable instructions stored thereupon that, when executed by one or more processing units of an artificial intelligence (AI) based recommendation system associated with care or treatment of a patient, cause the AI based recommendation system to:

receive, via a chat-based user interface, a prompt requesting viable donor sites for flap harvesting;

capture a plurality of images of the patient with a corresponding plurality of image capture devices, wherein each of the plurality of image capture devices is positioned at a different physical location with a different observation angle relative to the patient;

form three-dimensional (3D) image data of the patient by fusion of at least a portion of the plurality of images in a 3D model, wherein the 3D image data of the patient includes volumetric data and the fusion aligns pixels of the portion of the plurality of images based on at least one of depth, color, or intensity;

detect a plurality of anatomical treatment sites associated with the patient based on the 3D image data of the patient;

annotate the 3D image data of the patient with the plurality of anatomical treatment sites associated with the patient;

evaluate the annotated 3D image data of the patient with a foundation model to identify the viable donor sites, for flap harvesting, amongst the plurality of anatomical treatment sites associated with the patient; and

display the recommendation based on the evaluating via the chat-based user interface, wherein the recommendation includes:

the annotated 3D image data of the patient marking one or more of the viable donor sites identified, by the foundation model, for the flap harvesting based on a treatment criteria associated with the patient; and

for each viable donor site of the one or more viable donor sites, a size of a flap to be harvested, a depth of the flap to be harvested, and a volume of the flap to be harvested.

10 . The computer-readable storage medium of claim 9 , wherein the recommendation further includes tissue details.

11 . The computer-readable storage medium of claim 10 , wherein the computer-executable instructions stored thereupon, when executed by the one or more processing units of the AI based recommendation system, further cause the foundation model of AI based recommendation system to:

evaluate context data associated with least one of reference anatomy data, reference defect image data, reference donor image data, and reference recommendations; and

adjust the recommendation based on the context data.

12 . The computer-readable storage medium of claim 10 , wherein;

the prompt is at least one of text based, image based, or audio based; and

the recommendation is at least one of text based, image based, or audio based.

13 . An artificial intelligence (AI) based recommendation system associated with care or treatment of a patient, the AI based recommendation system comprising:

a processor; and

a computer-readable storage medium having computer-executable instructions stored thereupon that, when executed by the processor, cause the AI based recommendation system to:

receive, via a chat-based user interface, a prompt requesting viable donor sites for flap harvesting;

capture a plurality of images of the patient with a corresponding plurality of image capture devices, wherein each of the plurality of image capture devices is positioned at a different physical location with a different observation angle relative to the patient;

form three-dimensional (3D) image data of the patient by fusion of at least a portion of the plurality of images in a 3D model, wherein the 3D image data of the patient includes volumetric data and the fusion aligns pixels of the portion of the plurality of images based on at least one of depth, color, or intensity;

detect a plurality of anatomical treatment sites associated with the patient based on the 3D image data of the patient;

annotate the 3D image data of the patient with the plurality of anatomical treatment sites associated with the patient;

evaluate the annotated 3D image data of the patient with a foundation model to identify the viable donor sites, for the flap harvesting, amongst the plurality of anatomical treatment sites associated with the patient; and

display the recommendation based on the evaluating via the chat-based user interface, wherein the recommendation includes:

the annotated 3D image data of the patient marking one or more of the viable donor sites identified, by the foundation model, for the flap harvesting based on a treatment criteria associated with the patient; and

for each viable donor site of the one or more viable donor sites, a size of a flap to be harvested, a depth of the flap to be harvested, and a volume of the flap to be harvested.

14 . The AI based recommendation system of claim 13 , wherein the computer-executable instructions further cause the AI based recommendation system to receive the treatment criteria as an input to the AI based recommendation system via the chat-based user interface, wherein the treatment criteria includes at least one of a functional objective, an aesthetic objective, or a risk factor objective.

15 . The AI based recommendation system of claim 13 , wherein:

the prompt is at least one of text based, image based, or audio based; and

the recommendation is at least one of text based, image based, or audio based.

16 . The AI based recommendation system of claim 15 , wherein receiving the prompt comprises receiving the treatment criteria and patient medical records.