Granted Patent B2

US 12,505,905 · App. 18/952,233 · Granted Dec 23, 2025

Inventors: Jeffrey Chang (Berkeley, CA); Doktor Gurson (Berkeley, CA); Joseph Zachary Allen (Berkeley, CA); Christopher Johnson (Berkeley, CA)

Assignee: RAD AI, Inc.

G16H15/00G06F40/40G06T7/0012G06V10/77G06T2207/10116G06V2201/03

Abstract

Claims (33)

1. A method of using a multimodal model for generation of radiology outputs, the method comprising:

at a computing system, receiving a request from a radiologist to retrieve a report associated with a session with a patient; and

returning the report with a level of completion above a threshold level of completion, wherein the report is returned in a writing style of the radiologist, wherein returning the report comprises:

training the multimodal model, comprising a language-aligned vision encoder and an adapter between the language-aligned vision encoder and a fixed large language model (LLM), wherein the adapter is structured to pass information between the language-aligned image encoder and the LLM with an attention mechanism, to generate a trained multimodal model;

re-training the multimodal model to generate a re-trained multimodal model, wherein re-training the multimodal model comprises training only the adapter and fixing all other components of the multimodal model;

transforming a set of images generated during the session into a set of image representations;

returning a set of radiology outputs upon processing the set of image representations with the re-trained multimodal model, wherein the multimodal model is structured to process image-based inputs and language-based inputs;

transforming the set of radiology outputs into the report; and

transmitting the report to the radiologist.

2. The method of claim 1 , wherein training the multimodal model comprises training the multimodal model in a first stage of training and in a second stage of training.

3. The method of claim 2 , wherein the first stage of training involves a contrastive learning with language-image pre-training operation, with a contrastive loss function, structured to drive image representations and associated reports closer in a high-dimension space, and structured to drive mismatched image representations and unrelated reports, to generate a first trained representation of the multimodal model.

4. The method of claim 3 , wherein the second stage of training involves training the first trained representation of the multimodal model with bootstrapping language-image pre-training architecture applied to a Q-former comprising a shared embedding space.

5. The method of claim 1 , wherein returning the set of radiology outputs comprises detecting an anomaly captured in the set of images.

6. The method of claim 5 , further comprising determining that the anomaly is associated with a clinical indication, wherein the anomaly comprises one of a global anomaly and a local anomaly.

7. The method of claim 6 , further comprising retrieving a set of candidate actions to perform based upon the clinical indication.

8. The method of claim 7 , wherein the clinical indication comprises a hemorrhage, and wherein detecting the anomaly is performed with at least 70% sensitivity and 80% specificity.

9. The method of claim 7 , further comprising executing an action of the set of candidate actions, wherein the action comprises executing a critical results workflow with administration of care for treating the clinical indication.

10. The method of claim 1 , wherein the writing style comprises a word choice component and a length of an impressions section of the report.

11. The method of claim 1 , wherein receiving the request and returning the report is performed within a duration of 5 minutes, and wherein the threshold level of completion is 80%.

12. A method of using a multimodal model for generation of radiology outputs, the method comprising:

training the multimodal model, comprising a language-aligned vision encoder and an adapter between the language-aligned vision encoder and a fixed large language model (LLM), wherein the adapter is structured to pass information between the language-aligned image encoder and the LLM with an attention mechanism, to generate a trained multimodal model;

re-training the multimodal model to generate a re-trained multimodal model, wherein re-training the multimodal model comprises training only the adapter and fixing all other components of the multimodal model;

transforming a set of images generated during a session with a patient, into a set of image representations;

returning a set of radiology outputs upon processing the set of image representations with the re-trained multimodal model, wherein the multimodal model is structured to process image-based inputs and language-based inputs;

detecting an anomaly captured in the set of images, using the trained multimodal model;

determining that the anomaly is associated with a clinical indication;

retrieving a set of candidate actions to perform based upon the clinical indication;

transforming the set of radiology outputs into a report comprising a style and level of completion above a threshold level of completion;

transmitting the report to the radiologist; and

executing an action of the set of candidate actions, wherein the action comprises administering care according to a critical results workflow corresponding to the clinical indication.

13. The method of claim 12 , wherein training the multimodal model involves a contrastive learning with language-image pre-training operation, with a contrastive loss function, to generate a first trained representation of the multimodal model.

14. The method of claim 12 , wherein the style comprises a word choice component and a length of an impressions section of the report.

15. The method of claim 12 , wherein transmitting the report in response to a request by a radiologist is performed within a duration of 2 minutes, and wherein the threshold level of completion is 90%.

Assignments (1)

Continuity (2)

References Cited (108)