Artificial intelligence system for determining drug use through medical imaging
Systems and methods for establishing a patient's pharmaceutical use are provided. A neural network is trained on a dataset of medical images, such as ultrasound images, that are tagged with information concerning the use of the pharmaceutical by people who were imaged to produce the medical images. The trained neural network can then be provided with medical images of a patient, and the neural network can then make a determination as to the patient's pharmaceutical use.
1 . A system configured to make a medical determination, the system comprising:
a medical image generator configured to acquire non-invasive medical images of a body part of a patient during an imaging session, the medical image generator further configured to adapt generation of the medical images based on feedback received from image analysis logic by fine-tuning at least one of (i) image generation, (ii) focus, or (iii) image processing parameters;
an image storage storing a set of the medical images of the body part of a the patient, the set of medical images having been acquired during the imaging session;
image analysis logic comprising a trained neural network in communication with the image storage and configured to:
(i) process a plurality of the medical images of the set to provide a plurality of image-level determinations indicative of a quantitative estimate of an exposure of the patient to a pharmaceutical,
(ii) generate an aggregate determination regarding the quantitative estimate by aggregating the image-level determinations, wherein the aggregating comprises excluding at least one anatomical view or other information that is determined to be commonly uninformative and/or weighting a plurality of anatomical views in calculating the aggregate determination, and
(iii) determine a confidence score associated with the aggregate determination;
feedback logic configured, based on the confidence score and/or a classification of medical images already acquired during the imaging session, to guide acquisition of at least one additional medical image during the imaging session via a user interface;
the user interface configured to provide at least the aggregate determination and guidance output by the feedback logic to a user; and
a microprocessor configured to execute at least the image analysis logic and the feedback logic.
2 . The system of claim 1 wherein the image analysis logic includes first logic configured to analyze the set of medical images to determine a present exposure to the pharmaceutical at the time the set of medical images were acquired.
3 . The system of claim 1 wherein the image analysis logic includes second logic configured to determine a timeframe for a prior exposure to the pharmaceutical.
4 . The system of claim 1 wherein the image analysis logic is configured to employ a regression algorithm to provide the quantitative estimate as a range.
5 . The system of claim 1 wherein the image analysis logic is configured to employ a classification algorithm to provide the quantitative estimate as one of a plurality of categories.
6 . The system of claim 1 , wherein the quantitative estimate comprises an estimated dosage range of the pharmaceutical expressed in dosage units.
7 . The system of claim 1 , wherein the quantitative estimate comprises an estimate of a time period during which the pharmaceutical was used by the patient.
8 . The system of claim 1 wherein the quantitative estimate comprises an estimated dosage expressed as a measure of the pharmaceutical over a time duration.
9 . The system of claim 1 wherein the quantitative estimate is further based on a clinical data set.
10 . The system of claim 1 , wherein image analysis logic is further configured to determine the confidence score satisfies a threshold condition and, in response, identifying the at least one additional medical image.
11 . The system of claim 1 , wherein the feedback logic is further configured to guide acquisition of the at least one additional medical image by at least one of: (i) a recommended anatomical view, (ii) a recommended imaging plane, (iii) a recommended probe position, (iv) a recommended timepoint, or (v) a recommended acquisition setting.
12 . The system of claim 1 , wherein the non-invasive medical images comprise images generated by at least one of ultrasound imaging, magnetic resonance imaging, computed tomography imaging, positron emission tomography imaging, x ray imaging, or optical imaging.
13 . The system of claim 1 , wherein processing the plurality of the medical images further comprises generating a normalized image tensor comprising at least one of: image normalization, image registration, or image resampling into a standardized resolution.
14 . The system of claim 1 , wherein the feedback logic is further configured to generate a region-of-interest representation that identifies a target anatomical structure.
15 . The system of claim 1 , wherein the trained neural network comprises at least one of a convolutional neural network, a transformer network, or an ensemble of neural networks.
16 . The system of claim 1 , wherein the confidence score comprises at least one of a predictive variance, a confidence interval, or an entropy measure.
17 . The system of claim 1 , wherein the image analysis logic is further configured to receive at least one additional medical image identified by the feedback logic, and to compute an updated quantitative estimate using the at least one additional medical image.
18 . The system of claim 1 , wherein the trained neural network was trained using supervised learning on labeled training examples that associate non-invasive medical images with corresponding reference exposure values of the pharmaceutical.
19 . A method for training a neural network to make a medical determination, the method comprising:
receiving a set of non-invasive medical images of body parts of a plurality of patients, the medical images tagged with information concerning exposure by the respective patients to a pharmaceutical;
classifying the medical images according to views or features included within the medical images;
filtering the medical images to remove images that lack features or views that have been determined to have little or no determinative value and/or to remove medical images filtered according to quality or resolution;
dividing the set of medical images into a training set and a validation set;
providing the medical images of the training set, and their tags, to a neural network to train the neural network to determine a quantitative exposure of the plurality of patients to the pharmaceutical based on images of the body parts; and
providing medical images of a single patient from the validation set to the neural network to make a determination, and comparing the determination to the tags associated with the medical images.
20 . The method of claim 19 wherein the non-invasive medical images are ultrasound images.
21 . The method of claim 19 further comprising classifying the medical images before providing the medical images of the training set to the neural network.
22 . The method of claim 19 further comprising resizing the medical images before providing the medical images of the training set to the neural network.
23 . A method for making a medical determination, the method comprising:
during a medical imaging session, generating, using an image generator comprising a source and a detector, a set of non-invasive medical images of a body part of a patient;
storing the set of medical images;
providing a plurality of medical images of the set to a neural network that has been trained to determine, from medical images of the body part, a quantitative estimate of an exposure of the patient to a pharmaceutical;
receiving, from the neural network, a plurality of image-level determinations and a confidence score associated with the determinations;
generating an aggregate determination regarding the quantitative estimate by aggregating the image-level determinations, wherein the aggregating comprises excluding at least one anatomical view or other information that is determined to be commonly uninformative and/or weighting a plurality of anatomical views in calculating the aggregate determination;
when the confidence score indicates inadequate precision, accuracy, and/or reliability, providing feedback during the imaging session via a user interface to guide acquisition of at least one additional medical image and/or to adapt generation of medical images by fine-tuning at least one of (i) image generation, (ii) focus, or (iii) image processing parameters; and
providing the aggregate determination regarding the quantitative estimate.
24 . The method of claim 23 wherein the non-invasive medical images are ultrasound images.
25 . The method of claim 23 wherein the neural network employs a regression algorithm and receiving the plurality of image-level determinations further includes receiving one or more ranges.
26 . The method of claim 23 wherein the neural network employs a classification algorithm and receiving the plurality of image-level determinations further includes receiving one of a plurality of categories.
27 . The method of claim 23 wherein the quantitative estimate of exposure to the pharmaceutical further comprises an estimate of an effective dosage of the pharmaceutical.
28 . The method of claim 23 wherein the quantitative estimate of exposure to the pharmaceutical further comprises a time the pharmaceutical should be used by the patient.
29 . The method of claim 23 , wherein the quantitative estimate comprises an estimated dosage range of the pharmaceutical expressed in dosage units.
30 . The method of claim 23 , wherein the quantitative estimate comprises an estimate of a time period during which the pharmaceutical was used by the patient.
31 . The method of claim 23 wherein the quantitative estimate comprises an estimated dosage expressed as a measure of the pharmaceutical over a time duration.
32 . The method of claim 23 wherein the quantitative estimate is further based on a clinical data set.