Automated image acquisition for assisting a user to operate an ultrasound device
Aspects of the technology described herein relate to techniques for guiding an operator to use an ultrasound device. Thereby, operators with little or no experience operating ultrasound devices may capture medically relevant ultrasound images and/or interpret the contents of the obtained ultrasound images. For example, some of the techniques disclosed herein may be used to identify a particular anatomical view of a subject to image with an ultrasound device, guide an operator of the ultrasound device to capture an ultrasound image of the subject that contains the particular anatomical view, and/or analyze the captured ultrasound image to identify medical information about the subject.
1. An apparatus, comprising:
at least one processor configured to:
obtain an ultrasound image of a subject captured by an ultrasound device and determine, using an automated image processing technique, whether the ultrasound image contains a target anatomical view;
generate a first guidance plan in furtherance of capturing a second ultrasound image of the subject that contains the target anatomical view responsive to a determination that the ultrasound image does not contain the target anatomical view;
provide at least one instruction to an operator of the ultrasound device indicating how to reposition the ultrasound device as part of the first guidance plan;
determine that an action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan; and
generate a second guidance plan responsive to determining that the action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan.
2. The apparatus of claim 1 , wherein the at least one processor is configured to determine whether the ultrasound image contains the target anatomical view at least in part by analyzing the ultrasound image using a deep learning technique.
3. The apparatus of claim 2 , wherein the at least one processor is configured to determine whether the ultrasound image contains the target anatomical view at least in part by providing the ultrasound image as an input to a multi-layer neural network.
4. The apparatus of claim 3 , wherein the at least one processor is configured to determine whether the ultrasound image contains the target anatomical view at least in part by using the multi-layer neural network to obtain an output that is indicative of an anatomical view contained in the ultrasound image.
5. The apparatus of claim 2 , wherein the at least one processor is configured to determine whether the ultrasound image contains the target anatomical view at least in part by analyzing the ultrasound image using a multi-layer neural network comprising at least one layer selected from the group consisting of: a pooling layer, a rectified linear units (ReLU) layer, a convolution layer, a dense layer, a pad layer, a concatenate layer, and an upscale layer.
6. The apparatus of claim 1 , wherein the at least one processor is configured to determine whether the ultrasound image contains the target anatomical view at least in part by:
identifying an anatomical view contained in the ultrasound image using the automated image processing technique; and
determining whether the anatomical view contained in the ultrasound image matches the target anatomical view.
7. The apparatus of claim 6 , wherein the at least one processor is configured to, responsive to a determination that the anatomical view contained in the ultrasound image does not match the target anatomical view, generate the at least one instruction.
8. The apparatus of claim 1 , wherein the at least one processor is configured to:
provide an indication to the operator that the ultrasound device is properly positioned responsive to a determination that the ultrasound image contains the target anatomical view.
9. The apparatus of claim 8 , further comprising a display coupled to the at least one processor and configured to display the at least one instruction to the operator.
10. The apparatus of claim 8 , wherein the at least one processor is configured to provide the at least one instruction at least in part by providing an instruction to move the ultrasound device in a translational direction and/or a rotational direction.
11. The apparatus of claim 8 , wherein the at least one processor is configured to provide the at least one instruction to the operator at least in part by providing the at least one instruction to the subject.
12. A method, comprising:
using at least one computing device comprising at least one processor to perform:
obtaining an ultrasound image of a subject captured by an ultrasound device;
determining, using an automated image processing technique, whether the ultrasound image contains a target anatomical view;
responsive to determining that the ultrasound image does not contain the target anatomical view, generating a first guidance plan in furtherance of capturing a second ultrasound image of the subject that contains the target anatomical view;
providing at least one instruction to an operator of the ultrasound device indicating how to reposition the ultrasound device as part of the first guidance plan;
determining that an action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan;
generating a second guidance plan responsive to determining that the action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan; and
responsive to determining that the ultrasound image contains the target anatomical view, providing an indication to the operator that the ultrasound device is properly positioned.
13. The method of claim 12 , wherein determining whether the ultrasound image contains the target anatomical view comprises analyzing the ultrasound image using a deep learning technique.
14. The method of claim 13 , wherein determining whether the ultrasound image contains the target anatomical view comprises providing the ultrasound image as an input to a multi-layer neural network.
15. The method of claim 14 , wherein determining whether the ultrasound image contains the target anatomical view comprises using the multi-layer neural network to obtain an output that is indicative of an anatomical view contained in the ultrasound image.
16. The method of claim 13 , wherein determining whether the ultrasound image contains the target anatomical view comprises analyzing the ultrasound image using a multi-layer neural network comprising at least one layer selected from the group consisting of: a pooling layer, a rectified linear units (ReLU) layer, a convolution layer, a dense layer, a pad layer, a concatenate layer, and an upscale layer.
17. The method of claim 12 , wherein determining whether the ultrasound image contains the target anatomical view comprises:
identifying an anatomical view contained in the ultrasound image using the automated image processing technique; and
determining whether the anatomical view contained in the ultrasound image matches the target anatomical view.
18. The method of claim 17 , further comprising:
responsive to determining that the anatomical view contained in the ultrasound image does not match the target anatomical view, generating the at least one instruction using the anatomical view contained in the ultrasound image.
19. The method of claim 12 , wherein providing the at least one instruction comprises providing an instruction to move the ultrasound device in a translational direction and/or a rotational direction.
20. The method of claim 12 , wherein providing the at least one instruction to the operator comprises providing the at least one instruction to the subject.
21. A system, comprising:
an ultrasound device configured to capture an ultrasound image of a subject; and
a computing device communicatively coupled to the ultrasound device and configured to:
obtain the ultrasound image of the subject captured by the ultrasound device;
determine, using an automated image processing technique, whether the ultrasound image contains a target anatomical view;
responsive to a determination that the ultrasound image does not contain the target anatomical view, generate a first guidance plan in furtherance of capturing a second ultrasound image of the subject that contains the target anatomical view;
provide at least one instruction to an operator of the ultrasound device indicating how to reposition the ultrasound device to capture the second ultrasound image of the subject that contains the target anatomical view as part of the first guidance plan;
determine that an action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan;
generate a second guidance plan responsive to determining that the action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan; and
responsive to a determination that the ultrasound image contains the target anatomical view, provide an indication to the operator that the ultrasound device is properly positioned.
22. The system of claim 21 , wherein the ultrasound device comprises a plurality of ultrasonic transducers.
23. The system of claim 22 , wherein the plurality of ultrasonic transducers comprises an ultrasonic transducer selected from the group consisting of: a capacitive micromachined ultrasonic transducer (CMUT), a CMOS ultrasonic transducer (CUT), and a piezoelectric micromachined ultrasonic transducer (PMUT).
24. The system of claim 21 , wherein the computing device is a mobile smartphone or a tablet.
25. The system of claim 21 , wherein the computing device is configured to determine whether the ultrasound image contains the target anatomical view at least in part by analyzing the ultrasound image using a deep learning technique.
26. The system of claim 25 , wherein the computing device is configured to determine whether the ultrasound image contains the target anatomical view at least in part by providing the ultrasound image as an input to a multi-layer neural network.
27. The system of claim 26 , wherein the computing device is configured to determine whether the ultrasound image contains the target anatomical view at least in part by using the multi-layer neural network to obtain an output that is indicative of an anatomical view contained in the ultrasound image.
28. The system of claim 21 , wherein the computing device is configured to determine whether the ultrasound image contains the target anatomical at least in part by:
identifying an anatomical view contained in the ultrasound image using the automated image processing technique; and
determining whether the anatomical view contained in the ultrasound image matches the target anatomical view.
29. The system of claim 28 , wherein the computing device is configured to generate the at least one instruction using the anatomical view contained in the ultrasound image responsive to a determination that the anatomical view contained in the ultrasound image does not match the target anatomical view.
30. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to:
obtain an ultrasound image of a subject captured by an ultrasound device;
determine, using an automated image processing technique, whether the ultrasound image contains a target anatomical view;
responsive to a determination that the ultrasound image does not contain the target anatomical view, generate a first guidance plan in furtherance of capturing a second ultrasound image of the subject that contains the target anatomical view;
provide at least one instruction to an operator of the ultrasound device indicating how to reposition the ultrasound device as part of the first guidance plan;
determine that an action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan;
generate a second guidance plan responsive to determining that the action taken by the operator of the ultrasound device does not match the at least one instruction provided to the operator of the ultrasound device as part of the first guidance plan; and
responsive to a determination that the ultrasound image contains the target anatomical view, provide an indication to the operator that the ultrasound device is properly positioned.