Methods and apparatuses for ultrasound data collection
Aspects of the technology described herein relate to determining, by a processing device in operative communication with an ultrasound device, a position and/or orientation of the ultrasound device relative to the processing device, and displaying on a display screen of the processing device, based on the position and/or orientation of the ultrasound device relative to the processing device, an instruction for moving the ultrasound device.
1. A method of operating an ultrasound system, the method comprising:
determining, by a processing device in operative communication with an ultrasound device, a horizontal position, a vertical position, and a depth of the ultrasound device relative to the processing device using one or more first statistical models and a video collected by a camera of the processing device which depicts the ultrasound device;
determining, using a sensor on the ultrasound device, a sensor on the processing device, one or more second statistical models, and the video collected by the camera of the processing device which depicts the ultrasound device, an orientation of the ultrasound device relative to the processing device; and
displaying on a display screen of the processing device, based on at least one selected from a group consisting of:
the horizontal position,
the vertical position,
the depth, and
the orientation of the ultrasound device relative to the processing device, an instruction for moving the ultrasound device.
2. The method of claim 1 , wherein determining the orientation of the ultrasound device using the sensor on the ultrasound device and the sensor on the processing device comprises using an accelerometer and gyroscope on the ultrasound device and an accelerometer and gyroscope on the processing device.
3. The method of claim 1 , wherein determining the orientation of the ultrasound device comprises:
determining, using the sensor on the ultrasound device and the sensor on the processing device, an orientation of the ultrasound device relative to the processing device except for determining an orientation of the ultrasound device around a direction gravity as seen from a reference frame of the processing device; and
determining, using the one or more second statistical models and the video collected by the camera of the processing device which depicts the ultrasound device, the orientation of the ultrasound device around the direction gravity as seen from the reference frame of the processing device.
4. The method of claim 1 , wherein determining the horizontal position and the vertical position of the ultrasound device relative to the processing device comprises locating a tip of the ultrasound device in the video collected by the processing device which depicts the ultrasound device.
5. The method of claim 1 , further comprising receiving, by the processing device, a selection of whether a subject being imaged is lying on his/her left side or sitting.
6. The method of claim 1 , wherein a subject being imaged is oriented in a default orientation relative to a direction of gravity.
7. The method of claim 1 , further comprising instructing, by the processing device, a user how to orient the ultrasound device in an initial orientation around a direction of gravity.
8. The method of claim 7 , further comprising:
tracking, by the processing device, a deviation of the ultrasound device from an initial orientation relative to a subject being imaged by tracking a deviation of the ultrasound device from the initial orientation around the direction of gravity; and
displaying a direction indicator corresponding to the instruction for moving the ultrasound device by compensating for the deviation such that the directional indicator is in the desired direction relative to the subject being imaged.
9. The method of claim 1 , wherein displaying the instruction for moving the ultrasound device comprises:
at least one selected from a group consisting of translating and rotating a directional indicator corresponding to the instruction in three-dimensional space based on at least one selected from a group consisting of:
the horizontal position,
the vertical position,
the depth, and
the orientation of the ultrasound device relative to the processing device; and
projecting that three-dimensional arrow into two-dimensional space for display on the display screen of the processing device.
10. A non-transitory computer readable medium comprising processor-executable instructions which, when executed, cause a processor on a processing device in operative communication with an ultrasound device to:
determine a horizontal position, a vertical position, and a depth of the ultrasound device relative to the processing device using one or more first statistical models and a video collected by a camera of the processing device which depicts the ultrasound device;
determine, using sensor data from a sensor on the ultrasound device, sensor data from a sensor on the processing device, one or more second statistical models, and the video collected by the camera of the processing device which depicts the ultrasound device, an orientation of the ultrasound device relative to the processing device; and
display on a display screen of the processing device, based on at least one selected from a group consisting of:
the horizontal position,
the vertical position,
the depth, and
the orientation of the ultrasound device relative to the processing device, an instruction for moving the ultrasound device.
11. The non-transitory computer readable medium of claim 10 , wherein the instructions which cause the processor to determine the orientation of the ultrasound device relative to the processing device using sensor data from a sensor on the ultrasound device, sensor data from a sensor on the processing device, one or more second statistical models, and the video collected by the camera of the processing device which depicts the ultrasound device, comprise instructions which, when executed, cause the processor to determine the orientation of the ultrasound device using data from an accelerometer and gyroscope on the ultrasound device and data from an accelerometer and gyroscope on the processing device.
12. The non-transitory computer readable medium of claim 10 , wherein the instructions which cause the processor to determine the orientation of the ultrasound device relative to the processing device using sensor data from a sensor on the ultrasound device, sensor data from a sensor on the processing device, one or more second statistical models, and the video collected by the camera of the processing device which depicts the ultrasound device, comprise instructions which, when executed, cause the processor to determine the orientation of the ultrasound device relative to the processing device by:
determining, using the sensor on the ultrasound device and the sensor on the processing device, an orientation of the ultrasound device relative to the processing device except for determining an orientation of the ultrasound device around a direction gravity as seen from a reference frame of the processing device; and
determining, using the one or more second statistical models and the video collected by the camera of the processing device which depicts the ultrasound device, the orientation of the ultrasound device around the direction gravity as seen from the reference frame of the processing device.
13. The non-transitory computer readable medium of claim 10 , wherein the instructions which cause the processor to determine the horizontal position and the vertical position of the ultrasound device relative to the processing device comprise instructions which, when executed, cause the processor to locate a tip of the ultrasound device in the video collected by the processing device which depicts the ultrasound device.
14. The non-transitory computer readable medium of claim 10 , wherein the instructions, when executed, further cause the processor to receive a selection of whether a subject being imaged is lying on his/her leftside or sitting.
15. The non-transitory computer readable medium of claim 10 , wherein the instructions, when executed, further cause the processor to determine whether a subject being imaged is oriented in a default orientation relative to a direction of gravity.
16. The non-transitory computer readable medium of claim 10 , wherein the instructions, when executed, further cause the processor to instruct a user how to orient the ultrasound device in an initial orientation around a direction of gravity.