IP Library Granted Patent US 11,690,602
Granted Patent B2
US 11,690,602 · App. 16/285,573 · Granted Jul 4, 2023

Methods and apparatus for tele-medicine

Inventors: Maxim Zaslavsky (San Diego, CA); Matthew de Jonge (New York, NY); Tomer Gafner (Forest Hills, NY); Eamon Duffy (New Haven, CT); Jonathan M. Rothberg (Guilford, CT)
Assignee: BFLY OPERATIONS, INC.
A61B8/54A61B8/4263A61B8/463A61B8/465A61B8/565A61B8/582G06F3/0482G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,690,602
App. No.
16/285,573
Granted
Jul 4, 2023
Kind
B2
Abstract

Aspects of the technology described herein relate to ultrasound data collection using tele-medicine. An instructor electronic device may generate for display an instructor augmented reality interface and receive, on the instructor augmented reality interface, an instruction for moving an ultrasound imaging device. The instructor augmented reality interface may include a video showing the ultrasound imaging device and a superposition of arrows on the video, where each of the arrows corresponds to a possible instruction for moving the ultrasound imaging device. A user electronic device may receive, from the instructor electronic device, an instruction for moving an ultrasound imaging device, and generate for display, on a user augmented reality interface shown on the user electronic device, the instruction for moving the ultrasound imaging device. The user augmented reality interface may include the video showing the ultrasound imaging device and an arrow superimposed on the video that corresponds to the instruction.

Claims (64)

1. A system, comprising:

an ultrasound imaging device;

a user electronic device comprising a camera and a display screen, the user electronic device physically separate from and electronically connectable to the ultrasound imaging device; and

an instructor electronic device including an instructor augmented reality interface, the instructor electronic device in communication with the user electronic device, wherein the instructor augmented reality interface includes (a) a video of the ultrasound image device captured by and communicated from the user electronic device and (b) a plurality of selectable arrows superimposed on the video, each of the plurality of selectable arrows corresponding to an instruction to translate, rotate, or tilt the ultrasound imaging device,

wherein the user electronic device is configured to:

receive, from the instructor electronic device, an instruction for moving the ultrasound imaging device;

display, on a user augmented reality interface included on the user electronic device, the instruction for moving the ultrasound imaging device;

receive, from the instructor electronic device, a selection to change an imaging parameter for the ultrasound imaging device;

based on the received selection to change the imaging parameter, generate a command to change the imaging parameter for the ultrasound imaging device;

transmit, to the ultrasound imaging device, the command to change the imaging parameter for the ultrasound imaging device;

receive, from the instructor electronic device, an instruction to freeze an ultrasound image on the display screen of the user electronic device; and

based on the received instruction to freeze the ultrasound image, cease updating the ultrasound image on the display screen of the user electronic device.

2. The system of claim 1 , wherein the user electronic device is further configured to:

receive a video captured by the camera, the video depicting the ultrasound imaging device; and wherein:

the user augmented reality interface comprises the video depicting the ultrasound imaging device.

3. The system of claim 2 , wherein the user electronic device is configured, when displaying the instruction on the user augmented reality interface, to display an arrow superimposed on the video showing the ultrasound imaging device.

4. The system of claim 3 , wherein the ultrasound imaging device comprises one or more fiducial markers.

5. The system of claim 4 , wherein the user electronic device is further configured to:

determine a first transformation from a default pose of the camera relative to the one or more fiducial markers to a first pose of the camera relative to the one or more fiducial markers at a first time; and

display the user augmented reality interface based on the first transformation.

6. The system of claim 5 , wherein the user electronic device is further configured to:

determine a second transformation from the default pose of the camera relative to the one or more fiducial markers to a second pose of the camera relative to the one or more fiducial markers at a second time after the first time, wherein the first transformation is different from the second transformation; and

display the user augmented reality interface based on the second transformation.

7. The system of claim 4 , wherein the user electronic device is further configured to display the arrow in a direction that appears in the user augmented reality interface to be normal to one of the one or more fiducial markers.

8. The system of claim 4 , wherein the user electronic device is further configured to display the arrow such that the arrow appears in the user augmented reality interface to form a circular path parallel to a plane of one of the one or more fiducial markers.

9. The system of claim 4 , wherein the user electronic device is further configured to display the arrow such that the arrow appears in the user augmented reality interface to form a circular path orthogonal to a plane of one of the one or more fiducial markers.

10. The system of claim 1 , wherein the instructor electronic device is operated by an instructor.

11. The system of claim 1 , wherein the user electronic device is further configured to:

receive an instruction to stop moving the ultrasound imaging device; and

display, on the user electronic device, the instruction to stop moving the ultrasound imaging device.

12. The system of claim 1 , wherein the user electronic device is further configured to:

receive an instruction to stop moving the ultrasound imaging device; and

cease to display, on the user augmented reality interface included on the user electronic device, the instruction for moving the ultrasound imaging device.

13. The system of claim 1 , wherein the user electronic device is further configured to:

receive, from the instructor electronic device, a selection to change an imaging preset; and

change the imaging preset.

14. The system of claim 1 , wherein the user electronic device is further configured to:

receive, from the instructor electronic device, a selection to change an imaging gain; and

change the imaging gain.

15. The system of claim 1 , wherein the user electronic device is further configured to:

receive, from the instructor electronic device, a selection to change an imaging depth; and

change the imaging depth.

16. The system of claim 1 , wherein the user electronic device is further configured to:

receive, from the instructor electronic device, a selection to save one or more ultrasound images to memory; and

save the one or more ultrasound images to memory.

17. The system of claim 1 , wherein the user electronic device is further configured to:

receive an instruction to move the ultrasound imaging device into a particular anatomical region; and

display the instruction to move the ultrasound imaging device into the particular anatomical region.

18. The system of claim 1 , wherein the user electronic device is further configured to:

receive, from the instructor electronic device, an instruction to instruct a subject to take and hold a deep breath; and

display the instruction to instruct the subject to take and hold a deep breath.

19. The system of claim 1 , wherein the user electronic device is further configured to:

receive an instruction to move the ultrasound imaging device into view of the camera; and

display the instruction to move the ultrasound imaging device into view of the camera.

20. The system of claim 1 , wherein the user electronic device is further configured to:

receive, from the instructor electronic device, an instruction to press the ultrasound imaging device harder onto a subject; and

display the instruction to press the ultrasound imaging device harder onto the subject.

21. The system of claim 1 , wherein the user electronic device is further configured to:

receive an instruction to move the ultrasound imaging device in at least one of shorter or smaller increments; and

display the instruction to move the ultrasound imaging device in at least one of shorter or smaller increments.

22. The system of claim 1 , wherein the user electronic device is further configured to:

transmit, to the instructor electronic device, ultrasound data collected by at least one of the ultrasound imaging device or an ultrasound image generated from the ultrasound data.

23. The system of claim 1 , wherein the selection to change the imaging parameter for the ultrasound imaging device comprises a selection to change at least one of an imaging preset, an imaging gain, or an imaging depth for the ultrasound imaging device.

24. The system of claim 1 , wherein the user electronic device is a mobile smartphone, the mobile smartphone comprises the camera, and the wire is flexible, configured to mate with the user electronic device and configured to facilitate electronic communication between the user electronic device and the ultrasound imaging device.

Assignments (2)
CHANGE OF NAME Recorded Dec 28, 2021
From: BUTTERFLY NETWORK, INC.
To: BFLY OPERATIONS, INC.
Reel/Frame 058597/0750 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2019
From: ZASLAVSKY, MAXIM; DE JONGE, MATTHEW; GAFNER, TOMER; DUFFY, EAMON; ROTHBERG, JONATHAN M.
To: BUTTERFLY NETWORK, INC.
Reel/Frame 048445/0277 →
Continuity (3)
Provisional Application 62646333 · Mar 21, 2018
Provisional Application 62636127 · Feb 27, 2018
Related Publication 20190261957A1 · Aug 29, 2019