IP Library Granted Patent US 12,551,185
Granted Patent B2
US 12,551,185 · App. 18/795,036 · Granted Feb 17, 2026

On-screen markers for out-of-plane needle guidance

Inventors: Wendy Swan (Lake Forest Park, WA); Keith Williams (Seattle, WA); Andrew Lundberg (Woodinville, WA)
Assignee: FUJIFILM SonoSite, Inc.
A61B8/0841A61B8/12A61B34/20A61B2034/2063
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,551,185
App. No.
18/795,036
Granted
Feb 17, 2026
Kind
B2
Abstract

An ultrasound imaging system including an image processor configured to receive input data for capturing ultrasound images of a region of interest. The ultrasound images are taken along a first plane; and the input data further includes an indication that a needle will be inserted into the region of interest. The system can capture a plurality of ultrasound images of the region of interest along the first plane. The system can determine one or more high-confidence areas of the region of interest where the needle will intersect the first plane. Each high-confidence area is based on a probability that the needle will intersect the first plane at any portion of the high-confidence area; and display one or more on-screen markers corresponding to the one or more high-confidence areas in conjunction with the plurality of ultrasound images on the display.

Claims (75)

1 . An ultrasound imaging system comprising an image processor configured to:

receive input data for capturing ultrasound images of a region of interest along a first plane, the input data including an indication that an interventional instrument will be inserted along a second plane into the region of interest, the second plane being orthogonal to the first plane;

capture the ultrasound images sequentially along the first plane;

calculate, for a respective particular area of a plurality of particular areas, a probability that the interventional instrument, when inserted along the second plane, will intersect the first plane at the respective particular area;

determine one or more high-confidence areas within the region of interest where the interventional instrument will intersect the first plane based on the probability being above a threshold;

display at least one ultrasound image of the ultrasound images and one or more on-screen markers corresponding to the one or more high-confidence areas;

detect that the interventional instrument has intersected the first plane;

provide an indication on the at least one displayed ultrasound image of a location of the interventional instrument within the first plane; and

reduce a visibility of the one or more on-screen markers when the interventional instrument is detected to have intersected the first plane.

2 . The ultrasound imaging system of claim 1 , wherein the visibility of the one or more on-screen markers is automatically reduced when the interventional instrument is detected to have intersected the first plane.

3 . The ultrasound imaging system of claim 1 , wherein the visibility of the one or more on-screen markers is reduced by an operator of the ultrasound imaging system when the interventional instrument is detected to have intersected the first plane.

4 . The ultrasound imaging system of claim 1 , wherein the image processor is configured to increase a transparency of the one or more on-screen markers to reduce the visibility of the one or more on-screen markers.

5 . The ultrasound imaging system of claim 1 , wherein the image processor is configured to present the one or more on-screen markets in grey scale to reduce the visibility of the one or more on-screen markers.

6 . The ultrasound imaging system of claim 1 , wherein the determination of the one or more high-confidence areas is further based on calculating a potential deflection of the interventional instrument during insertion based at least on one of a density of tissue or physical properties of the interventional instrument.

7 . The ultrasound imaging system of claim 1 , wherein:

the ultrasound images along the first plane comprise an elevation thickness that varies as a function of depth; and

the determination of the one or more high-confidence areas is further based on the elevation thickness at each high-confidence area within the ultrasound images.

8 . The ultrasound imaging system of claim 1 , wherein the image processor is further configured to:

receive input data indicating a target within the region of interest;

determine whether a high-confidence area overlaps the target; and

notify an operator of the ultrasound imaging system whether the high-confidence area overlaps the target.

9 . The ultrasound imaging system of claim 8 , wherein the image processor is further configured to:

determine that the interventional instrument is being inserted into the region of interest along the second plane;

determine whether the high-confidence area remains overlapping with the target; and

in response to determining that the high-confidence area no longer overlaps the target, notify the operator that the high-confidence area does not overlap the target.

10 . The ultrasound imaging system of claim 1 , wherein the image processor is further configured to:

receive a selection of at least one selected on-screen markers from the one or more on-screen markers; and

display the at least one selected on-screen marker in a picture-in-picture on the display.

11 . The ultrasound imaging system of claim 1 , wherein the image processor is further configured to:

receive input data indicating a target within the region of interest;

prior to the interventional instrument being inserted, determine whether at least one of the one or more high-confidence areas overlaps the target;

notify an operator of the ultrasound imaging system that none of the one or more high-confidence areas overlaps the target; and

inform the operator that the operator should at least one of:

shift a position of a patient;

shift a position of an ultrasound transducer of the ultrasound imaging system;

use a different interventional instrument; or

use a different guide for the interventional instrument.

12 . One or more computer-readable non-transitory storage media embodying software operable when executed by one or more computing devices of an ultrasound imaging system to perform one or more operations comprising:

receiving input data for capturing ultrasound images of a region of interest along a first plane, the input data including an indication that an interventional instrument will be inserted along a second plane into the region of interest, the second plane being orthogonal to the first plane;

capturing the ultrasound images sequentially along the first plane;

calculating, for a respective particular area of a plurality of particular areas, a probability that the interventional instrument will intersect the respective particular area;

determining one or more high-confidence areas of the region of interest where the interventional instrument will intersect the first plane based on the probability being above a threshold;

displaying at least one ultrasound image of the ultrasound images and one or more on-screen markers corresponding to the one or more high-confidence areas;

detecting that the interventional instrument has intersected the first plane;

providing an indication on the at least one displayed ultrasound image of a location of the interventional instrument within the first plane; and

automatically reducing a visibility of the one or more on-screen markers when the interventional instrument is detected to have intersected the first plane.

13 . The storage media of claim 12 , wherein the interventional instrument is inserted along the second plane via a guidance system, the guidance system being affixed to an ultrasound transducer of the ultrasound imaging system.

14 . The storage media of claim 12 , wherein each of the one or more high-confidence areas corresponds respectively to a particular depth within the region of interest from an ultrasound transducer of the ultrasound imaging system.

15 . The storage media of claim 12 , wherein the determining of the one or more high-confidence areas is further based on calculating a potential deflection of the interventional instrument during insertion based at least on a density of tissue or a physical property of the interventional instrument.

16 . The storage media of claim 12 wherein:

the ultrasound images along the first plane comprise an elevation thickness that varies as a function of depth; and

the determining of the one or more high-confidence areas is further based on the elevation thickness at each high-confidence area within the ultrasound images.

17 . The storage media of claim 12 , wherein the operations further comprise:

receiving input data indicating a target within the region of interest;

prior to the interventional instrument being inserted, determining whether at least one of the one or more high-confidence areas overlaps the target;

notifying an operator of the ultrasound imaging system that none of the one or more high-confidence areas overlaps the target; and

informing the operator that the operator should perform at least one of:

shifting a position of a patient;

shifting a position of an ultrasound transducer of the ultrasound imaging system;

using a different interventional instrument; or

using a different guide for the interventional instrument.

18 . A method comprising:

receiving input data for capturing ultrasound images of a region of interest along a first plane, the input data including an indication that an interventional instrument will be inserted along a second plane into the region of interest, the second plane being orthogonal to the first plane;

capturing the ultrasound images sequentially along the first plane;

calculating, based on the input data and the ultrasound images, a probability that the interventional instrument will intersect the first plane at a respective particular area of a plurality of particular areas;

determining one or more high-confidence areas of the region of interest where the interventional instrument will intersect the first plane based on the probability being above a threshold;

displaying at least one ultrasound image of the ultrasound images and one or more on-screen markers corresponding to the one or more high-confidence areas;

dynamically changing the one or more on-screen markers as the interventional instrument moves closer to the first plane;

detecting that the interventional instrument has intersected the first plane; and

providing an indication on the at least one displayed ultrasound image of a location of the interventional instrument within the first plane.

19 . The method of claim 18 , wherein dynamically changing the one or more on-screen markers comprises:

changing a size of the one or more on-screen markers;

changing a shape of the one or more on-screen markers; or

changing a color of the one or more on-screen markers.

20 . The method of claim 18 , wherein each of the one or more high-confidence areas corresponds respectively to a particular depth within the region of interest from an ultrasound transducer of the ultrasound imaging system.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2024
From: SWAN, WENDY; WILLIAMS, KEITH; LUNDBERG, ANDREW
To: FUJIFILM SONOSITE, INC.
Reel/Frame 068186/0921 →
Continuity (3)
Continuation 18351221 · Jul 12, 2023
Continuation 16440676 · Jun 13, 2019
Related Publication 20240389971A1 · Nov 28, 2024
References Cited (36)
US 6733458B1 · Steins et al. · 2004 [cited by applicant]
US 9492097B2 · Wilkes et al. · 2016 [cited by applicant]
US 9554716B2 · Burnside et al. · 2017 [cited by applicant]
US 11730443B2 · Swan et al. · 2023 [cited by applicant]
US 12089986B2 · Swan et al. · 2024 [cited by applicant]
US 20050059883A1 · Peterson · 2005 [cited by applicant]
US 20070112272A1 · Park et al. · 2007 [cited by applicant]
US 20100298705A1 · Pelissier et al. · 2010 [cited by applicant]
US 20110245659A1 · Ma et al. · 2011 [cited by applicant]
US 20120059260A1 · Robinson · 2012 [cited by applicant]
US 20120099770A1 · Cagnan et al. · 2012 [cited by applicant]
US 20160000399A1 · Halmann et al. · 2016 [cited by applicant]
US 20160324501A1 · Vignon et al. · 2016 [cited by applicant]
US 20160374644A1 · Mauldin et al. · 2016 [cited by applicant]
US 20170065352A1 · Razzaque et al. · 2017 [cited by applicant]
US 20170095226A1 · Tanaka et al. · 2017 [cited by applicant]
US 20180366035A1 · Dunbar · 2018 [cited by examiner]
US 20210378758A1 · Vaidya et al. · 2021 [cited by applicant]
US 20230346336A1 · Swan et al. · 2023 [cited by applicant]
JP 2004113608A · 2004 [cited by applicant]
JP 2009125280 · 2009 [cited by applicant]
JP 2018023610 · 2018 [cited by applicant]
WO 2013059714 · 2013 [cited by applicant]
WO 2020083863 · 2020 [cited by applicant]
“Extended European Search Report”, EP Application No. 20823335.3, Jun. 2, 2023, 9 pages. [cited by applicant]
“Final Office Action”, U.S. Appl. No. 16/440,676, Nov. 17, 2021, 22 pages. [cited by applicant]
“Foreign Office Action”, JP Application No. 2021-573958, Jan. 16, 2024, 8 pages. [cited by applicant]
“International Search Report and Written Opinion”, Application No. PCT/US2020/037143, Sep. 18, 2020, 18 pages. [cited by applicant]
“Non-Final Office Action”, U.S. Appl. No. 16/440,676, May 14, 2021, 16 pages. [cited by applicant]
“Non-Final Office Action”, U.S. Appl. No. 16/440,676, Sep. 2, 2022, 25 pages. [cited by applicant]
“Non-Final Office Action”, U.S. Appl. No. 18/351,221, Mar. 29, 2024, 10 pages. [cited by applicant]
“Notice of Allowance”, U.S. Appl. No. 16/440,676, Mar. 22, 2023, 10 pages. [cited by applicant]
“Notice of Allowance”, U.S. Appl. No. 18/351,221, Jul. 2, 2024, 7 pages. [cited by applicant]
Kaya, et al., “Gabor Filter Based Localization of Needles in Ultrasound Guided Robotic Interventions”, Oct. 2014, 6 pages. [cited by applicant]
“Foreign Office Action”, JP Application No. 2021-573958, Jul. 30, 2024, 5 pages. [cited by applicant]
“Foreign Office Action”, EP Application No. 20823335.3, Apr. 22, 2025, 8 pages. [cited by applicant]