IP Library Granted Patent US 11,744,667
Granted Patent B2
US 11,744,667 · App. 16/729,803 · Granted Sep 5, 2023

Adaptive visualization by a surgical system

Inventors: Frederick E. Shelton, IV (Hillsboro, OH); Andrew C. Deck (Cincinnati, OH)
Assignee: Cilag GmbH International
A61B90/361A61B1/0005A61B1/00194A61B1/045A61B1/05A61B1/0605A61B1/0638G06T7/0012A61B1/000094G06T2207/10068G06T2207/20221G06T2207/30004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,744,667
App. No.
16/729,803
Filed
Dec 30, 2019
Granted
Sep 5, 2023
Kind
B2
Art Unit
2665
USPC
382/128
Abstract

Various adaptive surgical visualization systems are disclosed. Surgical visualizations can compensate for obscured, incomplete, damaged, or interfered with portions of captured images by substituting those portions of the images with corresponding portions of other images. The other images could include images that were previously generated by the surgical visualization system or images that were generated using multispectral imaging techniques.

Claims (58)

1. A surgical imaging system comprising:

a multispectral electromagnetic radiation (EMR) source configured to emit EMR at a first wavelength range and a second wavelength range;

an image sensor configured to sense the EMR at each of the first wavelength range and the second wavelength range reflected from a target site; and

a control circuit coupled to the image sensor, the control circuit configured to:

generate a first image of the target site of a subsurface structure according to the EMR emitted at the first wavelength range;

determine whether the first image is at least partially obstructed by an obscurant, wherein the obscurant obstructs the view of the subsurface structure;

select the second wavelength range to minimize absorption by the obscurant;

generate a second image of the target site according to the EMR emitted at the second wavelength range; and

generate a fused image comprising a fusion between an unobstructed segment of the first image at the first wavelength range and a segment of the second image corresponding to an obstructed segment of the first image at the second wavelength range.

2. The surgical imaging system of claim 1 , wherein the second wavelength range is configured to be substantially non-interactive with at least one of a surgical smoke, a fluid, or a tissue type.

3. The surgical imaging system of claim 1 , wherein the first wavelength range is within a visible portion of an EMR spectrum and the second wavelength range is outside of the visible portion of the EMR spectrum.

4. The surgical imaging system of claim 1 , wherein:

the multispectral EMR source is further configured to emit EMR at a third wavelength;

the image sensor is further configured to sense the EMR at the third wavelength; and

the control circuit is further configured to:

generate a third image of the target site according to the EMR emitted at the third wavelength; and

generate the fused image comprising the unobstructed segment of the first image, the segment of the second image corresponding to the obstructed segment of the first image, and a second segment of the third image corresponding to a second obstructed segment of the first image.

5. The surgical imaging system of claim 1 , wherein the control circuit is further configured to:

filter a subset wavelength range from the first wavelength range to define a filtered first wavelength range; and

generate the first image of the target site according to the EMR emitted at the filtered first wavelength range.

6. The surgical imaging system of claim 5 , wherein the subset wavelength range corresponds to a blue-green portion of a visible portion of an EMR spectrum.

7. The surgical imaging system of claim 1 , wherein the control circuit is configured to generate the fused image intraoperatively.

8. A surgical imaging system comprising:

a multispectral light source configured to emit electromagnetic radiation (EMR);

an image sensor configured to sense the EMR; and

a control circuit coupled to the image sensor, the control circuit configured to:

generate an image of a target site of a subsurface structure according to the emitted EMR;

determine whether the image is at least partially obstructed by an obscurant, wherein the obscurant obstructs the view of the subsurface structure;

retrieve a previous image of the target site at a wavelength range to minimize absorption by the obscurant; and

generate a fused image comprising a fusion between an unobstructed segment of the image and a segment of the previous image corresponding to an obstructed segment of the image at the wavelength range.

9. The surgical imaging system of claim 8 , further comprising:

a memory configured to store a plurality of time indexed images generated by the surgical imaging system;

wherein the control circuit is configured to retrieve the previous image from the plurality of time indexed images.

10. The surgical imaging system of claim 9 , wherein the control circuit is further configured to:

successively retrieve a plurality of previous images from the plurality of time indexed images; and

generate the fused image comprising a fusion between the unobstructed segment of the image and a plurality of segments of the plurality of previous images corresponding to the obstructed segment of the image.

11. The surgical imaging system of claim 8 , wherein the control circuit is configured to generate the fused image utilizing lucky-region fusion.

12. A method of imaging a target site via a surgical imaging system, the surgical imaging system comprising a multispectral electromagnetic radiation (EMR) source configured to emit EMR at a first wavelength range and a second wavelength range and an image sensor configured to sense the EMR at each of the first wavelength range and the second wavelength range reflected from the target site, the method comprising:

generating a first image of the target site of a subsurface structure according to the EMR emitted at the first wavelength range;

determining whether the first image is at least partially obstructed by an obscurant, wherein the obscurant obstructs the view of the subsurface structure;

selecting the second wavelength range to minimize absorption by the obscurant;

generating a second image of the target site according to the EMR emitted at the second wavelength range; and

generating a fused image comprising a fusion between an unobstructed segment of the first image at the first wavelength range and a segment of the second image corresponding to an obstructed segment of the first image at the second wavelength range.

13. The method of claim 12 , wherein the second wavelength range is configured to be substantially non-interactive with at least one of a surgical smoke, a fluid, or a tissue type.

14. The method of claim 12 , wherein the first wavelength range is within a visible portion of an EMR spectrum and the second wavelength range is outside of the visible portion of the EMR spectrum.

15. The method of claim 12 , wherein:

the multispectral EMR source is further configured to emit EMR at a third wavelength;

the image sensor is further configured to sense the EMR at the third wavelength; and

the method further comprises:

generating a third image of the target site according to the EMR emitted at the third wavelength; and

generating the fused image comprising the unobstructed segment of the first image, the segment of the second image corresponding to the obstructed segment of the first image, and a second segment of the third image corresponding to a second obstructed segment of the first image.

16. The method of claim 12 , further comprising:

filtering a subset wavelength range from the first wavelength range to define a filtered first wavelength range; and

generating the first image of the target site according to the EMR emitted at the filtered first wavelength range.

17. The method of claim 16 , wherein the subset wavelength range corresponds to a blue-green portion of a visible portion of an EMR spectrum.

18. The method of claim 12 , the fused image is generated intraoperatively.

19. The surgical imaging system of claim 1 , wherein the generated first image and second image are time-indexed.

20. The method of claim 12 , wherein the generated first image and second image are time-indexed.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2021
From: ETHICON LLC
To: CILAG GMBH INTERNATIONAL
Reel/Frame 056601/0339 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 27, 2020
From: SHELTON, FREDERICK E., IV; DECK, ANDREW C.
To: ETHICON LLC
Reel/Frame 052245/0781 →
Continuity (1)
Related Publication 20210196109A1 · Jul 1, 2021
Cited By (1)
US 12,633,392