IP Library Granted Patent US 11,080,854
Granted Patent B2
US 11,080,854 · App. 16/834,125 · Granted Aug 3, 2021

Augmented surgical reality environment

Inventors: Tony C. Carnes (Gainesville, FL); Edward M. Mckenna (Boulder, CO); Stephen Pack (Louisville, CO)
Assignee: COVIDIEN LP
G06T7/0012A61B90/361A61B90/37G06T5/10G06T5/50G06T7/13G06T11/008G06T19/006G06T19/20H04N5/232H04N5/33A61B2090/365A61B2576/00G06T2207/10048G06T2207/20024G06T2207/20221G06T2207/30004H04N5/23293
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,080,854
App. No.
16/834,125
Granted
Aug 3, 2021
Kind
B2
Abstract

The present disclosure is directed to an augmented reality surgical system for viewing an augmented image of a region of interest during a surgical procedure. The system includes an image capture device that captures an image of the region of interest. A controller receives the image and applies at least one image processing filter to the image. The image processing filter includes a spatial decomposition filter that decomposes the image into spatial frequency bands. A temporal filter is applied to the spatial frequency bands to generate temporally filtered bands. An adder adds each band spatial frequency band to a corresponding temporally filtered band to generate augmented bands. A reconstruction filter generates an augmented image by collapsing the augmented bands. A display displays the augmented image to a user.

Claims (39)

1. An image processing filter of an augmented reality surgical system for viewing an augmented image of a region of interest, the image processing filter comprising:

a spatial decomposition filter configured to decompose an image of a region of interest inside a patient into a plurality of spatial frequency bands;

a temporal filter configured to be applied to the plurality of spatial frequency bands to generate a plurality of temporally filtered bands;

an adder configured to add each band in the plurality of spatial frequency bands to a corresponding band in the plurality of temporally filtered bands to generate a plurality of augmented bands; and

a reconstruction filter configured to generate the augmented image by collapsing the plurality of augmented bands.

2. The image processing filter of claim 1 , wherein the image processing filter applies the at least one image processing filter to each image frame of a plurality of image frames of the image of a region of interest.

3. The image processing filter of claim 1 , wherein the temporal filter includes a bandpass filter.

4. The image processing filter of claim 3 , wherein a bandpass frequency of the bandpass filter is set by a clinician.

5. The image processing filter of claim 1 , wherein the temporal filter isolates at least one spatial frequency band from the plurality of spatial frequency bands to generate the plurality of temporally filtered bands.

6. The image processing filter of claim 1 , wherein the plurality of temporally filtered bands is amplified by an amplifier before each band in the plurality of spatial frequency bands is added to the corresponding band in the plurality of temporally filtered bands to generate the plurality of augmented bands.

7. The image processing filter of claim 1 , wherein the image processing filter uses an edge detection algorithm configured to highlight one or more edges in the image, and the one or more highlighted edges is added to the augmented image.

8. The image processing filter of claim 1 , wherein the image processing filter uses a hyper-spectral algorithm to combine a plurality of spectral images to generate a three dimensional hyper-spectral image cube that is added to the augmented image.

9. The image processing filter of claim 1 , wherein the image processing filter generates an infrared image from the image, and the infrared image is added to the augmented image.

10. A method for generating an augmented image of a region of interest during a surgical procedure, the method comprising:

decomposing at least one image to generate a plurality of spatial frequency bands; applying a temporal filter to the plurality of spatial frequency bands to generate a plurality of temporally filtered bands;

adding each band in the plurality of spatial frequency bands to a corresponding band in the plurality of temporally filtered bands to generate a plurality of augmented bands;

collapsing the plurality of augmented bands to generate the augmented image; and

displaying the augmented image on a display.

11. The method of claim 10 , further comprising isolating at least one spatial frequency band from the plurality of spatial frequency bands.

12. The method of claim 11 , further comprising amplifying the temporally filtered bands before adding each band in the plurality of spatial frequency bands to a corresponding band in the plurality of temporally filtered bands to generate a plurality of augmented bands.

13. The method of claim 10 , further comprising:

applying an edge detection algorithm configured to highlight one or more edges in the image; and

adding the one or more highlighted edges to the augmented image.

14. The method of claim 10 , further comprising:

adding a three dimensional hyper-spectral image cube, generated from a plurality of hyper-spectral images, to the augmented image.

15. The method of claim 10 , further comprising:

adding an infrared image to the augmented image.

16. A non-transitory computer readable medium storing instructions that, when executed by a processing device, cause the processing device to:

decompose an image of a region of interest inside a patient into a plurality of spatial frequency bands;

apply a temporal filter to the plurality of spatial frequency bands to generate a plurality of temporally filtered bands;

add each band in the plurality of spatial frequency bands to a corresponding band in the plurality of temporally filtered bands to generate a plurality of augmented bands; and

generate the augmented image by collapsing the plurality of augmented bands.

17. The non-transitory computer readable medium of claim 16 , further comprising additional stored instructions that, when executed by the processing device, cause the processing device to isolate at least one spatial frequency band containing information about an imperceptible property from the plurality of spatial frequency bands.

18. The non-transitory computer readable medium of claim 17 , further comprising additional stored instructions that, when executed by the processing device, cause the processing device to amplify the temporally filtered bands before adding each band in the plurality of spatial frequency bands to a corresponding band in the plurality of temporally filtered bands.

19. The non-transitory computer readable medium of claim 17 , further comprising additional stored instructions that, when executed by the processing device, cause the processing device to:

apply an edge detection algorithm configured to highlight one or more edges in the image data, the one or more edges being the imperceptible property; and

add the one or more highlighted edges to the augmented image.

20. The non-transitory computer readable medium of claim 16 , further comprising additional stored instructions that, when executed by the processing device, cause the processing device to:

combine a plurality of spectral images to generate a three dimensional hyper-spectral image cube that is added to the augmented image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2020
From: MCKENNA, EDWARD; PACK, STEPHEN
To: COVIDIEN LP
Reel/Frame 052260/0305 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2020
From: CARNES, TONY C.
To: COVIDIEN LP
Reel/Frame 052260/0471 →
Continuity (4)
Continuation 16210686 · Dec 5, 2018
Continuation 15327058
Provisional Application 62028974 · Jul 25, 2014
Related Publication 20200226758A1 · Jul 16, 2020