IP Library Granted Patent US 12695843
Granted Patent B2
US 12695843 · App. 17/451,578 · Granted Jul 28, 2026

Maintaining neighboring contextual awareness with zoom

Inventors: Anisha Goel (Santa Cruz, CA); Nicola Christin Rieke (Munich, DE)
Assignee: NVIDIA Corporation
H04N5/2628A61B1/00009A61B1/0005A61B1/044G06V10/25H04N23/69
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12695843
App. No.
17/451,578
Granted
Jul 28, 2026
Kind
B2
Abstract

An area of interest within a visual output of a surgical site is identified. An area of interest within a visual output of a surgical site is identified. A zoom operation on the area of interest to generate a magnified area of interest is performed. An occluded region around the area of interest within the visual output is determined based on an amount of zoom associated with the zoom operation. The occluded region is a region of the visual output that becomes occluded by placing a magnified area of interest over the area of interest within the visual output. A non-linear compression to the occluded region of the visual output to generate a compressed occluded region is applied. The magnified area of interest to include the compressed occluded region is updated.

Claims (56)

1 . A method comprising:

identifying an area of interest within a visual output of a surgical site;

performing a zoom operation on the area of interest to generate a magnified area of interest;

determining, based on an amount of zoom associated with the zoom operation, an occluded region around the area of interest within the visual output, wherein the occluded region is a region of the visual output generated based on a difference between the magnified area of interest and the area of interest within the visual output prior to performing the zoom operation;

applying a non-linear compression to the occluded region of the visual output to generate a compression zone within the magnified area of interest, wherein the compression zone has a magnification that varies based on radial distance, an inner portion of the compression zone having a first magnification based on a magnification of the magnified area of interest and an outer portion of the compression zone having a second magnification based on a magnification of a remainder of the visual output; and

updating the magnified area of interest by replacing at least one pixel of the occluded region with at least one pixel of the compression zone.

2 . The method of claim 1 , wherein the visual output of the surgical site is an image.

3 . The method of claim 1 , wherein identifying the area of interest within the visual output of the surgical site includes cropping the area of interest from the visual output of the surgical site.

4 . The method of claim 3 , further comprising:

stitching the magnified area of interest comprising the compression zone to the visual output of the surgical site.

5 . The method of claim 1 , wherein the visual output of the surgical site is one frame of a video of the surgical site, and wherein the identifying, the performing, the determining, the applying, the updating are performed for a plurality of frames of the video.

6 . The method of claim 5 , wherein the video is a live feed of the surgical site, the method further comprising:

outputting the live feed of the video comprising the visual output and the magnified area of interest.

7 . The method of claim 1 , wherein applying non-linear compression to the occluded region comprises:

obtaining, based on the zoom operation, a scale factor;

determining a zoom area percentage based on a radius between a center of the magnified area of interest and a radius of the magnified area of interest;

determining, based on the zoom area percentage and the scale factor, a power coefficient;

for each pixel of the occluded region, determining, based on the power coefficient, a new pixel value for the respective pixel; and

updating the respective pixel with the new pixel value.

8 . The method of claim 7 , wherein determining the new pixel value includes obtaining polar coordinates of the respective pixel, determining a new radius value by applying the power coefficient to a radius value of the polar coordinates of the respective pixel, and obtaining, based on the new radius value, cartesian coordinates of the new pixel.

9 . A system comprising:

an image capture device configured to acquire at least one frame of a surgical site; and

one or more processors, operatively coupled to the image capture device, the one or more processors to perform operations comprising:

identifying, based on the at least one frame of the surgical site, an area of interest within a visual output of the surgical site;

performing a zoom operation on the area of interest to generate a magnified area of interest;

determining, based on an amount of zoom associated with the zoom operation, an occluded region around the area of interest within the visual output, wherein the occluded region is a region of the visual output generated based on a difference between the magnified area of interest and the area of interest within the visual output prior to performing the zoom operation;

applying a non-linear compression to the occluded region of the visual output to generate a compression zone within the magnified area of interest, wherein the compression zone has a magnification that varies based on radial distance, an inner portion of the compression zone having a first magnification based on a magnification of the magnified area of interest and an outer portion of the compression zone having a second magnification based on a magnification of a remainder of the visual output; and

updating the magnified area of interest by replacing at least one pixel of the occluded region with at least one pixel of the compression zone.

10 . The system of claim 9 , wherein the visual output of the surgical site is an image.

11 . The system of claim 9 , wherein identifying the area of interest within the visual output of the surgical site includes cropping the area of interest from the visual output of the surgical site.

12 . The system of claim 11 , wherein the one or more processors is to perform further operations comprising:

stitching the magnified area of interest comprising the compression zone to the visual output of the surgical site.

13 . The system of claim 9 , wherein the at least one frame associated with the visual output of the surgical site is a frame of a video of the surgical site, and wherein the identifying, the performing, the determining, the applying, the updating are performed for a plurality of frames of the video.

14 . The system of claim 13 , wherein the video is a live feed of the surgical site, the one or more processors to perform operations further comprising:

outputting the live feed of the video comprising the visual output and the magnified area of interest.

15 . The system of claim 9 , wherein applying non-linear compression to the occluded region comprises:

obtaining, based on the zoom operation, a scale factor;

determining a zoom area percentage based on a radius between a center of the magnified area of interest and a radius of the magnified area of interest;

determining, based on the zoom area percentage and the scale factor, a power coefficient;

for each pixel of the occluded region, determining, based on the power coefficient, a new pixel value for the respective pixel; and

updating the respective pixel with the new pixel value.

16 . The system of claim 15 , wherein determining the new pixel value includes obtaining polar coordinates of the respective pixel, determining a new radius value by applying the power coefficient to a radius value of the polar coordinates of the respective pixel, and obtaining, based on the new radius value, cartesian coordinates of the new pixel.

17 . A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

identifying an area of interest within a visual output of a surgical site;

performing a zoom operation on the area of interest to generate a magnified area of interest;

determining, based on an amount of zoom associated with the zoom operation, an occluded region around the area of interest within the visual output, wherein the occluded region is a region of the visual output generated based on a difference between the magnified area of interest and the area of interest within the visual output prior to performing the zoom operation;

applying a non-linear compression to the occluded region of the visual output to generate a compression zone within the magnified area of interest, wherein the compression zone has a magnification that varies based on radial distance, an inner portion of the compression zone having a first magnification based on a magnification of the magnified area of interest and an outer portion of the compression zone having a second magnification based on a magnification of a remainder of the visual output; and

updating the magnified area of interest by replacing at least one pixel of the occluded region with at least one pixel of the compression zone.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein applying non-linear compression to the occluded region comprises:

obtaining, based on the zoom operation, a scale factor;

determining a zoom area percentage based on a radius between a center of the magnified area of interest and a radius of the magnified area of interest;

determining, based on the zoom area percentage and the scale factor, a power coefficient;

for each pixel of the occluded region, determining, based on the power coefficient, a new pixel value for the respective pixel; and

updating the respective pixel with the new pixel value.

19 . The non-transitory computer-readable storage medium of claim 17 , wherein identifying the area of interest within the visual output of a surgical site includes cropping the area of interest from the visual output of the surgical site.

20 . The non-transitory computer-readable storage medium of claim 17 , wherein the visual output of the surgical site is one frame of a video of the surgical site, and wherein the identifying, the performing, the determining, the applying, the updating are performed for a plurality of frames of the video.