Granted Patent B2

US 12,205,315 · App. 17/781,272 · Granted Jan 21, 2025

Inventors: Pourya Shirazian (Menlo Park, CA); Mahdi Azizian (San Jose, CA); A. Jonathan McLeod (Sunnyvale, CA); Azad Shademan (Campbell, CA)

Assignee: Intuitive Surgical Operations, Inc.

G06T7/62G06T7/593G06T15/06G06T2207/10012G06T2207/10028G06T2207/30004G06T2210/21

Abstract

Claims (88)

1. A system comprising:

a memory storing instructions; and

a processor communicatively coupled to the memory and configured to execute the instructions to:

access, during a surgical procedure that involves resecting a piece of tissue from a body, a plurality of depth datasets for the resected piece of tissue, each depth dataset in the plurality of depth datasets captured as a different portion of a surface of the resected piece of tissue is presented to an imaging device by an instrument that holds the resected piece of tissue in a manner that sequentially presents the different portions of the surface to the imaging device;

determine, during the surgical procedure and based on the plurality of depth datasets, an estimated volume of the resected piece of tissue;

indicate, during the surgical procedure, whether the estimated volume of the resected piece of tissue is within a predetermined threshold of an expected volume of the resected piece of tissue; and

provide, during the surgical procedure and prior to the resected piece of tissue being removed from the body, the estimated volume of the resected piece of tissue for presentation to a member of a surgical team performing the surgical procedure.

2. The system of claim 1 , wherein each depth dataset in the plurality of depth datasets includes, for a respective portion of the surface of the resected piece of tissue:

depth data representative of the respective portion of the surface;

metadata representative of a pose of the imaging device as the respective portion of the surface is presented to the imaging device by the instrument; and

metadata representative of a pose of the instrument as the respective portion of the surface is presented to the imaging device by the instrument.

3. The system of claim 1 , wherein:

the processor is further configured to execute the instructions to generate a three-dimensional (3D) occupancy map that includes a set of voxels identified to be occupied by the resected piece of tissue;

the generating of the 3D occupancy map includes:

performing a raytracing operation involving a set of virtual rays simulated to extend from a point associated with the imaging device to various points of intersection in the body, the raytracing operation including determining that one or more virtual rays of the set of virtual rays intersect with one or more points on the surface of the resected piece of tissue and that one or more other virtual rays of the set of virtual rays are determined not to intersect with the surface of the resected piece of tissue; and

based on the raytracing operation, allocating, within a voxel data structure stored by the system to implement the 3D occupancy map, a respective occupied voxel for each of the one or more points on the surface of the resected piece of tissue with which a virtual ray is determined to intersect as part of the raytracing operation; and

the determining of the estimated volume of the resected piece of tissue is performed based on the 3D occupancy map.

4. The system of claim 3 , wherein:

at least one of the virtual rays determined to intersect with a first point of the one or more points on the surface is further determined to intersect, after passing through the resected piece of tissue, with a second point of the one or more points on the surface of the resected piece of tissue; and

the generating of the 3D occupancy map further includes allocating, within the voxel data structure, an additional occupied voxel associated with an internal point disposed within the resected piece of tissue between the first and second points on the surface of the resected piece of tissue.

5. The system of claim 3 , wherein the generating of the 3D occupancy map further includes allocating, within the voxel data structure, an additional occupied voxel associated with a point on the surface of the resected piece of tissue that:

is not determined by the raytracing operation to intersect with a virtual ray of the set of virtual rays, and

is disposed between two points on the surface of the resected piece of tissue that are determined by the raytracing operation to intersect with virtual rays of the set of virtual rays.

6. The system of claim 1 , wherein the processor is further configured to execute the instructions to implement, in addition to implementing a first volume detection technique that includes the accessing of the plurality of depth datasets and the determining of the estimated volume of the resected piece of tissue, a second volume detection technique that is configured to perform at least one of:

verifying an accuracy of the first volume detection technique; or

refining the estimated volume determined using the first volume detection technique.

7. The system of claim 6 , wherein the second volume detection technique includes:

receiving user input from a member of a surgical team performing the surgical procedure, the user input representative of a parameter of a geometric shape having a volume defined as a function of the parameter;

providing, to the member of the surgical team as the user input is provided, a representation of the geometric shape in relation to the resected piece of tissue, the representation configured to facilitate the member of the surgical team in selecting the parameter so as to make the volume of the geometric shape approximate the volume of the resected piece of tissue; and

determining, based on the volume of the geometric shape for the parameter represented by the received user input, an additional estimated volume of the resected piece of tissue.

8. The system of claim 6 , wherein the second volume detection technique includes:

accessing an additional plurality of depth datasets for a cavity left by the resected piece of tissue;

determining, based on the additional plurality of depth datasets, an estimated volume of the cavity left by the resected piece of tissue; and

determining, based on the estimated volume of the cavity, an additional estimated volume of the resected piece of tissue.

9. The system of claim 6 , wherein the second volume detection technique includes:

determining a force value that is applied to the instrument to allow the instrument to hold the resected piece of tissue in place;

determining, based on the force value, a mass of the resected piece of tissue;

accessing an estimated density value for the resected piece of tissue; and

determining, based on the force value and the estimated density value, an additional estimated volume of the resected piece of tissue.

10. The system of claim 6 , wherein the second volume detection technique includes:

dividing a geometric shape into a plurality of individually-sizable sectors, each individually-sizable sector having a volume defined as a function of a parameter associated with the individually-sizable sector, and a volume of the geometric shape defined as a sum of the volumes of all of the individually-sizable sectors;

setting the respective parameters defining the volumes of each of the plurality of individually-sizable sectors in such a way as to make the individually-sizable sectors conform to corresponding parts of the surface of the resected piece of tissue;

determining the volume of the geometric shape by summing the volumes of all of the plurality of individually-sizable sectors after the respective parameters have been set; and

determining, based on the volume of the geometric shape, an additional estimated volume of the resected piece of tissue.

11. The system of claim 1 , wherein the plurality of depth datasets accessed for the resected piece of tissue collectively include depth data representative of an entirety of the surface of the resected piece of tissue.

12. The system of claim 1 , wherein:

the imaging device is implemented as a stereoscopic imaging device that includes stereoscopic imaging elements; and

the accessing of the plurality of depth datasets includes generating each of the plurality of depth datasets by determining depth data for the respective portion of the surface of the resected piece of tissue using a stereoscopic depth detection technique that employs the stereoscopic imaging elements of the stereoscopic imaging device.

13. A system comprising:

a memory storing instructions; and

a processor communicatively coupled to the memory and configured to execute the instructions to:

access, during a surgical procedure that involves resecting a piece of tissue from a body, a plurality of depth datasets for the resected piece of tissue, each depth dataset in the plurality of depth datasets captured as a different portion of a surface of the resected piece of tissue is presented to an imaging device by an instrument that holds the resected piece of tissue in a manner that sequentially presents the different portions of the surface to the imaging device;

access an expected volume of the resected piece of tissue, the expected volume determined prior to the surgical procedure;

determine, during the surgical procedure and based on the plurality of depth datasets, an estimated volume of the resected piece of tissue;

compare, during the surgical procedure, the estimated volume of the resected piece of tissue with the expected volume of the resected piece of tissue;

indicate, during the surgical procedure to a member of a surgical team performing the surgical procedure, whether the estimated volume is within a predetermined threshold of the expected volume; and

provide, during the surgical procedure and prior to the resected piece of tissue being removed from the body, the estimated volume of the resected piece of tissue for presentation to the member of the surgical team.

14. The system of claim 13 , wherein each depth dataset in the plurality of depth datasets includes, for a respective portion of the surface of the resected piece of tissue:

depth data representative of the respective portion of the surface;

metadata representative of a pose of the imaging device as the respective portion of the surface is presented to the imaging device by the instrument; and

metadata representative of a pose of the instrument as the respective portion of the surface is presented to the imaging device by the instrument.

15. The system of claim 13 , wherein:

the processor is further configured to execute the instructions to generate a three-dimensional (3D) occupancy map that includes a set of voxels identified to be occupied by the resected piece of tissue;

the generating of the 3D occupancy map includes:

performing a raytracing operation involving a set of virtual rays simulated to extend from a point associated with the imaging device to various points of intersection in the body, the raytracing operation including determining that one or more virtual rays of the set of virtual rays intersect with one or more points on the surface of the resected piece of tissue and that one or more other virtual rays of the set of virtual rays are determined not to intersect with the surface of the resected piece of tissue; and

based on the raytracing operation, allocating, within a voxel data structure stored by the system to implement the 3D occupancy map, a respective occupied voxel for each of the one or more points on the surface of the resected piece of tissue with which a virtual ray is determined to intersect as part of the raytracing operation; and

the determining of the estimated volume of the resected piece of tissue is performed based on the 3D occupancy map.

16. A method comprising:

accessing, by a tissue volume detection system during a surgical procedure that involves resecting a piece of tissue from a body, a plurality of depth datasets for the resected piece of tissue, each depth dataset in the plurality of depth datasets captured as a different portion of a surface of the resected piece of tissue is presented to an imaging device by an instrument that holds the resected piece of tissue in a manner that sequentially presents the different portions of the surface to the imaging device;

determining, by the tissue volume detection system during the surgical procedure and based on the plurality of depth datasets, an estimated volume of the resected piece of tissue;

indicating, by the tissue volume detection system during the surgical procedure, whether the estimated volume of the resected piece of tissue is within a predetermined threshold of an expected volume of the resected piece of tissue; and

providing, by the tissue volume detection system during the surgical procedure and prior to the resected piece of tissue being removed from the body, the estimated volume of the resected piece of tissue for presentation to a member of a surgical team performing the surgical procedure.

17. The method of claim 16 , wherein each depth dataset in the plurality of depth datasets includes, for a respective portion of the surface of the resected piece of tissue:

depth data representative of the respective portion of the surface;

metadata representative of a pose of the imaging device as the respective portion of the surface is presented to the imaging device by the instrument; and

metadata representative of a pose of the instrument as the respective portion of the surface is presented to the imaging device by the instrument.

18. The method of claim 16 , further comprising generating, by the tissue volume detection system during the surgical procedure, a three-dimensional (3D) occupancy map that includes a set of voxels identified to be occupied by the resected piece of tissue; wherein:

the generating of the 3D occupancy map includes:

performing a raytracing operation involving a set of virtual rays simulated to extend from a point associated with the imaging device to various points of intersection in the body, the raytracing operation including determining that one or more virtual rays of the set of virtual rays intersect with one or more points on the surface of the resected piece of tissue and that one or more other virtual rays of the set of virtual rays are determined not to intersect with the surface of the resected piece of tissue; and

based on the raytracing operation, allocating, within a voxel data structure stored by the tissue volume detection system to implement the 3D occupancy map, a respective occupied voxel for each of the points on the surface of the resected piece of tissue with which a virtual ray is determined to intersect as part of the raytracing operation; and

the determining of the estimated volume of the resected piece of tissue is performed based on the 3D occupancy map.

19. The method of claim 16 , further comprising implementing, by the tissue volume detection system in addition to implementing a first volume detection technique that includes the accessing of the plurality of depth datasets and the determining of the estimated volume of the resected piece of tissue, a second volume detection technique that is configured to perform at least one of:

verifying an accuracy of the first volume detection technique; or

refining the estimated volume determined using the first volume detection technique.

20. The method of claim 19 , wherein the second volume detection technique includes:

receiving user input from a member of a surgical team performing the surgical procedure, the user input representative of a parameter of a geometric shape having a volume defined as a function of the parameter;

providing, to the member of the surgical team as the user input is provided, a representation of the geometric shape in relation to the resected piece of tissue, the representation configured to facilitate the member of the surgical team in selecting the parameter so as to make the volume of the geometric shape approximate the volume of the resected piece of tissue; and

determining, based on the volume of the geometric shape for the parameter represented by the received user input, an additional estimated volume of the resected piece of tissue.

Assignments (1)

Continuity (2)

References Cited (10)