IP Library › Granted Patent US 12,161,430
Granted Patent B2
US 12,161,430 · App. 17/261,589 · Granted Dec 10, 2024

Surgical support system, data processing apparatus and method

Inventors: Christopher Wright (London, GB); Matthew Lawrenson (Lausanne, CH); Naoyuki Hirota (Tokyo, JP)
Assignee: Sony Corporation
A61B34/25A61B5/163A61B34/30A61B90/08G06F3/013A61B2017/00119A61B2017/00216A61B2034/254A61B2034/258A61B2090/0807
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,161,430
App. No.
17/261,589
Filed
Jan 20, 2021
Granted
Dec 10, 2024
Kind
B2
Art Unit
3656
USPC
700/245
Abstract

A surgical support system including: an eye behaviour monitoring apparatus that monitors eye behaviour of a surgeon performing a surgical procedure to obtain eye behaviour data of the surgeon; a surgical data generating apparatus that generates surgical data associated with the surgical procedure being performed by the surgeon; a surgical intervention apparatus that performs an intervention procedure for intervening in the surgeon's performance of the surgical procedure; and a data processing apparatus that: determines a value of an intervention parameter associated with the surgical procedure using the obtained eye behaviour data; determines an acceptable range of the value of the intervention parameter using the generated surgical data, and if the determined value of the intervention parameter is outside the determined acceptable range of the value of the intervention parameter, controls the surgical intervention apparatus to perform the intervention procedure for intervening in the surgeon's performance of the surgical procedure.

Claims (60)

1. A surgical support system comprising:

an eye monitor to monitor eye behaviour of a surgeon performing a surgical procedure to obtain eye behaviour data of the surgeon;

a surgical data sensor to generate surgical data associated with the surgical procedure being performed by the surgeon;

a surgical intervention circuit configured to perform an intervention procedure for intervening in the surgeon's performance of the surgical procedure; and

a processing circuit configured to:

determine a value of an intervention parameter associated with the surgical procedure using the obtained eye behaviour data, wherein the intervention parameter is indicative of a likelihood of a potential error event in the surgical procedure based on eye behaviour data of the surgeon;

determine an acceptable range of the value of the intervention parameter using the generated surgical data and the potential error event, wherein the acceptable range of the value of the intervention parameter being indicative of an acceptable likelihood of the surgeon making the potential error event, and

if the determined value of the intervention parameter is outside the determined acceptable range of the value of the intervention parameter, control the surgical intervention circuit to perform the intervention procedure for intervening in the surgeon's performance of the surgical procedure, wherein the intervention procedure includes:

outputting an alert signal indicating that the determined likelihood of the surgeon making the potential error event in the surgical procedure exceeds the acceptable likelihood; and/or

providing an adjustment signal to a surgical apparatus used by the surgeon to perform a surgical function in the surgical procedure to control the surgical apparatus to adjust the performed surgical function to pre-empt the potential error event in the surgical procedure.

2. A surgical support system according to claim 1 , wherein the eye behaviour data is one of pupil monitoring data and eye tracking data.

3. A surgical support system according to claim 1 , wherein the processing circuit is configured to:

to compare the obtained eye behaviour data of the surgeon with predetermined eye behaviour data associated with a predetermined likelihood of a surgeon making an error in the surgical procedure; and

to determine the likelihood of the surgeon making an error in the surgical procedure using the predetermined likelihood associated with the predetermined eye behaviour data and a result of the comparison between the obtained eye behaviour data of the surgeon and the predetermined eye behaviour data.

4. A surgical support system according to claim 3 , wherein the generated surgical data is indicative of at least one of a type of the surgical procedure, a current stage of the surgical procedure and a current condition of the patient of the surgical procedure.

5. A surgical support system according to claim 4 , wherein the processing circuit is configured to select the predetermined eye behaviour data using the generated surgical data.

6. A surgical support system according to claim 4 , wherein the processing circuit is configured to select an intervention procedure using the generated surgical data and the potential error event and control the surgical intervention circuit to perform the selected intervention procedure.

7. A surgical support system according to claim 1 , wherein the alert signal is a visual indication displayed on a display on which a surgical image is displayed.

8. A surgical support system according to claim 7 , wherein the visual indication is superimposed in a vicinity of a region of interest in the surgical image.

9. A surgical support system according to claim 1 , wherein the processing circuit is configured:

to determine, using the obtained eye behaviour data, a region of a field of view of the surgeon with which an error the likelihood of which has been determined is associated; and

to determine the intervention procedure based on a characteristic of the determined region of the field of view of the surgeon and the potential error event.

10. A surgical support system according to claim 9 , wherein the processing circuit is configured:

to output, for display, an image representative of the region of the field of view of the surgeon for viewing by the surgeon;

to control the eye behaviour monitoring apparatus to obtain further eye behaviour data of the surgeon as the surgeon views the displayed image; and

to determine the intervention procedure using the further eye behaviour data of the surgeon.

11. A surgical support system according to claim 10 , wherein the processing circuit is configured:

to determine, using the obtained further eye behaviour data, a sub-region of the region of the field of view of the surgeon with which an error the likelihood of which has been determined is associated; and

to determine the intervention procedure based on a characteristic of the determined sub-region of the region of the field of view of the surgeon.

12. A surgical support system according to claim 1 , wherein the processing circuit is configured:

to determine, using the obtained eye behaviour data, a region of a field of view of the surgeon with which an error the likelihood of which has been determined is associated;

to output, for display, an image representative of the region of the field of view of the surgeon for viewing by the surgeon; and

to control the eye monitor to obtain further eye behaviour data of the surgeon as the surgeon views the displayed image;

wherein the intervention circuit is configured to adjust a display characteristic of the displayed image and determine, using the further eye behaviour data of the surgeon, an adjusted display characteristic which reduces the likelihood of the error associated with the region of the field of view of the surgeon.

13. A surgical support system according to claim 1 , wherein the processing circuit is further configured to: determine a current condition from the generated surgical data; select an intervention procedure using the current condition of the patient of the surgical procedure; and output a signal to control the surgical intervention circuit to perform the selected intervention procedure.

14. A data processing apparatus for a surgical support system, the surgical support system comprising an eye monitor to monitor eye behaviour of a surgeon performing a surgical procedure to obtain eye behaviour data of the surgeon, a surgical data sensor to generate surgical data associated with the surgical procedure being performed by the surgeon and a surgical intervention circuit to perform an intervention procedure for intervening in the surgeon's performance of the surgical procedure, wherein the data processing apparatus comprises circuitry configured to:

receive the obtained eye behaviour data;

receive the generated surgical data;

determine a value of an intervention parameter associated with the surgical procedure using the obtained eye behaviour data, wherein the intervention parameter is indicative of a likelihood of a potential error event in the surgical procedure based on eye behaviour data of the surgeon,

determine an acceptable range of the value of the intervention parameter using the generated surgical data and the potential error event, wherein the acceptable range of the value of the intervention parameter being indicative of an acceptable likelihood of the surgeon making the potential error event, and

if the determined value of the intervention parameter is outside the determined acceptable range of the value of the intervention parameter, output a signal to control the surgical intervention circuit to perform the intervention for intervening in the surgeon's performance of the surgical procedure,

intervention circuit to perform the intervention procedure for intervening in the surgeon's performance of the surgical procedure, wherein the intervention procedure includes:

outputting an alert signal indicating that the determined likelihood of the surgeon making the potential error event in the surgical procedure exceeds the acceptable likelihood; and/or

providing an adjustment signal to a surgical apparatus used by the surgeon to perform a surgical function in the surgical procedure to control the surgical apparatus to adjust the performed surgical function to pre-empt the potential error event in the surgical procedure.

15. A data processing apparatus according to claim 14 , wherein the generated surgical data is indicative of at least one of a type of the surgical procedure, a current stage of the surgical procedure and a current condition of the patient of the surgical procedure.

16. A method of operating a surgical support system, the surgical support system comprising an eye monitor configured to monitor eye behaviour of a surgeon performing a surgical procedure to obtain eye behaviour data of the surgeon, a surgical data sensor to generate surgical data associated with the surgical procedure being performed by the surgeon and a surgical intervention circuit configured to perform an intervention procedure for intervening in the surgeon's performance of the surgical procedure, wherein the method comprises:

receiving the obtained eye behaviour data;

receiving the generated surgical data;

determining a value of an intervention parameter associated with the surgical procedure using the obtained eye behaviour data, wherein the intervention parameter is indicative of a likelihood of a potential error event in the surgical procedure based on eye behaviour data of the surgeon,

determining an acceptable range of the value of the intervention parameter using the generated surgical data and the potential error event, wherein the acceptable range of the value of the intervention parameter being indicative of an acceptable likelihood of the surgeon making the potential error event,

determining whether the value of the intervention parameter is within the acceptable range, and

in response to the value of the intervention parameter being outside the acceptable range, outputting a signal to control the surgical intervention circuit to perform the intervention procedure for intervening in the surgeon's performance of the surgical procedure, wherein the intervention procedure includes:

outputting an alert signal indicating that the determined likelihood of the surgeon making the potential error event in the surgical procedure exceeds the acceptable likelihood; and/or

providing an adjustment signal to a surgical apparatus used by the surgeon to perform a surgical function in the surgical procedure to control the surgical apparatus to adjust the performed surgical function to pre-empt the potential error event in the surgical procedure.

17. A non-transitory, computer readable storage medium storing a program for controlling a computer to perform a method according to claim 6 .

18. A method according to claim 16 , further comprising:

selecting an intervention procedure using the generated surgical data and the potential error event; and

outputting a signal to control the surgical intervention circuit to perform the selected intervention procedure.

19. A method according to claim 16 , further comprising: determining a current condition from the generated surgical data; selecting an intervention procedure using the current condition of the patient of the surgical procedure; and outputting a signal to control the surgical intervention circuit to perform the selected intervention procedure.

20. A method according to claim 16 , wherein the generated surgical data is indicative of at least one of a type of the surgical procedure, a current stage of the surgical procedure and a current condition of the patient of the surgical procedure.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 20, 2021
From: WRIGHT, CHRISTOPHER; LAWRENSON, MATTHEW; HIROTA, NAOYUKI
To: SONY CORPORATION
Reel/Frame 054962/0312 →
Priority Claims (1)
EP 18200263 · Oct 12, 2018 · regional
Continuity (1)
Related Publication 20210259789A1 · Aug 26, 2021
Cited By (4)
US 12,633,121 US 12,661,189 US 12,678,239 US 12,682,637