Granted Patent B2

US 12,582,498 · App. 18/009,830 · Granted Mar 24, 2026

Inventors: Nicolas Padoy (Strasbourg, FR); Pietro Mascagni (Rome, IT); Bernard Dallemagne (Beaufays, BE)

Assignees: FONDATION DE COOPERATION SCIENTIFIQUE; UNIVERSITÉ DE STRASBOURG; CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE—CNRS; UNIVERSITÀ CATTOLICA DEL SACRO CUORE; INSTITUT DE RECHERCHE CONTRE LES CANCERS DE L'APPAREIL DIGESTIF

A61B90/361A61B90/37G06V10/70G16H30/20G16H50/30

Abstract

Claims (51)

1 . A device for processing a video stream related to a specific operative procedure, said device comprising:

a video stream reception interface,

a processor, and

a memory storing instructions, such that when these instructions are executed by the processor, they configure the device for:

receiving, via the video stream reception interface, the video stream comprising a sequence of images including an image to be processed which represents at least a portion of an anatomical element, said image to be processed being formed by processing elements;

determining, by means of a processing function, whether or not a criterion is satisfied in the image to be processed, the processing function being composed of a first parameterized function and a second parameterized function:

parameters of the first parameterized function being obtained by a machine learning algorithm on the basis of an image from a reference sequence of images such that the result of the first parameterized function applied to the image allows determining the processing elements of the image which represent the portion of the anatomical element, the reference sequence of images being previously recorded and related to the specific operative procedure, the image from the reference sequence of images representing at least a portion of an anatomical element, and

parameters of the second parameterized function being obtained by a machine learning algorithm on the basis of the image from the reference sequence of images combined with a result of the first parameterized function applied to the image from the reference sequence of images such that the result of the second parameterized function applied to the image, combined with a result of the first parameterized function applied to the image, allows determining whether or not the criterion is satisfied; and

determining a state of progress associated with the image to be processed, based on whether or not the criterion is satisfied, the state of progress being representative of a state of progress of an operative step of the specific operating procedure.

2 . The device according to claim 1 , wherein, when the instructions are executed by the processor, they configure the device for:

based on the determined state of progress, storing images from the sequence of images that are within a time interval around the image to be processed.

3 . The device according to claim 2 , wherein the number of images stored is dependent on the criticality of the operative step and/or on the determined state of progress.

4 . The device according to claim 1 , further comprising a display means and wherein, when the instructions are executed by the processor, they configure the device for

displaying on the display means, with the image to be processed, information dependent on the state of progress.

5 . The device according to claim 4 , wherein, when the instructions are executed by the processor, they configure the device for:

determining a deviation from an operative protocol, on the basis of the state of progress;

where the information dependent on the state of progress is dependent on the determined deviation.

6 . The device according to claim 4 , wherein the display of information dependent on the state of progress is according to a criticality of the operative step represented by the state of progress.

7 . The device according to claim 4 , wherein the information dependent on the state of progress comprises information indicating that the criterion is not validated.

8 . The device according to claim 4 , wherein the information dependent on the state of progress comprises information validating a step of an operative step or information authorizing the start of a next operative step.

9 . The device according to claim 4 , wherein, when the instructions are executed by the processor, they configure the device for:

determining a deviation between at least one value of a characteristic of a group of processing elements of the image which represent the portion of the anatomical element and a reference value, the reference value being an average of the value of the characteristic of groups of processing elements which represent the portion of the anatomical element and related to the same state of progress as the state of progress;

determining a level of risk associated with this deviation;

wherein the information dependent on the state of progress includes the level of risk.

10 . The device according to claim 4 , wherein the information dependent on the state of progress includes:

a zone in which a surgical action is to be performed; and/or

a zone in which there is no surgical action; and/or

a zone in which there is an anatomical element requiring special attention; and/or

a zone corresponding to processing elements that are considered in order to determine the state of progress.

11 . The device according to claim 4 , wherein the information dependent on the state of progress includes images from the reference sequence of images, starting with the image corresponding to the state of progress.

12 . A non-transitory computer-readable medium encoded with executable instructions which, when executed, causes an apparatus comprising a processor operatively coupled with a memory, to perform a method for processing a video stream related to a specific operative procedure, the method comprising:

receiving, via the video stream reception interface, the video stream comprising a sequence of images including an image to be processed which represents at least a portion of an anatomical element, said image to be processed being formed by processing elements;

determining, by means of a processing function, whether or not a criterion is satisfied in the image to be processed, the processing function being composed of a first parameterized function and a second parameterized function:

parameters of the first parameterized function being obtained by a machine learning algorithm on the basis of an image from a reference sequence of images such that the result of the first parameterized function applied to the image allows determining the processing elements of the image which represent the portion of the anatomical element, the reference sequence of images being previously recorded and related to the specific operative procedure, the image from the reference sequence of images representing at least a portion of an anatomical element,

parameters of the second parameterized function being obtained by a machine learning algorithm on the basis of the image from the reference sequence of images that is combined with a result of the first parameterized function applied to the image from the reference sequence of images such that the result of the second parameterized function applied to the image combined with a result of the first parameterized function applied to the image allows determining whether or not the criterion is satisfied;

determining a state of progress associated with the image to be processed, based on whether or not the criterion is satisfied, the state of progress being representative of a state of progress of an operative step of the specific operative procedure.

13 . The non-transitory computer-readable medium according to claim 12 , wherein, when the instructions are executed by the processor, they configure the device for:

based on the determined state of progress, storing images from the sequence of images that are within a time interval around the image to be processed.

14 . The non-transitory computer-readable medium according to claim 12 , wherein the number of images stored is dependent on the criticality of the operative step and/or on the determined state of progress.

15 . The non-transitory computer-readable medium according to claim 12 , further comprising a display means and wherein, when the instructions are executed by the processor, they configure the device for:

displaying on the display means, with the image to be processed, information dependent on the state of progress.

16 . The non-transitory computer-readable medium according to claim 15 wherein, when the instructions are executed by the processor, they configure the device for:

determining a deviation from an operative protocol, on the basis of the state of progress;

where the information dependent on the state of progress is dependent on the determined deviation.

17 . The non-transitory computer-readable medium according to claim 15 , wherein the display of information dependent on the state of progress is according to a criticality of the operative step represented by the state of progress.

18 . The non-transitory computer-readable medium according to claim 15 , wherein the information dependent on the state of progress comprises information indicating that the criterion is not validated.

19 . The non-transitory computer-readable medium according to claim 15 , wherein the information dependent on the state of progress comprises information validating a step of an operative step or information authorizing the start of a next operative step.

20 . The non-transitory computer-readable medium according to claim 15 wherein, when the instructions are executed by the processor, they configure the device for:

determining a deviation between at least one value of a characteristic of a group of processing elements of the image which represent the portion of the anatomical element and a reference value, the reference value being an average of the value of the characteristic of groups of processing elements which represent the portion of the anatomical element and related to the same state of progress as the state of progress;

determining a level of risk associated with this deviation;

wherein the information dependent on the state of progress includes the level of risk.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (44)