Medical optical system, data processing system, computer program, and non-volatile computer-readable storage medium

1 . A medical optical system comprising:

an endomicroscope for recording histological images representing each of a plurality of microscopic tissue sections of a macroscopic tissue region, the macroscopic tissue region comprising a tumor, wherein each histological image has associated navigation data representing a position and an orientation of the endomicroscope used to capture the histological image;

a classification device that processes, using a neural network, the microscopic tissue sections represented in the histological images, to classify each of the microscopic tissue sections as a tissue section representing the tumor or a tissue section representing healthy tissue and for outputting a classification result for each classified microscopic tissue section, wherein the neural network is:

trained using a plurality of training samples, each particular training sample of the plurality of training samples representing a training tissue section representing a tumor or a training tissue section representing healthy tissue, and associated with a corresponding assigned classification indicating whether the training tissue section represents healthy tissue or tumor tissue; and

trained to predict, for each particular training sample of the plurality of training samples, the corresponding assigned classification based on of the training tissue section represented by the respective training sample; and

a combination device which generates a macroscopic classification image by combining the classification results and the navigation data indicating a spatial position of each microscopic tissue section within the macroscopic tissue region, the macroscopic classification image composed of a plurality of sections, each section comprising an indication of whether a corresponding microscopic tissue section corresponds to the tumor or the healthy tissue, and wherein spatial positioning of the microscopic tissue sections in the macroscopic classification image corresponds to relative positions of the microscopic tissue sections in the macroscopic tissue region;

optical observation equipment for producing an overview image of the macroscopic tissue region; and

an overlay apparatus configured to overlay the macroscopic classification image on the overview image, wherein the overlay apparatus uses the navigation data to register the macroscopic classification image with the overview image to ensure spatial accuracy of the overlay of the microscopic classification image on the overview image.

2 . The medical optical system as claimed in claim 1 , wherein the combination device is configured to derive a macroscopic profile of the tumor based on the classification results.

3 . The medical optical system as claimed in claim 1 , wherein the classification device is configured to undertake the classifying of each microscopic tissue section based on also at least one of the following alternatives:

a decay behavior of fluorescence radiation emitted by the microscopic tissue section;

an intensity of fluorescence radiation emitted by the microscopic tissue section; or

a spectral reflection property of the microscopic tissue section.

4 . The medical optical system as claimed in claim 1 , wherein the classification device is configured to classify each microscopic tissue section into a number of classes, of which one class represents healthy tissue and the remaining classes represent different types of tumor tissue.

5 . The medical optical system as claimed in claim 1 , wherein the classification device is configured to also use data from images obtained by the optical observation equipment for classifying microscopic tissue sections.

6 . The medical optical system as claimed in claim 1 , further comprising a treatment system for a local treatment of tissue and a positioning device for positioning the treatment system based on the navigation data such that the macroscopic tissue region is treated, the positioning device being designed to undertake the positioning based on the macroscopic classification image.

7 . The medical optical system as claimed in claim 6 , wherein the treatment system comprises an irradiation system for directed irradiation of the macroscopic tissue region, and the positioning device is designed to align the irradiation system with the macroscopic tissue region based on the macroscopic classification image and the navigation data for purposes of positioning said irradiation system.

8 . The medical optical system as claimed in claim 6 , wherein the treatment system comprises an applicator for a local application of therapeutic radiation at or in the macroscopic tissue region, and the positioning device is designed to guide the applicator to the macroscopic tissue region by means of a robot, the guidance being implemented based on the macroscopic classification image and the navigation data.

9 . The medical optical system as claimed in claim 1 , further comprising a scanning device for scanning the macroscopic tissue region with the endomicroscope for purposes of obtaining the histological images for a plurality of microscopic tissue sections of the macroscopic tissue region.

10 . The medical optical system as claimed in claim 1 , further comprising a navigation system configured to gather the navigation data.

11 . A data processing system comprising:

a receiving interface for receiving histological images representing a corresponding morphology of each of a plurality of microscopic tissue sections of a macroscopic tissue region, the macroscopic tissue region comprising a tumor, wherein each histological image has associated navigation data representing a position and an orientation of an endomicroscope used to capture the histological image;

a classification device that processes, using a neural network, the microscopic tissue sections represented in the histological images, to classify each of the microscopic tissue sections represented in the histological images, as a tissue section representing the tumor or a tissue section representing healthy tissue in each case, and for outputting a classification result for each classified microscopic tissue section, wherein the neural network is:

trained using a plurality of training samples, each particular training sample of the plurality of training samples representing a training tissue section representing a tumor or a training tissue section representing healthy tissue, and associated with a corresponding assigned classification indicating whether the training tissue section represents healthy tissue or tumor tissue; and

trained to predict, for each particular training sample of the plurality of training samples, the corresponding assigned classification based on the morphology of the training tissue section represented by the respective training sample; and

a combination device which generates a macroscopic classification image by combining the classification results and the navigation data indicating a spatial position of each microscopic tissue section within the macroscopic tissue region, the macroscopic classification image composed of a plurality of sections, each section comprising an indication of whether a corresponding microscopic tissue section corresponds to the tumor or the healthy tissue, and wherein spatial positioning of the microscopic tissue sections in the macroscopic classification image corresponds to relative positions of the microscopic tissue sections in the macroscopic tissue region;

optical observation equipment for producing an overview image of the macroscopic tissue region; and

an overlay apparatus configured to overlay the macroscopic classification image on the overview image, wherein the overlay apparatus uses the navigation data to register the macroscopic classification image with the overview image to ensure spatial accuracy of the overlay of the microscopic classification image on the overview image.

12 . A non-volatile computer-readable storage medium with instructions stored thereon, wherein said instructions, when executed on a computer, cause the computer to:

receive histological images representing a corresponding morphology of each of a plurality of microscopic tissue sections of a macroscopic tissue region, the macroscopic tissue region comprising a tumor, wherein each histological image has associated navigation data representing a position and an orientation of an endomicroscope used to capture the histological image;

process, using a neural network, using a neural network, the microscopic tissue sections represented in the histological images, to classify each of the microscopic tissue sections represented in the histological images, as a tissue section representing the tumor or a tissue section representing healthy tissue in each case, and for outputting a classification result for each classified microscopic tissue section, wherein the neural network is, wherein the neural network is:

trained using a plurality of training samples, each particular training sample of the plurality of training samples representing a training tissue section representing a tumor or a training tissue section representing healthy tissue, and associated with a corresponding assigned classification indicating whether the morphology of the training tissue section represents healthy tissue or tumor tissue; and

trained to predict, for each particular training sample of the plurality of training samples, the corresponding assigned classification based on the morphology of the training tissue section represented by the respective training sample; and

generate a macroscopic classification image by combining the classification results and the navigation data indicating a spatial position of each microscopic tissue section within the macroscopic tissue region, the macroscopic classification image composed of a plurality of sections, each section comprising an indication of whether a corresponding microscopic tissue section corresponds to the tumor or the healthy tissue, and wherein spatial positioning of the microscopic tissue sections in the macroscopic classification image corresponds to relative positions of the microscopic tissue sections in the macroscopic tissue region;

produce an overview image of the macroscopic tissue region; and

overlay the macroscopic classification image on the overview image using the navigation data to register the macroscopic classification image with the overview image to ensure spatial accuracy of the overlay of the microscopic classification image on the overview image.

13 . A medical optical system comprising:

an endomicroscope for recording histological images representing a corresponding morphology of each of a plurality of microscopic tissue sections of a macroscopic tissue region, the macroscopic tissue region comprising a tumor;

a classification device that processes, using a neural network, the microscopic tissue sections represented in the histological images, to classify each of the microscopic tissue sections as a tissue section representing the tumor or a tissue section representing healthy tissue, and for outputting a classification result for each classified microscopic tissue section, wherein the neural network is:

trained using a plurality of training samples, each particular training sample of the plurality of training samples representing a training tissue section representing a tumor or a training tissue section representing healthy tissue, and associated with a corresponding assigned classification indicating whether the training tissue section represents healthy tissue or tumor tissue; and

trained to predict, for each particular training sample of the plurality of training samples, the corresponding assigned classification based on the morphology of the training tissue section represented by the respective training sample; and

a navigation system for gathering navigation data representing a site of the macroscopic tissue region at which the microscopic tissue section underlying respective classification data is situated, the navigation data representing a position and an orientation of the endomicroscope used to capture each histological image; and

a combination device which generates a macroscopic classification image by combining the classification results and the navigation data indicating a spatial position of each microscopic tissue section within the macroscopic tissue region, the macroscopic classification image composed of a plurality of sections, each section comprising an indication of whether a corresponding microscopic tissue section corresponds to the tumor or the healthy tissue, and wherein spatial positioning of the microscopic tissue sections in the macroscopic classification image corresponds to relative positions of the microscopic tissue sections in the macroscopic tissue region;

optical observation equipment for producing an overview image of the macroscopic tissue region; and

an overlay apparatus configured to overlay the macroscopic classification image on the overview image using the navigation data to register the macroscopic classification image with the overview image to ensure spatial accuracy of the overlay of the microscopic classification image on the overview image.

14 . A medical optical system as claimed in claim 13 , wherein, in the macroscopic classification image, regions representing individual classification results are arranged with respect to one another in relative positioning which corresponds to the relative positioning of the microscopic tissue sections by use of the navigation data.

15 . A medical optical system as claimed in claim 13 , wherein a spatial resolution with which a tissue section is imaged in a histological image is no greater than 20 μm.

16 . A medical optical system comprising:

an endomicroscope for recording histological images representing a corresponding decay of fluorescence radiation emitted by each of a plurality of microscopic tissue sections of a macroscopic tissue region, the macroscopic tissue region comprising a tumor, wherein each histological image has associated navigation data representing a position and an orientation of the endomicroscope used to capture the histological image;

a classification device that processes, using a neural network, the microscopic tissue sections represented in the histological images to classify each of the microscopic tissue sections as a tissue section representing the tumor or a tissue section representing healthy tissue based on the corresponding decay of fluorescence radiation emitted by the respective microscopic tissue section, and for outputting a classification result for each classified microscopic tissue section, wherein the neural network is:

trained using a plurality of training samples, each particular training sample of the plurality of training samples representing a training decay of fluorescence radiation emitted by a training tissue section and associated with a corresponding assigned classification indicating whether the training decay of fluorescence radiation of the training tissue section represents healthy tissue or tumor tissue; and

trained to predict, for each particular training sample of the plurality of training samples, the corresponding assigned classification based on the training decay of fluorescence radiation;

a combination device which generates a macroscopic classification image by combining the classification results and the navigation data indicating a spatial position of each microscopic tissue section within the macroscopic tissue region, the macroscopic classification image composed of a plurality of sections, each section comprising an indication of whether a corresponding microscopic tissue section corresponds to the tumor or the healthy tissue, and wherein spatial positioning of the microscopic tissue sections in the macroscopic classification image corresponds to relative positions of the microscopic tissue sections in the macroscopic tissue region;

optical observation equipment for producing an overview image of the macroscopic tissue region; and

an overlay apparatus being configured to overlay the macroscopic classification image on the overview image, wherein the overlay apparatus uses the navigation data to register the macroscopic classification image with the overview image to ensure spatial accuracy of the overlay of the microscopic classification image on the overview image.

17 . A medical optical system comprising:

an endomicroscope for recording histological images representing each of a plurality of microscopic tissue sections of a macroscopic tissue region, the macroscopic tissue region comprising a tumor, wherein each histological image has associated navigation data representing a position and an orientation of the endomicroscope used to capture the histological image;

optical observation equipment for producing a fluorescence image and an overview image corresponding to the macroscopic tissue region;

a classification device that processes, using a neural network, the microscopic tissue sections represented in the histological images and the fluorescence image, to classify each of the microscopic tissue sections as a tissue section representing the tumor or a tissue section representing healthy tissue and for outputting a classification result for each classified microscopic tissue section, wherein the neural network is:

trained using a plurality of training samples, each particular training sample of the plurality of training samples representing a training tissue section representing a tumor or a training tissue section representing healthy tissue, and associated with a corresponding fluorescence image and a corresponding assigned classification indicating whether the training tissue section represents healthy tissue or tumor tissue; and

trained to predict, for each particular training sample of the plurality of training samples, the corresponding assigned classification based on of the training tissue section represented by the respective training sample and the corresponding fluorescence image; and

a combination device which generates a macroscopic classification image by combining the classification results each indicating whether the corresponding microscopic tissue section represents the tumor tissue or the healthy tissue, the macroscopic classification image composed of a plurality of sections, each section comprising an indication of whether a corresponding microscopic tissue section corresponds to the tumor or the healthy tissue; and

an overlay apparatus configured to overlay the macroscopic classification image on the overview image, wherein the overlay apparatus uses the navigation data to register the macroscopic classification image with the overview image to ensure spatial accuracy of the overlay of the microscopic classification image on the overview image.

18 . The medical optical system of claim 17 , wherein the optical observation equipment comprises a surgical microscope, and wherein the overlay of microscopic classification image on the overview image is presented in an eyepiece of the surgical microscope.