Microscopy system and method for processing microscopy images
In a method for processing microscope images, at least a first image data set ( 30 ) of a microscope ( 10 ) is received. At least a first generative model ( 40 ) that describes the first image data set ( 30 ) is estimated with a first computing device ( 20 ) based on the first image data set ( 30 ). Either a first generated image data set ( 50 ) is generated by the first generative model ( 40 ) and transmitted to a data exploitation device ( 60 ), or the first generative model ( 40 ) is transmitted to a data exploitation device ( 60 ) and subsequently a first generated image data set ( 50 ) is generated by means of the first generative model ( 40 ). Generated image data of the first generated image data set ( 50 ) is entirely data generated from the first generative model ( 40 ) and does not comprise processed image data of the first image data set ( 30 ) captured by the microscope ( 10 ). The first generated image data set ( 50 ) is then exploited by means of the data exploitation device ( 60 ).
1 . Method for processing microscope images, comprising:
receiving at least a first image data set of a microscope;
using a first computing device to train at least a first generative model with the first image data set as training data, such that the first generative model is configured to map a random input without any input image data to generated image data which has at least some information content in common with image data of the first image data set;
generating a first generated image data set from random inputs without any input image data to the first generative model and transmitting the first generated image data set to a data exploitation device, or transmitting the first generative model to a data exploitation device and subsequently generating a first generated image data set through inputting random inputs without any input image data to the first generative model;
wherein generated image data of the first generated image data set is entirely data generated from the first generative model and does not comprise processed image data of the first image data set captured by the microscope;
exploiting the first generated image data set using the data exploitation device;
training a respective generative model from each of a plurality of image data sets with a respective computing device, the training including training a second generative model to map random inputs using a second image data set without using the first image data set and wherein training of the first generative model to map random inputs does not use the second image data set, such that each of the first and second generative models is configured to map a random input without any input image data to respective generated image data which has at least some information content in common with image data of the respective image data set;
generating a respective generated image data set with each of the first and second generative models from random inputs without any input image data after training of each generative model is concluded; and
jointly exploiting the plurality of generated image data sets generated after concluding the training with the data exploitation device.
2 . The method according to claim 1 , further comprising:
using each microscope of a plurality of microscopes to capture a corresponding one of the plurality of image data sets, each of the plurality of microscopes comprising one of the computing devices.
3 . The method according to claim 1 ,
wherein the data exploitation device uses the plurality of generated image data sets as training data for a machine learning algorithm.
4 . The method according to claim 3 ,
wherein the machine learning algorithm trained with the plurality of generated image data sets originating at different microscopes is transmitted to the computing devices of a plurality of microscopes.
5 . The method according to claim 1 ,
wherein each computing device comprises a program which calculates the respective generative model based on the respective image data set.
6 . The method according to claim 5 ,
wherein the program comprises a learning algorithm which calculates the respective generative model based on the respective image data set by means of generative adversarial networks.
7 . The method according to claim 5 ,
wherein the program comprises a simulation or rendering software with which the generative model is generated.
8 . The method according to claim 5 ,
wherein the program is designed to provide a limitation of the generated generative model by means of which the generative model does not reproduce certain information of the image data set which is used to calculate the generated generative model.
9 . The method according to claim 1 ,
wherein an input tool is provided by the first computing device via which parameters of the first generative model can be entered by a user.
10 . The method according to claim 1 ,
wherein the first computing device provides an annotation tool via which a user can enter annotations relating to microscope images or image components of the first image data set,
wherein the annotations entered by the user are utilized to configure the estimated first generative model to also generate annotations for the first generated image data set.
11 . The method according to claim 1 ,
wherein the first generated image data set is used as test data for an image processing algorithm which generates processing results from the first generated image data set, and
wherein a quality of the processing results is subsequently assessed.
12 . The method according to claim 1 ,
wherein exploiting the first generated image data set by means of the data exploitation device comprises estimating a quality of an associated microscope.
13 . A non-transitory computer-readable medium comprising a computer program with commands that, when the computer program is executed by a computer, cause the execution of the method according to claim 1 .
14 . Microscopy system, comprising:
a first computing device which is configured to receive at least a first image data set of a microscope and to train at least a first generative model with the first image data set as training data, such that the first generative model is configured to map a random input without any input image data to generated image data which appears to come from a distribution of image data of the first image data set;
a data exploitation device which is configured to exploit a first generated image data set generated from random inputs without any input image data to the first generative model,
wherein either the first computing device is configured to generate the first generated image data set by means of the first generative model and to transmit the first generated image data set to the data exploitation device, or the data exploitation device is configured to receive the first generative model from the first computing device and to generate the first generated image data set using the first generative model, wherein generated image data of the first generated image data set is entirely data generated from the first generative model and does not comprise processed image data of the first image data set captured by the microscope;
at least a second computing device configured for receiving a second image data set and training a second generative model to map random inputs using the second image data set without using the first image data set, wherein training of the first generative model to map random inputs does not use the second image data set, such that each of the first and second generative models is configured to map a random input without any input image data to respective generated image data which has at least some information content in common with image data of the respective image data set;
generating a respective generated image data set with each of the first and second generative models from random inputs without any input image data after training of each generative model is concluded; and
wherein the data exploitation device is configured for jointly exploiting the first and second generated image data sets generated after concluding the training of the first and second generative models.
15 . The microscopy system according to claim 14 , further comprising:
a plurality of microscopes, each microscope including a respective computing device, wherein each of the plurality of microscopes is configured to capture a corresponding one of the plurality of image data sets.
16 . The microscopy system according to claim 14 ,
wherein the data exploitation device uses the plurality of generated image data sets as training data for a machine learning algorithm.
17 . The microscopy system according to claim 16 ,
wherein the machine learning algorithm trained with the plurality of generated image data sets originating at different microscopes is transmitted to the computing devices of a plurality of microscopes.
18 . The microscopy system according to claim 14 ,
wherein each computing device comprises a program which calculates the respective generative model based on the respective image data set.
19 . The microscopy system according to claim 14 ,
wherein the first computing device provides an annotation tool via which a user can enter annotations relating to microscope images or image components of the first image data set,
wherein the annotations entered by the user are utilized to configure the estimated first generative model to also generate annotations for the first generated image data set.
20 . The microscopy system according to claim 14 ,
wherein the data exploitation device is configured to exploit the first generated image data set to estimate a quality of the microscope used to capture the first image data set.