Method of training a machine learning model in order to create at least one virtual histological stained image
A method includes providing a first predetermined number of unstained tissue samples and a second predetermined number of stained tissue samples, and training a machine learning model with the first predetermined number of unstained tissue samples and the second predetermined number of stained tissue samples in order to create at least one virtual histological stained image.
1 . A method, comprising:
providing a first predetermined number of unstained tissue samples and a second predetermined number of stained tissue samples;
generating a first number of brightfield digital images of the first predetermined number of unstained tissue samples;
generating a second number of brightfield digital images of the second predetermined number of stained tissue samples;
generating a plurality of paired patches of the first number of brightfield digital images and the second number of brightfield digital images; and
training a machine learning model with the plurality of paired patches by learning feature embeddings between the first number of brightfield digital images and the second number of brightfield digital images in order to digitally create at least one virtual histological stained image from an algorithm.
2 . The method according to claim 1 , wherein the machine learning model is a neural network system, and wherein training further comprises testing the neural network system with the plurality of paired patches in order to assess performance of the neural network system on unseen data.
3 . The method according to claim 2 , further comprising merging the plurality of paired patches using alpha blending in order to create the at least one virtual histological stained image.
4 . The method according to claim 1 , further comprising employing a scale invariant transformation to identify key points from a first image of the first number of brightfield digital images and a second image of the second number of brightfield digital images.
5 . The method according to claim 4 , wherein employing the scale invariant transformation further comprises matching the key points of the first image to the key points of the second image in order to calculate a Homography matrix.
6 . The method according to claim 5 , wherein employing the scale invariant transformation further comprises applying the Homography matrix to the first image in order to register the first image with the second image and create a set of data.
7 . The method according to claim 6 , wherein training the machine learning model further comprises using the set of data to learn nonlinear data mapping between tissue in the first predetermined number of unstained tissue samples and the second predetermined number of stained tissue samples.
8 . The method according to claim 6 , wherein the key points of the first image and the key points of the second image comprise nuclei.
9 . The method according to claim 1 , wherein training the machine learning model further comprises employing at least one encoder and at least one projection head with each of a first domain corresponding to the first predetermined number of unstained tissue samples and a second domain corresponding to the second predetermined number of stained tissue samples in order to bridge a gap between the first and second domains.
10 . The method according to claim 9 , further comprising employing a first generator for mapping the first domain to the second domain.
11 . The method according to claim 10 , further comprising employing a second generator for learning an inverse mapping of the second domain to the first domain.
12 . The method according to claim 11 , further comprising employing at least one discriminator in order to ensure that first images of the first number of brightfield digital images and second images of the second number of brightfield digital images belong to the first and second domains, respectively, while the first and second generators are being employed.
13 . The method according to claim 1 , wherein the machine learning model is a neural network system, and wherein training the machine learning model comprises employing cycle consistency by performing a calculation of a first digital difference between one of the second number of brightfield digital images and the at least one virtual histological stained image.
14 . The method according to claim 13 , wherein training the neural network system further comprises employing the cycle consistency by performing another calculation of a second digital difference between the at least one virtual histological stained image and the one of the second number of brightfield digital images, in order to cause the at least one histological stained image to be identical to the one of the second number of brightfield digital images.
15 . The method according to claim 1 , wherein providing the first predetermined number of unstained tissue samples and the second predetermined number of stained tissue samples further comprises preprocessing the first predetermined number of unstained tissue samples in order to remove background via adaptive thresholding.
16 . The method according to claim 1 , wherein the machine learning model is a dual contrastive learning general adversarial network (GAN) model.
17 . The method according to claim 16 , further comprising employing the dual contrastive learning GAN model to learn a correspondence between the first number of brightfield digital images and the second number of brightfield digital images using separated embeddings.
18 . The method according to claim 16 , further comprising applying contrastive learning with the dual contrastive learning GAN model by applying a noise contrastive estimation framework.
19 . The method according to claim 1 , wherein training the machine learning model is further performed in order to allow for diagnosis by a computerized machine.