Systems and methods to train a cell object detector
Systems and methods to train a cell object detector are described.
1 . A computer-implemented method comprising:
obtaining training data that includes a first set of cell images and corresponding first labels;
augmenting the training data with a second set of cell images and corresponding second labels;
training a first neural network to perform coarse detection using the augmented training data to detect one or more cell images that include candidate objects; and
training a second neural network to perform fine discrimination to identify an object of a particular type of cell object by providing one or more cell images that include the candidate objects detected by the first neural network as input to the second neural network and labels associated with the one or more cell images, wherein after the training the second neural network can distinguish between objects of a particular type and look-alike objects.
2 . The method of claim 1 , wherein training the second neural network further includes:
receiving an initial outline of a potential object within an image in the first set of cell images;
moving a center of a sample window to each pixel within the initial outline; and
at each pixel within the initial outline:
obtaining a sample of image data corresponding to the sample window;
performing one or more transformations on the sample of image data to generate one or more training images; and
providing the training images to the second neural network as training data.
3 . The method of claim 2 , wherein the transformations are selected from a group of flips, rotations, translations, and combinations thereof.
4 . The method of claim 1 , wherein the corresponding first labels identify one or more types of objects in the first set of cell images including the particular type of cell object and wherein the corresponding second labels identify the one or more look-alike objects in the second set of cell images that are similar in visual appearance to the particular type of cell object.
5 . The method of claim 1 , wherein the second set of cell images includes false positives from a previously trained neural network.
6 . The computer-implemented method of claim 1 , further comprising:
obtaining, as output of the second neural network, an indication of whether the particular type of cell object is present in a cell image.
7 . The method of claim 1 , wherein the first neural network is trained by perturbing original image data using synthetic random transformation of colors or geometry with the corresponding first labels or the corresponding second labels held fixed.
8 . The computer implemented method of claim 1 , wherein the look-alike objects comprise hard negatives.
9 . The computer implemented method of claim 1 , wherein the look-alike objects are stained a similar color shade in the one or more cell images to the color shade of objects of the particular type, but have at least one structural difference from the objects of the particular type.
10 . The computer-implemented method of claim 9 , wherein the at least one structural difference of the look-alike objects is a difference in a texture of a structure.
11 . The computer-implemented method of claim 9 , wherein the at least one structural difference of the look-alike objects is a difference in a color of a structure.
12 . The computer-implemented method of claim 9 , wherein the at least one structural difference of the look-alike objects is a difference in a boundary characteristic.
13 . The computer-implemented method of claim 12 , wherein the at least one structural difference of the look-alike objects is the look-alike objects have a smoother boundary than the objects of the particular type.
14 . The computer-implemented method of claim 1 , wherein the look-alike objects are not distinguishable by the first neural network from the objects of the particular type, but have at least one structural difference from the objects of the particular type.
15 . The computer implemented method of claim 1 , wherein the look-alike objects are stained a similar color shade in the cell images to the color shade of objects of the particular type, but have at least one feature difference from the objects of the particular type.
16 . The computer implemented method of claim 1 , wherein the objects of the particular type comprise mitotic or epi-stroma figures.
17 . A non-transitory computer-readable medium with instructions stored thereon that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
obtaining training data that includes a first set of cell images and corresponding first labels;
augmenting the training data with a second set of cell images and corresponding second labels;
training a first neural network to perform coarse detection using the augmented training data to detect one or more cell images that include candidate objects; and
training a second neural network to perform fine discrimination to identify an object of a particular type of cell object by providing one or more cell images that include the candidate objects detected by the first neural network as input to the second neural network and labels associated with the one or more cell images, wherein after the training the second neural network can distinguish between objects of a particular type and look-alike objects.
18 . The non-transitory computer-readable medium of claim 17 , wherein training the second neural network further includes:
receiving an initial outline of a potential object within an image in the first set of cell images;
moving a center of a sample window to each pixel within the initial outline;
at each pixel location within the initial outline:
obtaining a sample of image data corresponding to the sample window;
performing one or more transformations on the sample of image data to generate one or more training images; and
providing the training images to the second neural network as training data.
19 . The non-transitory computer-readable medium of claim 18 , wherein the transformations are selected from a group of flips, rotations, translations, and combinations thereof.
20 . The non-transitory computer-readable medium of claim 17 , wherein the corresponding first labels identify one or more types of objects in the first set of cell images including the particular type of cell object and wherein the corresponding second labels identify the one or more look-alike objects in the second set of cell images that are similar in visual appearance to the particular type of cell object.
21 . The non-transitory computer-readable medium of claim 17 , wherein the second set of cell images includes false positives from a previously trained neural network.
22 . The non-transitory computer-readable medium of claim 17 , wherein the operations further include:
obtaining, as output of the second neural network, an indication of whether the particular type of cell object is present in a cell image.
23 . The non-transitory computer-readable medium of claim 17 , wherein the first neural network is trained by perturbing original image data using synthetic random transformation of colors or geometry with the corresponding first labels or the corresponding second labels held fixed.
24 . A system, comprising:
one or more processors coupled to a computer readable memory having stored thereon software instructions that, when executed by the one or more processors, cause the one or more processors to perform or control performance of operations including:
obtaining training data that includes a first set of cell images and corresponding first labels;
augmenting the training data with a second set of cell images and corresponding second labels;
training a first neural network to perform coarse detection using the augmented training data to detect one or more cell images that include candidate objects; and
training a second neural network to perform fine discrimination to identify an object of a particular type of cell object by providing one or more cell images that include the candidate objects detected by the first neural network as input to the second neural network and labels associated with the one or more cell images, wherein after the training the second neural network can distinguish between objects of a particular type and look-alike objects.
25 . The system of claim 24 , wherein training the second neural network further includes:
receiving an initial outline of a potential object within an image in the first set of cell images;
moving a center of a sample window to each pixel within the initial outline; and
at each pixel within the initial outline:
obtaining a sample of image data corresponding to the sample window;
performing one or more transformations on the sample of image data to generate one or more training images; and
providing the training images to the second neural network as training data.
26 . The system of claim 25 , wherein the transformations are selected from a group of flips, rotations, translations, and combinations thereof.
27 . The system of claim 24 , wherein the corresponding first labels identify one or more types of objects in the first set of cell images including the particular type of cell object and wherein the corresponding second labels identify the one or more look-alike objects in the second set of cell images that are similar in visual appearance to the particular type of cell object.
28 . The system of claim 24 , wherein the second set of cell images includes false positives from a previously trained neural network.
29 . The system of claim 24 , wherein the operations further include:
obtaining, as output of the second neural network, an indication of whether the particular type of cell object is present in a cell image.