Cell image analysis apparatus, cell image analysis system, method of generating training data, method of generating trained model, training data generation program, and method of producing training data
A cell image analysis apparatus that can achieve less time and effort for labeling for generation of teaching data than in a conventional example is provided. The cell image analysis apparatus includes an image obtaining unit that obtains a cell image including a removal target that is obtained by a microscope for observation of a cell, a teaching data generator that specifies a removal target region including the removal target within the cell image by performing predetermined image processing and generates as teaching data for machine learning, a label image that represents a location of the removal target region in the cell image, and a training data set generator that generates a set of the cell image and the label image as a training data set to be used in machine learning.
1. A cellular image analysis apparatus capable of generating teaching data to be used for machine learning, the cellular image analysis apparatus comprising:
an image obtaining unit that obtains a first cellular image of an object that includes a removal target, the first cellular image being obtained by a microscope for observation of a cell;
a teaching data generator that specifies a removal target region of the first cellular image that includes an image of the removal target within the first cellular image by performing predetermined image processing on the first cellular image and generates as the teaching data for machine learning, a label image that represents a location of the removal target region within the first cellular image; and
a training data set generator that generates a set of the first cellular image and the label image as a first training data set to be used for the machine learning.
2. The cellular image analysis apparatus according to claim 1 , wherein
the image obtaining unit further obtains a second cellular image obtained by the microscope after removal of the removal target from the object, and
the predetermined image processing includes specifying the removal target region based on a result of comparison between the first cellular image and the second cellular image.
3. The cellular image analysis apparatus according to claim 2 , wherein
the predetermined image processing includes specifying the removal target region based on a subtraction image obtained by subtracting the second cellular image from the first cellular image.
4. The cellular image analysis apparatus according to claim 2 , wherein the image obtaining unit obtains the second cellular image in response to sensing a removal operation by the microscope to remove the removal target.
5. The cellular image analysis apparatus according to claim 1 , comprising:
a trained-model generator that carries out machine learning using a plurality of training data sets, including the first training data set, generated by the training data set generator and generates a trained model for identifying the removal target within an image; and
a detector that detects the removal target region in an input image input to the cellular image analysis apparatus based on the trained model.
6. The cellular image analysis apparatus according to claim 5 , wherein
machine learning carried out by the trained-model generator includes transfer learning using as an initial model, a part or entirety of the trained model trained in advance.
7. The cellular image analysis apparatus according to claim 5 , further comprising:
a storage device that stores a plurality of trained models generated by the trained- model generator; and
an input portion that accepts a selection operation to select one trained model from among the plurality of trained models, wherein
the detector detects the removal target region in the input image based on the trained model selected by the selection operation.
8. The cellular image analysis apparatus according to claim 5 , further comprising:
a display; and
a display processing unit that causes the display to show the removal target region detected by the detector as being superimposed on the input image.
9. The cellular image analysis apparatus according to claim 5 , comprising a removal mechanism controller that controls a removal mechanism of the microscope to remove the removal target based on a result of detection of the removal target region detected by the detector.
10. The cell removing apparatus of claim 1 ,
wherein the teaching data generator generates the label image of the first training data set without manual labeling of the first cellular image.
11. A cell removing apparatus capable of generating teaching data to be used for machine learning, the cell removing apparatus comprising:
a microscope for providing an observable cellular image of an object for observation by a user and to generate a corresponding first cellular image of the object;
a removal mechanism that removes a removal target from the object represented in a predetermined region within the observable cellular image;
a teaching data generator that specifies the predetermined region within the observable cellular image as a removal target region and generates as the teaching data for machine learning, a label image that represents a location of the removal target region within the first cellular image; and
a training data set generator that generates a set of the first cellular image and the label image as a first training data set to be used for the machine learning.
12. The cell removing apparatus of claim 11 ,
wherein the predetermined region within the observable cellular image is at the center of the observable cellular image.
13. The cell removing apparatus of claim 11 , wherein the observable cellular image is a through-the-lens image.
14. A cell removing apparatus according to claim 13 ,
wherein the image obtaining unit further obtains a second cellular image obtained by the microscope after removal of the removal target by the removal mechanism, and
wherein the predetermined image processing includes specifying the removal target region based on a result of comparison between the first cellular image and the second cellular image.
15. The cell removing apparatus of claim 11 , wherein the object is moveable or an imaging portion is moveable to include the removal target in the predetermined region.
16. The cell removing apparatus of claim 11 ,
wherein the removal mechanism is configured to remove the removal target in response to an acceptance of a removal operation performed by the user, and
wherein the first cellular image is obtained at the time of or immediately before the acceptance of the removal operation.
17. The cell removing apparatus of claim 11 ,
wherein the teaching data generator generates the label image of the first training data set without manual labeling of the first cellular image.
18. A method comprising:
receiving a cellular image and a label image that represents a location of a removal target region within the cellular image;
generating, by carrying out machine learning using the received cellular image and label image, a trained model that uses the cellular image as an input image and provides an image that represents a location of a removal target region within the input image as an output image; and
transmitting the generated trained model to a cellular image analysis apparatus.
19. The cell removing apparatus of claim 18 ,
wherein the cellular image and the label image from a training data set for the machine learning, and
wherein the label image is generated without manual labeling of the cellular image.
20. A method of generating teaching data to be used for machine learning, the method comprising:
obtaining a cellular image of an object that includes a removal target, the cellular image being obtained by a microscope for observation of a cell;
specifying a removal target region of the cellular image that includes an image of the removal target within the cellular image by performing predetermined image processing on the cellular image and
generating as the teaching data for machine learning, a label image that represents a location of the removal target region within the cellular image; and
generating a set of the cellular image and the label image as a training data set to be used for the machine learning.
21. The cell removing apparatus of claim 20 , wherein the teaching data generator generates the label image of the first training data set without manual labeling of the cellular image.
22. The method of claim 20 , wherein the location of the removal target region represented by the label image is generated by a comparison of the cellular image of the first training data set with a corresponding cellular image taken by the microscope after the removal target has been removed from the object.
23. A cell removing apparatus, comprising:
an image analysis apparatus according to claim 1 ; and
a removal mechanism configured to remove the target region identified via the machine learning.
24. A cell removing apparatus according to claim 23 ,
wherein the image obtaining unit further obtains a second cellular image obtained by the microscope after removal of the removal target by the removal mechanism, and
wherein the predetermined image processing includes specifying the removal target region based on a result of comparison between the first cellular image and the second cellular image.