Granted Patent B2

US 12,272,161 · App. 17/596,279 · Granted Apr 8, 2025

Inventor: Constantin Kappel (Schriesheim, DE)

Assignee: Leica Microsystems CMS GmbH

G06V20/69G06V10/454G06V10/774G06V10/82G06V20/698

Abstract

Claims (48)

1. A system comprising one or more processors, one or more interfaces, and one or more storage devices, wherein the system, using the one or more processors, is configured to:

obtain biology-related image-based input data via the one or more interfaces;

select a trained visual recognition machine-learning algorithm from a plurality of trained visual recognition machine-learning algorithms based on the biology-related image-based input data, wherein the trained visual recognition machine-learning algorithm is selected so that the values of more than 5 entries of the high-dimensional representation are larger than 10% of a largest absolute value of the entries of the high-dimensional representation;

generate a high-dimensional representation of the biology-related image-based input data by the trained visual recognition machine-learning algorithm executed by the one or more processors, wherein the high-dimensional representation comprises at least 3 entries each having a different value; and

at least one of:

store the high-dimensional representation of the biology-related image-based input data together with the biology-related image-based input data by the one or more storage devices; or

output biology-related language-based output data corresponding to the high-dimensional representation.

2. The system of claim 1 , wherein the biology-related image-based input data is image data of an image of at least one of a biological structure comprising a nucleotide sequence, a biological structure comprising a protein sequence, a biological molecule, biological tissue, a biological structure with a specific behavior, or a biological structure with a specific biological function or a specific biological activity.

3. The system of claim 1 , wherein the values of one or more entries of the high-dimensional representation are proportional to a likelihood of a presence of a specific biological function or a specific biological activity.

4. The system of claim 1 , wherein the biology-related language- based output data is at least one of a nucleotide sequence, a protein sequence, a description of a biological molecule or biological structure, a description of a behavior of a biological molecule or biological structure, or a description of a biological function or a biological activity.

5. The system of claim 1 , further comprising a microscope configured to obtain the biology-related image-based input data by taking an image of a biological specimen.

6. The system of claim 1 , wherein the high-dimensional representation is a numerical representation.

7. The system of claim 1 , wherein the high-dimensional representation comprises more than 100 dimensions.

8. The system of claim 1 , wherein the high-dimensional representation is a vector.

9. The system of claim 1 , wherein the trained visual recognition machine-learning algorithm is selected so that more than 50% of values of the entries of the high-dimensional representation are unequal 0.

10. The system of claim 1 , wherein the trained visual recognition machine-learning algorithm comprises a trained visual recognition neural network.

11. The system of claim 10 , wherein the trained visual recognition neural network comprises more than 30 layers.

12. The system of claim 10 , wherein the trained visual recognition neural network is a convolutional neural network or a capsule network.

13. The system of claim 10 , wherein the trained visual recognition neural network comprises a plurality of convolution layers and a plurality of pooling layers.

14. The system of claim 10 , wherein the trained visual recognition neural network uses a rectified linear unit activation function.

15. The system of claim 1 , wherein the system is further configured to determine, using the one or more processors, the biology-related language-based output data based on the high-dimensional representation by a decoder of a trained language recognition machine-learning algorithm executed by the one or more processors.

16. The system of claim 15 , wherein the biology-related language-based output data is an entry of a vocabulary trained by the trained language recognition machine-learning algorithm.

17. The system of claim 1 , wherein the system, using the one or more processors further configured to:

obtain a plurality of biology-related image-based data sets via the one or more interfaces, wherein the plurality of biology-related image-based data sets are stored in a database;

generate an individual high-dimensional representation for every biology-related image-based input data set of the plurality of biology-related image-based data sets by the trained visual recognition machine-learning algorithm executed by the one or more processors; and

at least one of:

store the individual high-dimensional representations together with the corresponding biology-related image-based input data sets by the one or more storage devices, or

output biology-related language-based output data sets corresponding to the individual high-dimensional representations.

18. The system of claim 17 , wherein the system, using the one or more processors, is further configured to:

receive biology-related language-based search data via the one or more interfaces;

generate a high-dimensional representation of the biology-related language-based search data by a trained language recognition machine-learning algorithm executed by the one or more processors;

compare the high-dimensional representation of the biology-related language-based search data with the individual high-dimensional representations of the plurality of biology-related image-based data sets; and

output a biology-related image-based data set of the plurality of biology-related image-based data sets based on the comparison.

19. The system of one of claim 1 , wherein the system, using the one or more processors, is further configured to select the trained visual recognition machine-learning algorithm using a classification algorithm configured to classify the biology-related image-based input data.

20. The system of claim 1 , wherein the system, using the one or more processors configured to:

select a second trained visual recognition machine-learning algorithm from the plurality of trained visual recognition machine-learning algorithms;

generate a second high-dimensional representation of the biology-related image-based input data by the second trained visual recognition machine-learning algorithm executed by the one or more processors, wherein the second high-dimensional representation comprises at least 3 entries each having a different value;

at least one of:

store the second high-dimensional representation of the biology-related image-based input data together with the high-dimensional representation and the biology-related image-based input data by the one or more storage devices, or output the biology-related language-based output data and second biology-related language-based output data corresponding to the second high-dimensional representation.

21. The system of claim 1 , wherein the system is configured to control an operation of a microscope.

22. A microscope comprising a system of claim 1 .

23. A method for processing biology-related image-based input data, the method comprising:

obtaining biology-related image-based input data;

selecting a trained visual recognition machine-learning algorithm from a plurality of trained visual recognition machine-learning algorithms based on the biology-related image-based input data, wherein the trained visual recognition machine-learning algorithm is selected so that the values of more than 5 entries of the high-dimensional representation are larger than 10% of a largest absolute value of the entries of the high-dimensional representation;

generating a high-dimensional representation of the biology-related image-based input data by the trained visual recognition machine-learning algorithm, wherein the high-dimensional representation comprises at least 3 entries each having a different value; and

at least one of:

storing the high-dimensional representation of the biology-related image-based input data together with the biology-related image-based input data, or outputting biology-related language-based output data corresponding to the high-dimensional representation.

24. A non-transitory, computer readable medium having a program code for performing a method according to claim 23 when the program is executed by processor.

Assignments (1)

Continuity (1)

References Cited (31)

Cited By (2)

US 12,511,921 US 12,731,661