Image-based assay using intelligent monitoring structures
The present invention is related to correct the errors in instruments, operation, and others using intelligent monitoring structures and machine learning, and others.
1. An apparatus for assaying a sample using an optical system suspected of having imperfections, comprising:
a sample holder comprising a first plate and a second plate, wherein each of the first and second plates comprises a respective sample contact area contacting the sample that contains or is suspected of containing an analyte, and the first and second plates sandwich at least a part of the sample into a thin layer of a substantially uniform thickness of 200 um or less;
a plurality of monitoring marks that are distributed in one of the sample contact areas, wherein the one or more monitoring marks comprise at least one predetermined parameter associated with a geometric property or an optical property of the monitoring marks, a lateral linear dimension of 1 um or larger, and at least one lateral linear dimension of 300 um or less;
an optical imaging system that is suspected of having an imperfection, wherein the optical imaging system is configured to take one or more images of the at least part of the sample, wherein the imperfection is associated with a local image distortion and/or a local illumination variation, and wherein the imperfection of the optical system causes an image imperfection in the image; and
a computer-readable medium configured to store a correction algorithm that corrects the image imperfection using the at least one predetermined parameter and the image of the monitoring marks in the at least part of the sample.
2. The apparatus of claim 1 , further comprising:
a second computer-readable medium storing a machine learning model, wherein the machine learning model is configured to transform an image take by the optical imaging system into a transformed image, and the training and/or the image transformation of the machine learning model are performed after the images of the at least part of the sample has been corrected by the correction algorithm.
3. The apparatus of claim 1 , wherein the correction algorithm corrects a local image distortion of the image of the at least part of the sample.
4. The apparatus of claim 1 , wherein the correction algorithm corrects a local illumination variation of the image of the at least part of the sample.
5. The apparatus of claim 1 , wherein the correction algorithm is a machine learning model.
6. The apparatus of claim 1 , wherein the correction algorithm is a computer vision.
7. The apparatus of claim 1 , wherein the correction algorithm is a combination of machine learning and a computer vision.
8. The apparatus of claim 1 , wherein the monitoring marks are periodic structures with a predetermined period.
9. The apparatus of claim 1 , wherein the monitoring marks are the spacers, wherein the spacers are the structures between the first and second plates and regulated the spacing between the plates.
10. The apparatus of claim 1 , wherein each monitoring mark comprises either a protrusion or a trench from the sample contact area of the plate, wherein the protrusion or the trench comprises a flat surface that is substantially parallel to a neighbor surface that is a portion of the sample contact area adjacent the protrusion or the trench.
11. The apparatus of claim 1 , wherein each of the monitoring marks is a pillar with a flat top surface.
12. The apparatus of claim 1 , wherein the distance between the monitoring marks is equal or less than about 120 um.
13. The apparatus of claim 1 , wherein the distance between the monitoring marks is in the range of 120 um to 200 um.
14. The apparatus of claim 1 , wherein the monitoring mark is a cylindrical structure with a cross-section of a triangle, a rectangle, a square, a circle, or a polygon.
15. The apparatus of claim 1 , wherein the analyte is a protein, peptide, DNA, RNA, nucleic acid, small molecule, cell, or nanoparticle.
16. The apparatus of claim 1 , wherein the analyte is a blood cell.
17. The apparatus of claim 1 , wherein the correction algorithm comprises a homographic transform.
18. The apparatus of claim 1 , wherein the monitoring marks comprise at least 4 non-colinear pillars.
19. The apparatus of claim 1 , wherein each of the monitoring marks is a pillar with a flat top surface of area 10,000 um{circumflex over ( )}2 or less.
20. The apparatus of claim 1 , wherein each of the monitoring marks has a sharp edge.
21. The apparatus of claim 1 , wherein the predetermined geometric property of the monitoring marks comprises the shape and dimension of the monitoring marks and/or the center-to-center distance between the monitoring marks.
22. A method of assaying a sample using an optical system suspected of having imperfections, the method comprising:
providing a sample holder comprising a first plate and a second plate, wherein each of the first and second plates has a respective sample contact area contacting the sample that contains or is suspected of containing an analyte, and the first and second plates sandwich at least a part of the sample into a thin layer of a substantially uniform thickness of 200 um or less;
providing a plurality of monitoring marks that are distributed in one of the sample contact areas, wherein the monitoring marks comprise at least one predetermined parameter associated with a geometric property or an optical property of the monitoring marks, a lateral linear dimension of 1 um or larger, and at least one lateral linear dimension of 300 um or less;
providing an optical imaging system that is suspected of having an imperfection, wherein the imperfection is associated with a local image distortion and/or a local illumination variation;
imaging, using the optical imaging system, one or more images of the at least part of the sample, wherein the imperfection of the optical system causes an image imperfection in the image, and
correcting, using a correction algorithm stored in a computer-readable medium, the image imperfection using the at least one predetermined parameter and the one or more images of the monitoring marks in the at least part of the sample.
23. The method of claim 22 , further comprising:
using a machine learning model to perform an image transformation of the image of the at least part of the sample; and
using the transformed image to measure the analyte;
wherein the training and/or the image transformation of the machine learning model are performed after the image of the at least part of the sample has been corrected by the correction algorithm.
24. The method of claim 22 , wherein the correction algorithm corrects a local image distortion of the image of the at least part of the sample.
25. The method of claim 22 , wherein the correction algorithm corrects a local illumination variation of the image of the at least part of the sample.
26. The method of claim 22 , wherein the correction algorithm is a machine learning model.
27. The method of claim 22 , wherein the correction algorithm is a computer vision.
28. The method of claim 22 , wherein the correction algorithm is a combination of machine learning and a computer vision.
29. The method of claim 22 , wherein the monitoring marker are periodic structures with a predetermined period.
30. The method of claim 22 , wherein the monitoring marks are the spacers, wherein the spacers are the structures between the first and second plates and regulated the spacing between the plates.
31. The method of claim 22 , wherein each monitoring mark comprises either a protrusion or a trench from the sample contact area of the plate, wherein the protrusion or the trench comprises a flat surface that is substantially parallel to a neighbor surface that is a portion of the sample contact area adjacent the protrusion or the trench.
32. The method of claim 22 , wherein each of the monitoring marks is a pillar with a flat top surface.
33. The method of claim 22 , wherein the distance between the monitoring marks is equal or less than about 120 um.
34. The method of claim 22 , wherein the distance between the monitoring marks is in the range of 120 um to 200 um.
35. The method of claim 22 , wherein the monitoring mark is a cylindrical structure with a cross-section of a triangle, a rectangle, a square, a circle, or a polygon.
36. The method of claim 22 , wherein the analyte is a protein, peptide, DNA, RNA, nucleic acid, small molecule, cell, or nanoparticle.
37. The method of claim 22 , wherein the analyte is a blood cell.
38. The method of claim 22 , wherein the correction algorithm comprises a homographic transform.
39. The method of claim 22 , wherein the monitoring marks comprise at least 4 non-colinear pillars.
40. The method of claim 22 , wherein each of the monitoring marks is a pillar with a flat top surface of area 10,000 um{circumflex over ( )}2 or less.
41. The method of claim 22 , wherein each of the monitoring marks has a sharp edge.
42. The method of claim 22 , wherein the predetermined geometric property of the monitoring marks comprises the shape and dimension of the monitoring marks and/or the center-to-center distance between the monitoring marks.