IP Library › Granted Patent US 12,198,344
Granted Patent B2
US 12,198,344 · App. 18/527,037 · Granted Jan 14, 2025

Autonomous cell imaging and modeling system

Inventors: Hervé Marie-Nelly (San Francisco, CA); Jeevaa Velayutham (Shah Alam, MY); Zachary Phillips (San Francisco, CA); Shengjiang Tu (Foster City, CA)
Assignee: Insitro, Inc.
G06T7/0012A61B5/4848G01N15/1429G16H20/00G06T2207/10064G06T2207/20036G06T2207/30024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,198,344
App. No.
18/527,037
Granted
Jan 14, 2025
Kind
B2
Abstract

The present disclosure relates generally to an autonomous cell imaging and modeling platform, and more specifically to machine-learning techniques for using microscopy imaging data to continuously study live biological cells. The autonomous cell imaging and modeling platform can be applied to evaluate various cellular processes, such as cellular differentiation, optimization of cell culture (e.g., in-plate cytometry), disease modeling, histopathology imaging, and genetic and chemical screening, using a dynamic universal imaging system. In some embodiments, the platform comprises a set of label-free computational imaging techniques, self-supervised learning models, and robotic devices configured in an autonomous imaging system to study positional and morphological characteristics in particular cellular substructures of a cell culture in an efficient and non-destructive manner over time.

Claims (68)

1. A system for modeling a characteristic of interest of a cell culture comprising one or more live mammalian cells, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors to:

obtain a first set of one or more images capturing the cell culture at a first time point;

input the first set of one or more images into a trained machine-learning model to obtain a first set of embedding vectors representing positional and morphological characteristics in particular cellular substructures of the cell culture;

evaluate the characteristic of interest of the cell culture corresponding to the first time point based on the first set of embedding vectors to obtain a first set of one or more values;

obtain a second set of one or more images capturing the cell culture at a second time point;

input the second set of one or more images into the trained machine-learning model to obtain a second set of embedding vectors representing positional and morphological characteristics in particular cellular substructures of the cell culture;

evaluate the characteristic of interest of the cell culture corresponding to the second time point based on the second set of embedding vectors to obtain a second set of one or more values; and

determine, from the first set of one or more values and the second set of one or more values, a change of the characteristic of interest in the cell culture.

2. The system of claim 1 , wherein the characteristic of interest is cell proliferation of the cell culture, wherein:

i) the first set of one or more values indicates a predicted first proliferation level;

ii) the second set of one or more values indicates a predicted second proliferation level; and

iii) a rate of proliferation of the cell culture is determined from the predicted first proliferation level and the predicted second proliferation level.

3. The system of claim 2 , wherein the one or more programs, when executed by the one or more processors, cause the system to:

determine the in vitro and/or metabolic fitness of the cell culture based on the predicted first proliferation level and the predicted second proliferation level.

4. The system of claim 2 , wherein the one or more programs, when executed by the one or more processors, cause the system to: compare the rate of proliferation to a predefined threshold.

5. The system of claim 4 , wherein the one or more programs, when executed by the one or more processors, cause the system to:

based on the rate of proliferation being determined to exceed the predefined threshold:

determine that the rate of proliferation is an abnormal proliferation rate; and

terminate growth of the cell culture before a predefined endpoint of cell growth.

6. The system of claim 2 , wherein the one or more programs, when executed by the one or more processors, cause the system to:

predict a confluence of the cell culture for a third time point after the second time point based on the rate of proliferation of the cell culture.

7. The system of claim 6 , wherein the one or more programs, when executed by the one or more processors, cause the system to:

determine timing for passaging of the cell culture based on the predicted confluence of the cell culture.

8. The system of claim 2 , wherein the one or more programs, when executed by the one or more processors, cause the system to:

determine timing for passaging of the cell culture based on the rate of proliferation.

9. The system of claim 2 , wherein the trained machine-learning model is a first machine-learning model, predicting a cell proliferation level comprises:

inputting the first set of embedding vectors into a second machine-learning model to obtain the cell proliferation level corresponding to the first time point; and

inputting the second set of embedding vectors into the second machine-learning model to obtain the cell proliferation level corresponding to the second time point.

10. The system of claim 9 , wherein the one or more programs, when executed by the one or more processors, cause the system to:

generate a time trend based on the predicted first cell proliferation level and the predicted second cell proliferation level.

11. The system of claim 9 , wherein the second machine-learning model is a linear regression classifier.

12. The system of claim 9 , wherein the second machine-learning model is trained using a set of embedding vectors and a corresponding set of cell proliferation levels.

13. The system of claim 2 , wherein the characteristic of interest is health of the cell culture, wherein:

i) the first set of one or more values indicates a predicted first cell health level;

ii) the second set of one or more values indicates a predicted second cell health level; and

iii) a change of the health level of the cell culture is determined from the predicted first cell health level and the predicted second cell health level.

14. The system of claim 2 , wherein the characteristic of interest is development of the cell culture, wherein:

i) the first set of one or more values indicates a predicted first cell development level;

ii) the second set of one or more values indicates a predicted second cell development level; and

iii) a change of the development level of the cell culture is determined from the predicted first cell development level and the predicted second cell development level.

15. The system of claim 2 , wherein the first set of one or more of images and the second set of one or more images comprise phase images.

16. The system of claim 2 , wherein the first set of one or more of images and the second set of one or more images are generated from fluorescence images or autofluorescence images.

17. The system of claim 2 , wherein the trained machine-learning model is a self-supervised machine-learning model.

18. The system of claim 2 , wherein the trained machine-learning model is trained using unlabeled images of biological samples.

19. The system of claim 2 , wherein the one or more live biological cells are healthy cells.

20. The system of claim 2 , wherein the one or more live biological cells are diseased cells.

21. The system of claim 2 , wherein the one or more programs, when executed by the one or more processors, cause the system to, prior to obtaining a first set of one or more images:

apply a perturbation and/or a therapeutic agent to the cell culture.

22. The system of claim 21 , wherein the perturbation and/or the therapeutic agent is a chemical treatment, a genetic treatment, or any combination thereof.

23. A non-transitory computer-readable storage medium storing one or more programs for modeling a characteristic of interest of a cell culture comprising one or more live mammalian cells, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device having a display, cause the electronic device to perform operations comprising:

obtaining a first set of one or more images capturing the cell culture at a first time point;

inputting the first set of one or more images into a trained machine-learning model to obtain a first set of embedding vectors representing positional and morphological characteristics in particular cellular substructures of the cell culture;

evaluating the characteristic of interest of the cell culture corresponding to the first time point based on the first set of embedding vectors to obtain a first set of one or more values;

obtaining a second set of one or more images capturing the cell culture at a second time point;

inputting the second set one or more images into the trained machine-learning model to obtain a second set of embedding vectors representing positional and morphological characteristics in particular cellular substructures of the cell culture;

evaluating the characteristic of interest of the cell culture corresponding to the second time point based on the second set of embedding vectors to obtain a second set of one or more values; and

determining, from the first set of one or more values and the second set of one or more values, a change of the characteristic of interest in the cell culture.

24. A method of modeling a characteristic of interest of a cell culture comprising one or more live mammalian cells, comprising:

obtaining a first set of one or more images capturing the cell culture at a first time point;

inputting the first set of one or more images into a trained machine-learning model to obtain a first set of embedding vectors representing positional and morphological characteristics in particular cellular substructures of the cell culture;

evaluating the characteristic of interest of the cell culture corresponding to the first time point based on the first set of embedding vectors to obtain a first set of one or more values;

obtaining a second set of one or more images capturing the cell culture at a second time point;

inputting the second set one or more images into the trained machine-learning model to obtain a second set of embedding vectors representing positional and morphological characteristics in particular cellular substructures of the cell culture;

evaluating the characteristic of interest of the cell culture corresponding to the second time point based on the second set of embedding vectors to obtain a second set of one or more values; and

determining, from the first set of one or more values and the second set of one or more values, a change of the characteristic of interest in the cell culture.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 4, 2023
From: MARIE-NELLY, HERVÉ; VELAYUTHAM, JEEVA; PHILLIPS, ZACHARY; TU, SHENGJIANG
To: INSITRO, INC.
Reel/Frame 065755/0767 →
Continuity (4)
Division 18111405 · Feb 17, 2023
Continuation PCTUS2022080200 · Nov 19, 2022
Provisional Application 63281536 · Nov 19, 2021
Related Publication 20240104734A1 · Mar 28, 2024
References Cited (61)
US 10467754B1 · Ando et al. · 2019 [cited by applicant]
US 11423256B2 · Marie-Nelly et al. · 2022 [cited by applicant]
US 11875506B1 · Marie-Nelly et al. · 2024 [cited by applicant]
US 11978206B2 · Marie-Nelly et al. · 2024 [cited by applicant]
US 12002559B2 · Casale et al. · 2024 [cited by applicant]
US 20080201083A1 · Hata · 2008 [cited by examiner]
US 20170204359A1 · Ando · 2017 [cited by examiner]
US 20190295721A1 · Madabhushi et al. · 2019 [cited by applicant]
US 20190369098A1 · Hedge et al. · 2019 [cited by applicant]
US 20190371471A1 · Tan et al. · 2019 [cited by applicant]
US 20200005461A1 · Yip · 2020 [cited by applicant]
US 20200105413A1 · Madimirova et al. · 2020 [cited by applicant]
US 20200258223A1 · Yip et al. · 2020 [cited by applicant]
US 20200388287A1 · Anushiravani et al. · 2020 [cited by applicant]
US 20210172931A1 · Larsen et al. · 2021 [cited by applicant]
US 20210200989A1 · Courtiol et al. · 2021 [cited by applicant]
US 20210210205A1 · Drake et al. · 2021 [cited by applicant]
US 20210256699A1 · Wainrib et al. · 2021 [cited by applicant]
US 20210271847A1 · Courtiol et al. · 2021 [cited by applicant]
US 20210374553A1 · Li et al. · 2021 [cited by applicant]
US 20220059240A1 · Schaeffer et al. · 2022 [cited by applicant]
US 20220261668A1 · Stumpe et al. · 2022 [cited by applicant]
US 20220292674A1 · Braman et al. · 2022 [cited by applicant]
US 20220367053A1 · Mahmood et al. · 2022 [cited by applicant]
US 20230026189A1 · Kamato · 2023 [cited by examiner]
US 20230036156A1 · Ho · 2023 [cited by examiner]
US 20230142909A1 · Zhao · 2023 [cited by applicant]
US 20230154627A1 · Irving et al. · 2023 [cited by applicant]
US 20230245477A1 · Rothrock et al. · 2023 [cited by applicant]
US 20230360758A1 · Casale et al. · 2023 [cited by applicant]
US 20240119593A1 · Marie-Nelly et al. · 2024 [cited by applicant]
US 20240273718A1 · Probert et al. · 2024 [cited by applicant]
US 20240274254A1 · Casale et al. · 2024 [cited by applicant]
US 20240274255A1 · Casale et al. · 2024 [cited by applicant]
WO WO2010082096A2 · 2010 [cited by applicant]
WO WO2020113237A1 · 2020 [cited by applicant]
WO WO2023023507A1 · 2023 [cited by applicant]
Goldsborough, Peter, et al. “CytoGAN: generative modeling of cell images.” BioRxiv (2017): 227645. (Year: 2017). [cited by examiner]
Ubbens, Jordan, et al. “Latent space phenotyping: automatic image-based phenotyping for treatment studies.” Plant Phenomics (2020). (Year: 2020). [cited by examiner]
Galton, (1886). “Regression Towards Mediocrity in Hereditary Stature,” The Journal of the Anthropological Institute of Great Britain and Ireland, 15:246-263. [cited by applicant]
International Search Report and Written Opinion received for International Patent Application No. PCT/US2022/080200 mailed on May 23, 2023, 20 pages. [cited by applicant]
Wells et al., (2009). “Phase contrast microscopy analysis of breast tissue: differences in benign vs. malignant epithelium and stroma,” Anal Quant Cytol Histol., 31(4):197-207, 18 pages. [cited by applicant]
Taylor-Weiner et al., (2021). “A machine learning approach enables quantitative measurement of liver histology and disease monitoring in NASH,” Hepatology, 74(1):133-147. [cited by applicant]
Arslan et al., (2022). “Deep learning can predict multi-omic biomarkers from routine pathology images: A systematic large-scale study,” BioRxiv, 477189, 44 pages. [cited by applicant]
Courtiol et al., (2019). “Deep learning-based classification of mesothelioma improves prediction of patient outcome,” Nature Medicine, 25(10):1519-1525. [cited by applicant]
Courtiol et al., (2020). “Classification And Disease Localization In Histopathology Using Only Global Labels: A Weakly-Supervised Approach,” arXiv, 1802.02212, 13 pages. [cited by applicant]
De Jong et al., (2021). “Towards Realizing the Vision of Precision Medicine: AI Based Prediction of Clinical Drug Response Authors,” Brain, 144:1738-1750. [cited by applicant]
Ingale et al., (2023). “Prediction of met overexpression in non-small cell lung adenocarcinomas from hematoxylin and eosin images,” arXiv, 2310.07682, 45 pages. [cited by applicant]
International Search Report and Written Opinion received for International Patent Application No. PCT/US2022/075006 mailed on Jan. 27, 2023, 19 pages. [cited by applicant]
Kopf et al., (2021). “Latent representation learning in biology and translational medicine,” Patterns, 2(3):100198, 15 pages. [cited by applicant]
Liu et al., (2018). “An integrated toga pan-cancer clinical data resource to drive high-quality survival outcome analytics,” Cell, 173(2):400-416, 29 pages. [cited by applicant]
Amaro et al., (2021). “A Machine Learning Approach Enables Quantitative Measurement of Liver Histology and Disease Monitoring in NASH,” Hepatology, 74(1):133-147. [cited by applicant]
Chao et al., (2021). “MAPS: machine-assisted phenotype scoring enables rapid functional assessment of genetic variants by high-content microscopy,” BMC Bioinformatics, 22:202, 19 pages. [cited by applicant]
Clark et al., (2011). “Analysis of efficacy and side effects in CATIE demonstrates drug response subgroups and potential for personalized medicine,” Schizophrenia Research, 132(2-3):114-120, 15 pages. [cited by applicant]
Han et al., (2019). “GCN-MF: disease-gene association identification by graph convolutional networks and matrix factorization,” Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data M… [cited by applicant]
International Search Report and Written Opinion received for International Patent Application No. PCT/US2024/015870 mailed on Aug. 5, 2024, 36 pages. [cited by applicant]
Invitation to Pay Additional Fees, Partial Search Report and Provisional Opinion for International Patent Application No. PCT/US2024/015870 mailed on Jun. 14, 2024, 25 pages. [cited by applicant]
Li et al., (2015). “Identification of type 2 diabetes subgroups through topological analysis of patient similarity,” Science Translational Medicine 7(311):311ra174, 17 pages. [cited by applicant]
Li et al., (2021). “Deep learning-based predictive biomarker of pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer,” J Transl Med, 19:348, 13 pages. [cited by applicant]
Schulam et al., (2016). “Disease trajectory maps,” Advances in Neural Information Processing Systems, 29, 9 pages. [cited by applicant]
Sui et al., (2022). “A deep learning model designed for Raman spectroscopy with a novel hyperparameter optimization method,” Spectrochim Acta A Mol Biomol Spectrosc, 280:121560, 9 pages. [cited by applicant]
Cited By (1)
US 12,462,386