IP Library Granted Patent US 12,694,522
Granted Patent B2
US 12,694,522 · App. 18/463,321 · Granted Jul 28, 2026

Systems and methods of correcting batch effect in biological images

Inventors: Chi Kiu Michelle Lo (Hong Kong, HK); Man Dik Dickson Siu (Hong Kong, HK); Kin Man Kevin Tsia (Hong Kong, HK)
Assignee: The University of Hong Kong
G06T7/0012G06T5/77G06V20/698G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,694,522
App. No.
18/463,321
Filed
Sep 8, 2023
Granted
Jul 28, 2026
Kind
B2
Art Unit
2665
USPC
382/128
Abstract

A system for the rectification of batch-induced distortions and the extraction of phenotypic attributes from biological images is presented. This system encompasses a deep-learning generative adversarial network model, expertly tailored to converse with and modify the contrast of input images, resulting in the creation of output images. A discriminator, operating as a feedback mechanism, discriminates the output image against the input image, ensuring precise image contrast conversion. Integral to this system is a morphology distillator, having a dual classifier framework comprising a batch classifier and a cell type/state classifier. This distillator adeptly identifies phenotypic characteristics and batch-related disparities, subsequently eradicating these batch variations from the output image. The resultant augmentation enriches the cellular information within the image.

Claims (18)

1 . A system for correcting batch effects by removing systematic variations in biological images from equipment, reagents, sample preparation, or imaging conditions and extracting phenotypic features in biological images, comprising:

a deep-learning generative adversarial network model trained to determine an underlying structure of an input image and remove image artifacts, wherein the deep-learning generative adversarial network model comprises an autoencoder architecture for:

extracting morphological features of the input image;

reconstructing an output image; and

pretraining components of the deep-learning generative adversarial network model for image reconstruction;

a discriminator that classifies the output image by comparing it with the input image for identifying and rectifying residual batch-related variations, serving as a part of a concurrent refinement cycle to achieve accurate image reconstruction and contrast conversion; and

a morphology distillator comprising a batch classifier and a cell type/state classifier for recognizing phenotypic features and batch variations and removing the batch variations in the output image through a feedback mechanism in the concurrent refinement cycle, wherein the batch classifier employs domain adaptation to mitigate the influence of batch effects while retaining biologically significant cellular information identified by the cell type/state classifier;

wherein the deep-learning generative adversarial network model is retrained by the concurrent refinement cycle with parameters of the batch classifier and the cell-type/state classifier being frozen during batch effect correction; and

wherein the frozen parameters of the batch classifier and the cell-type/state classifier are used to update parameters of the autoencoder architecture and the discriminator, thereby guiding the batch effect correction while preserving the biologically significant cellular information and ensuring image prediction accuracy.

2 . The system of claim 1 , wherein the batch classifier and the cell type/state classifier are pretrained by the autoencoder to identify batch and cell information.

3 . The system of claim 1 , wherein the deep-learning generative adversarial network model can be optionally pretrained with an image contrast translation functionality, so as to perform an image contrast conversion for providing an augmented output image.

4 . The system of claim 1 , wherein the morphology distillator is integrated into a bottleneck region and the output layer of the deep-learning generative adversarial network model, establishing a feedback loop to separate the phenotypic features and batch variations.

5 . The system of claim 1 , wherein the cell type/state classifier within the morphology distillator employs a self-supervised learning mechanism to capture and distinguish cellular phenotypes to provide automated cellular classification.

6 . The system of claim 1 , wherein the batch distillation potency of the deep-learning generative adversarial network model can be elevated by a process of segmenting the batch classifier within the morphology distillator at a bottleneck stage into multiple miniature classifiers, and/or a process of periodic retraining at a predetermined interval.

7 . The system of claim 1 , wherein the discriminator employs a convolutional neural network to assess and guide the contrast adjustment process for translation of image contrast.

8 . The system of claim 1 , wherein the morphology distillator incorporates a self-attention mechanism to selectively focus on informative regions within the input image, to extract phenotypic features from the input image.

9 . The system of claim 1 , wherein the training to determine an underlying structure of an input image and remove image artifacts includes training on diverse and heterogeneous biological datasets to learn different imaging modalities, cell types, and experimental conditions.

10 . The system of claim 1 , wherein the system is implemented in a cloud-based platform for remote access to batch correction and phenotypic feature extraction.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2023
From: LO, CHI KIU MICHELLE; SIU, MAN DIK DICKSON; TSIA, KIN MAN KEVIN
To: THE UNIVERSITY OF HONG KONG
Reel/Frame 064945/0715 →
Continuity (2)
Provisional Application 63410291 · Sep 27, 2022
Related Publication 20240104730A1 · Mar 28, 2024
References Cited (40)
US 11164312B2 · Saltz · 2021 [cited by examiner]
US 11501429B2 · Stamatoyannopoulos · 2022 [cited by examiner]
US 11908548B2 · Dutta · 2024 [cited by examiner]
US 12175670B2 · Courtiol · 2024 [cited by examiner]
US 20150110381A1 · Parvin · 2015 [cited by examiner]
US 20170372193A1 · Mailhe et al. · 2017 [cited by applicant]
US 20190287761A1 · Schoenmakers et al. · 2019 [cited by applicant]
US 20210097686A1 · Takeuchi · 2021 [cited by examiner]
US 20210264214A1 · Ozcan et al. · 2021 [cited by applicant]
US 20230059717A1 · Madabhushi et al. · 2023 [cited by applicant]
AU 2016211333A1 · 2017 [cited by examiner]
CA 3066775A1 · 2019 [cited by examiner]
CA 3104851A1 · 2020 [cited by examiner]
CN 115115803A · 2022 [cited by examiner]
WO WO2019018693A2 · 2019 [cited by examiner]
WO WO2020216324A1 · 2020 [cited by examiner]
WO WO2021198112A1 · 2021 [cited by examiner]
WO WO2021237117A1 · 2021 [cited by examiner]
WO WO2022076654A1 · 2022 [cited by examiner]
Witmer, Adam, and Bir Bhanu. “Generative adversarial networks for morphological-temporal classification of stem cell images.” Sensors 22.1 (2021): 206. (Year: 2021). [cited by examiner]
Min, Seonwoo, Byunghan Lee, and Sungroh Yoon. “Deep learning in bioinformatics.” Briefings in bioinformatics 18.5 (2017): 851-869. (Year: 2017). [cited by examiner]
Uhler, Caroline, and G. V. Shivashankar. “Machine learning approaches to single-cell data integration and translation.” Proceedings of the IEEE 110.5 (2022): 557-576. (Year: 2022). [cited by examiner]
Tran, Hoa Thi Nhu, et al. “A benchmark of batch-effect correction methods for single-cell RNA sequencing data.” Genome biology 21.1 (2020): 1-32. [cited by applicant]
Isola, Phillip, et al. “Image-to-image translation with conditional adversarial networks.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, 5967-5976. [cited by applicant]
Johnson, W. Evan, Cheng Li, and Ariel Rabinovic. “Adjusting batch effects in microarray expression data using empirical Bayes methods.” Biostatistics 8.1 (2007): 118-127. [cited by applicant]
Haghverdi, Laleh, et al. “Batch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors.” Nature biotechnology 36.5 (2018): 421-427. [cited by applicant]
Korsunsky, Ilya, et al. “Fast, sensitive and accurate integration of single-cell data with Harmony.” Nature methods 16.12 (2019): 1289-1296. [cited by applicant]
Lotfollahi, Mohammad, F. Alexander Wolf, and Fabian J. Theis. “Generative modeling and latent space arithmetics predict single-cell perturbation response across cell types, studies and species.” bioRxiv (2018): 478503. [cited by applicant]
Leek, Jeffrey T., et al. “Tackling the widespread and critical impact of batch effects in high-throughput data.” Nature Reviews Genetics 11.10 (2010): 733-739. [cited by applicant]
Goh, Wilson Wen Bin, Wei Wang, and Limsoon Wong. “Why batch effects matter in omics data, and how to avoid them.” Trends in biotechnology 35.6 (2017): 498-507. [cited by applicant]
Ziegler, Slava, Sonja Sievers, and Herbert Waldmann. “Morphological profiling of small molecules.” Cell Chemical Biology 28.3 (2021): 300-319. [cited by applicant]
Lawson, Michael, and Johan Elf. “Imaging-based screens of pool-synthesized cell libraries.” Nature Methods 18.4 (2021): 358-365. [cited by applicant]
Phillip, Jude M., et al. “A robust unsupervised machine-learning method to quantify the morphological heterogeneity of cells and nuclei.” Nature protocols 16.2 (2021): 754-774. [cited by applicant]
Caicedo, Juan C., et al. “Data-analysis strategies for image-based cell profiling.” Nature methods 14.9 (2017): 849-863. [cited by applicant]
Mullard, Asher. “Machine learning brings cell imaging promises into focus.” Nature Reviews Drug Discovery 18.9 (2019): 653-656. [cited by applicant]
Rohban, Mohammad Hossein, et al. “Systematic morphological profiling of human gene and allele function via Cell Painting.” Elife 6 (2017). [cited by applicant]
Wu, Pei-Hsun, et al. “Single-cell morphology encodes metastatic potential.” Science Advances 6.4 (2020): eaaw6938. [cited by applicant]
Wesley Wei Qian et al., “Batch equalization with a generative adversarial network,” Bioinformatics, 2020, 36(26), 1875-1883. [cited by applicant]
Haochen Yan et al., “Virtual optofluidic time-stretch quantitative phase imaging,” APL Photonics, 2020, 5, 046103. [cited by applicant]
Dahye Kim et al., “Unsupervised feature elimination via generative adversarial networks: application to hair removal in melanoma classification,” IEEE Access, 2021, 9, 42610-42620. [cited by applicant]