Methods and systems for enhancing nucleic acid sequencing quality in high-throughput sequencing processes with machine learning
This disclosure provides an improved filtering techniques that can provide higher sequencing accuracy for processing high-throughput sequencing data. The filtering structure uses a hierarchical network structure comprising one or more network blocks for obtaining high quality sequences. Each network block comprises a base network module (or simply a base network) and a sequence filter. The base network generates one or more sequencing quality indicators. The sequencing quality indicators can represent qualities of accuracy of the basecalling of individual bases in a sequence, the quality of one or more sequences individually, or the quality of a group of sequences as a whole. The sequence filter generates the filtered results based on the various filtering strategies based on the one or more sequencing quality indicators.
1 . A computer-implemented method for enhanced training of one or more neural-network models to enhance the quality of basecalling results obtained by a high-throughput process for sequencing nucleic acid molecules using the one or more neural-network models, the method being performed by one or more computing devices, each comprising a processor and a memory, the method comprising:
obtaining, by the one or more computing devices, training data comprising fluorescence signal data corresponding to a plurality of nucleic acid sequences processed by a nucleic acid sequencing system;
processing, by the one or more computing devices, the training data using the one or more neural-network models thereby generating one or both of a confidence level and a basecalling prediction for each base in the plurality of nucleic acid sequences;
filtering, by the one or more computing devices, the training data to determine a filtered set of training data comprising fluorescence signal data corresponding to high-quality sequences in the plurality of nucleic acid sequences, wherein filtering comprises identifying the high-quality sequences using one or both of the confidence level and the basecalling prediction for each base in the plurality of sequences; and
using, by the one or more computing devices, the filtered set of training data, to train one or more neural-network models to generate basecalling predictions;
wherein the one or more neural-network based models are included in one or more network blocks comprising a first network block and a second network block, and
wherein generating one or both of a confidence level and a basecalling prediction comprises:
determining one or more first passing indices and a dataset quality index using one or more neural-network based models of the first network block,
providing the first passing indices to one or more neural-network based models of the second network block,
providing the dataset quality index to a sequence filter of the second network block; and
determining, based on the one or more neural-network based models of the second network block, confidence level scores associated with bases included in sequences received by the second network block.
2 . The method of claim 1 , wherein identifying a high-quality sequence of the high-quality sequences comprises, with respect to a sequence of the plurality of nucleic acid sequences:
comparing the confidence level of each base in the sequence to a confidence level threshold to determine if each base is a high-quality base;
counting a number of high-quality bases in the sequence; and
determining, based on the number of high-quality bases in the sequence, if the sequence is a high-quality sequence.
3 . The method of claim 1 , wherein the training data comprises labeled basecalling data, and wherein identifying a high-quality sequence of the high-quality sequences comprises, with respect to a sequence of the plurality of nucleic acid sequences:
determining, based on labeled basecalling data, if bases included in the sequence are correctly classified in the labeled basecalling data;
counting a number of correctly-classified bases in the sequence; and
determining, based on the number of correctly-classified bases in the sequence, if the sequence is a high-quality sequence to determine if the sequence passes filtering.
4 . The method of claim 3 , further comprising:
obtaining a retraining dataset including only sequences that passed filtering;
retraining the one or more neural-network models using the retraining dataset to obtain one or more retained neural-network models,
re-determining confidence level scores and basecalling predictions based on the one or more retrained neural-network models; and
filtering the retraining data based on one or more re-determined confidence level scores and basecalling predictions.
5 . The method of claim 1 , wherein filtering the first group of sequences of nucleic acid comprises:
performing base calling quality filtering (BCQF) based on the dataset quality index and the confidence level scores associated with bases included in the sequences received by the second network block;
generating, based on BCQF results, second passing indices representing quality of the sequences received by the second network block; and
filtering the sequences received by the second network block based on the second passing indices to obtain the second group of sequences of nucleic acid.
6 . A system for enhanced training of one or more neural network models for enhancing quality of basecalling results obtained by a high-throughput process for sequencing nucleic acid molecules, the system comprises:
one or more processors of at least one computing device;
a communication interface; and
a memory storing one or more instructions, when executed by the one or more processors, cause the one or more processors to perform steps comprising:
obtaining, by the one or more computing devices, training data comprising fluorescence signal data corresponding to a plurality of nucleic acid sequences processed by a nucleic acid sequencing system;
processing, by the one or more computing devices, the training data using the one or more neural networks thereby generating one or both of a confidence level and a basecalling prediction for each base in the plurality of nucleic acid sequences;
filtering, by the one or more computing devices, the training data to determine a filtered set of training data comprising fluorescence signal data corresponding to high-quality sequences in the plurality of nucleic acid sequences, wherein filtering comprises identifying the high-quality sequences using one or both of the confidence level and the basecalling prediction for each base in the plurality of sequences; and
using, by the one or more computing devices, the filtered set of training data, to train one or more neural-network models to generate basecalling predictions;
wherein the one or more neural-network based models are included in one or more network blocks comprising a first network block and a second network block, and
wherein generating one or both of a confidence level and a basecalling prediction comprises:
determining one or more first passing indices and a dataset quality index using one or more neural-network based models of the first network block,
providing the first passing indices to one or more neural-network based models of the second network block,
providing the dataset quality index to a sequence filter of the second network block; and
determining, based on the one or more neural-network based models of the second network block, confidence level scores associated with bases included in sequences received by the second network block.
7 . The system of claim 6 , wherein the one or more of the plurality of sequencing quality indicators comprises at least one of:
sequence quality filtering network (SQFN) passing indices;
dataset quality indices; and
confidence level scores.
8 . The system of claim 7 , wherein the one or more neural-network based models are a part of a single network block configured to determine the confidence level scores and the first group of sequences of nucleic acid.
9 . The system of claim 7 , wherein:
at least one of the one or more neural-network based models is included in a backbone network block,
the one or more neural-network based models include a first decoder and a second decoder, the first decoder and the second decoder being configured to receive outputs from the backbone network block,
the first decoder is configured to determine one or more confidence level scores, and
the second decoder is configured to determine the first group of sequences of nucleic acid.
10 . The system of claim 6 , wherein the one or more neural-network based models are a part of a single network block configured to determine one or more SQFN passing indices and one or more dataset quality indices.
11 . The system of claim 7 , wherein:
the first network block is being configured to determine SQFN passing indices and dataset quality indices;
the one or more neural-network based models include a first decoder and a second decoder, the first decoder and the second decoder being configured to receive outputs from a backbone network block,
the first decoder is configured to determine one or more confidence level scores, and
the second decoder is configured to determine the first group of sequences of nucleic acid.
12 . The system of claim 7 , wherein:
at least one of the one or more neural-network based models is included in a backbone network block,
the one or more neural-network based models include a first decoder, a second decoder, and a third decoder, the first, second, and third decoders being configured to receive outputs from the backbone network block,
the first decoder is configured to determine the SQFN passing indices and the dataset quality indices,
the second decoder is configured to determine one or more confidence level scores, and
the third decoder is configured to determine the first group of sequences of nucleic acid.
13 . The system of claim 6 , wherein:
the one or more neural-network based models are included in one or more network blocks comprising the first network block, the second network block, and a third network block;
the first network block is configured to determine SQFN passing indices and dataset quality indices;
the second network block is configured to determine confidence level scores; and
the third network block is configured to determine the first group of sequences of nucleic acid.
14 . A non-transitory computer readable medium comprising a memory storing one or more instructions which, when executed by one or more processors of at least one computing device, cause the at least one computing device to perform processing comprising:
obtaining, by the one or more computing devices, training data comprising fluorescence signal data corresponding to a plurality of nucleic acid sequences processed by a nucleic acid sequencing system;
processing, by the one or more computing devices, the training data using one or more neural networks to generate one or both of a confidence level and a basecalling prediction for each base in the plurality of nucleic acid sequences;
filtering, by the one or more computing devices, the training data to determine a filtered set of training data comprising fluorescence signal data corresponding to high-quality sequences in the plurality of nucleic acid sequences, wherein filtering comprises identifying the high-quality sequences using one or both of the confidence level and the basecalling prediction for each base in the plurality of sequences; and
using, by the one or more computing devices, the filtered set of training data, to train one or more neural-network models to generate basecalling predictions;
wherein the one or more neural-network based models are included in one or more network blocks comprising a first network block and a second network block, and
wherein generating one or both of a confidence level and a basecalling prediction comprises:
determining one or more first passing indices and a dataset quality index using one or more neural-network based models of the first network block,
providing the first passing indices to one or more neural-network based models of the second network block,
providing the dataset quality index to a sequence filter of the second network block; and
determining, based on the one or more neural-network based models of the second network block, confidence level scores associated with bases included in sequences received by the second network block.
15 . The non-transitory computer readable medium of claim 14 wherein identifying a high-quality sequence of the high-quality sequences comprises, with respect to a sequence of the plurality of nucleic acid sequences:
comparing the confidence level of each base in the sequence to a confidence level threshold to determine if each base is a high-quality base;
counting a number of high-quality bases in the sequence; and
determining, based on the number of high-quality bases in the sequence, if the sequence is a high-quality sequence.
16 . The non-transitory computer readable medium of claim 14 wherein the training data comprises labeled basecalling data, and wherein identifying a high-quality sequence of the high-quality sequences comprises, with respect to a sequence of the plurality of nucleic acid sequences:
determining, based on labeled basecalling data, if bases included in the sequence are correctly classified in the labeled basecalling data;
counting a number of correctly-classified bases in the sequence; and
determining, based on the number of correctly-classified bases in the sequence, if the sequence is a high-quality sequence to determine if the sequence passes filtering.
17 . The non-transitory computer readable medium of claim 16 wherein the one or more instructions cause the at least one computing device to perform processing comprising:
obtaining a retraining dataset including only sequences that passed filtering;
retraining the one or more neural network models using the retraining dataset to obtain one or more retained neural-network models;
re-determining confidence level scores and basecalling predictions based on the one or more retrained neural-network models; and
filtering the retraining data based on one or more re-determined confidence level scores and basecalling predictions.