Quality score compression for improving downstream genotyping accuracy
This disclosure provides for a highly-efficient and scalable compression tool that compresses quality scores, preferably by capitalizing on sequence redundancy. In one embodiment, compression is achieved by smoothing a large fraction of quality score values based on k-mer neighborhood of their corresponding positions in read sequences. The approach exploits the intuition that any divergent base in a k-mer likely corresponds to either a single-nucleotide polymorphism (SNP) or sequencing error; thus, a preferred approach is to only preserve quality scores for probable variant locations and compress quality scores of concordant bases, preferably by resetting them to a default value. By viewing individual read datasets through the lens of k-mer frequencies in a corpus of reads, the approach herein ensures that compression “lossiness” does not affect accuracy in a deleterious way.
1. A method of processing genomic data in a computing machine, the computing machine having a processor and a cache, comprising:
receiving, as a dictionary, a set of data comprising commonly-occurring k-mers extracted from a read dataset;
compressing NGS quality scores in a given read dataset by identifying k-mers from each read within a given mismatch distance from other k-mers in the dictionary to generate compressed quality scores; and
providing the compressed quality scores to a genotyping analysis pipeline;
wherein processing and cache efficiency in the computing machine are improved by compressing the quality scores using a locality sensitive hashing function.
2. The method as described in claim 1 wherein the dictionary has the property that any given read dataset is covered from k-mers within a given Hamming distance.
3. The method as described in claim 2 wherein the step of compressing quality scores comprises:
setting to a default value any quality score value corresponding to a position that is concordant with at least one supporting k-mer; and
saving the quality score value at any position that is divergent from all supporting k-mers.
4. The method as described in claim 1 wherein the read dataset is a dataset associated with a population.
5. The method as described in claim 1 further including generating the dictionary.
6. The method as described in claim 5 wherein the dictionary is generated using a counting algorithm and a collection of hash buckets to further improve cache efficiency of the computing machine.
7. The method as described in claim 1 wherein compressing quality scores includes:
breaking a read sequence in the read dataset into overlapping k-mers;
with respect to the read sequence, identifying k-mers in the dictionary that are within a given mismatch distance from supporting k-mers; and
annotating a position different from a dictionary k-mer as a possible variant unless covered by a dictionary k-mer corresponding to a different supporting k-mer.
8. The method as described in claim 7 wherein when two dictionary k-mers correspond to the same supporting k-mer, all mismatches are preserved unless a mismatch portion is covered by a dictionary k-mer corresponding to a different supporting k-mer.
9. The method as described in claim 8 further including;
smoothing the quality scores; and
setting all positions exhibiting a given quality value to a default value while retaining scores of uncovered and possible variant loci.
10. The method as described in claim 1 further including performing an additional compression of the compressed quality scores.
11. The method as described in claim 1 wherein compression of the NGS quality scores is lossy.
12. Apparatus to facilitate processing genomic data, the apparatus being a computing machine, comprising:
one or more processors and a cache;
computer memory storing computer program instructions executed by the one or more processors, the instructions configured to:
generate, as a dictionary, a set of data comprising commonly-occurring k-mers extracted from a read dataset; and
compress quality scores in any given read dataset by identifying k-mers from each read within a given mismatch distance from other k-mers in the dictionary, thereby generating compressed quality scores;
wherein processing and cache efficiency in the computing machine is improved by compressing the quality scores using a locality sensitive hashing function.