Methods to compress, encrypt and retrieve genomic alignment data
A genomic data decoder may jointly compress and encrypt genomic data alignment information while preserving the privacy of sensitive genomic data elements at retrieval stage. Genomic data alignment information organized as a read-based alignment data stream may be transposed into a position-based alignment data stream. The position-based alignment information may been coded into a reference-based alignment data stream. The reference-based alignment data stream may be encrypted with a combination of order-preserving encryption of the genomic position information and symmetric encryption of the reference-based alignment differential data. Differential encoding and entropy coding schemes may further compress the reference-based alignment data stream. The resulting compressed and encrypted stream may be indexed and stored in a biobank storage unit. A genomic data decoder may efficiently retrieve, decrypt and decode a specific subset of the resulting compressed and encrypted stream without leaking information on the other genomic data subsets in the resulting stream.
1. A method to encode genomic data alignment information organized as a read-based alignment information data stream, comprising:
Transposing, with a processor, the read-based alignment information data stream into a position-based alignment information data stream,
wherein a character is a start marker for each short read in the position-based alignment information data stream, the start marker followed by metadata information regarding at least a nucleotide base identified at a position with an associated quality score;
Encoding, with a processor, the position-based alignment information data stream into a reference-based compressed position data stream; and
Encrypting, with a processor, the reference-based compressed position data stream into a compressed encrypted alignment data stream, including independently encrypting variant information for each row of a data structure in a storage that stores the compressed encrypted alignment data stream, providing privacy control of specific compressed encrypted alignment data within the stored compressed encrypted alignment data stream, the encrypting the reference-based compressed position data stream into a compressed encrypted alignment data stream comprising, first, an order-preserving encryption scheme, and second, encrypting sensitive information at each position,
wherein the method results in increased storage efficiency or faster genomic data queries.
2. The method of claim 1 ,
wherein encoding the position-based alignment information data stream into a reference-based compressed position data stream comprises differential encoding.
3. The method of claim 2 ,
wherein differential encoding comprises recording, for each position in the reference-based compressed position data stream, the alignment differences relative to the alignment reference sequence, and
wherein only the differences for each position with respect to the reference-based compressed position data stream are recorded.
4. The method of claim 1 ,
wherein encoding the position-based alignment information data stream into a reference-based compressed position data file further comprises entropy coding.
5. The method of claim 1 ,
wherein the order preserving encryption scheme is configured to retrieve resulting encrypted data for each row of the data structure without decrypting a whole block data.
6. The method of claim 1 ,
wherein encrypting the reference-based compression position data stream into a compressed encrypted alignment data stream comprises encrypting the position-based alignment information with a symmetric encryption scheme.
7. The method of claim 6 ,
wherein the symmetric encryption scheme is a stream cipher.
8. The method of claim 7 ,
wherein the symmetric encryption scheme is a block cipher operating in a stream cipher mode.
9. A method to retrieve genomic data alignment information from a compressed encrypted alignment data stream, recorded on a storage, comprising:
Receiving a genomic alignment range query [Pos1, Pos2] from a genomic data analysis system;
Retrieving from the storage, with a processor, the subset of the compressed encrypted alignment data stream corresponding to the genomic alignment range [Pos1, Pos2] in the compressed encrypted alignment data stream;
Decrypting, with a processor, the compressed encrypted alignment data stream into a reference-based compressed position data stream corresponding to the genomic alignment range [Pos1, Pos2], including independently decrypting variant information for each row of a data structure in the storage that stores the compressed encrypted alignment data stream, providing privacy control of specific compressed encrypted alignment data within the stored compressed encrypted alignment data stream; and
Decoding, with a processor, the reference-based compressed position data stream into a position-based alignment information data stream corresponding to the genomic alignment range [Pos1, Pos2],
wherein the method results in increased storage efficiency or faster genomic data queries, and
wherein decoding the reference-based compressed position data stream comprises retrieving a metadata information block and decoding the metadata information block in accordance with an encoding embodiment.
10. The method of claim 9 , further comprising:
reverse transposing, with a processor, the position-based alignment information data stream into a read-based alignment information data,
wherein a character is a start marker for each short read in the portion-based alignment information data stream, the start marker followed by metadata information regarding at least a nucleotide base identified at a position with an associated quality score.
11. The method of claim 9 ,
wherein retrieving the subset of the compressed encrypted alignment data stream for the genomic alignment range [Pos1, Pos2] comprises retrieving the symmetric encrypted data and the metadata stored in data blocks between the order-preserving encrypted position associated with Pos1 and the order-preserving encrypted position associated with Pos2.
12. The method of claim 11 ,
wherein decrypting the compressed encrypted alignment data stream into a reference-based compressed position data stream corresponding to the genomic alignment range [Pos1, Pos2] comprises symmetric decryption of the symmetric encrypted data between the order-preserving encrypted position associated with Pos1 and the order-preserving encrypted position associated with Pos2.
13. The method of claim 12 ,
wherein the symmetric decryption scheme is a stream decipher.
14. The method of claim 12 ,
wherein the symmetric decryption scheme is a block decipher operating in a stream decipher mode.
15. The method of claim 9 ,
wherein decoding the position-based alignment information data stream into reference-based compressed position data stream comprises entropy decoding.
16. The method of claim 9 ,
wherein decoding the position-based alignment information data stream into reference-based compressed position data stream comprises differential decoding.
17. The method of claim 1 ,
wherein encoding the position-based alignment information data stream into a reference-based compressed position data file further comprises text coding algorithms, and
wherein the reference-based compressed position data file is a compact binary reference-based compressed position data file.
18. The method of claim 1 ,
wherein encoding the position-based alignment information data stream into a reference-based compressed position data file further comprises variable length coding,
wherein the variable length coding is configured to compress differences found in reference-based compression, and
wherein the variable length coding is configured to compress differences found in mapping quality scores.
19. The method of claim 15 ,
wherein the entropy decoding is VLC decoding.
20. The method of claim 9 ,
wherein the encoding embodiment is a gunzip reverse algorithm, and
wherein decoding comprises concatenating the reference-based compressed position data stream, the position-based alignment information data stream, and the metadata information block to reconstruct the genomic data alignment information.