Merging alignment and sorting to optimize computer operations for gene sequencing pipeline
In accordance with embodiments, a processing unit performs alignment of a short read (SR) against a reference sequence. The reference sequence comprises a first sequence of base pairs (bps). The SR comprises a second sequence of bps. The processing unit determines whether the SR is aligned. If the SR is not aligned, the processing unit receives the next SR and processes the next SR by repeating. If the SR is aligned, in response to the determination that the SR is aligned with the reference sequence at a first position in the reference sequence, the processing unit generates an SR metadata entry corresponding to the SR. The processing unit inserts the SR metadata entry to a linked list in a SR metadata collection. The position of the linked list in the SR metadata collection corresponds to the first position of the reference sequence where the SR is aligned.
1. A method, the method comprising:
performing, by at least one processing unit, alignment of a short read (SR) against a reference sequence, wherein the reference sequence comprises a first sequence of base pairs (bps), and the SR comprises a second sequence of bps;
in response to determination that the SR is aligned with the reference sequence at a first position in the reference sequence:
generating, by the at least one processing unit, an SR metadata entry corresponding to the SR; and
inserting, by the at least one processing unit, the SR metadata entry to a linked list in a SR metadata collection, wherein the SR metadata collection includes an array of linked lists, an array index of an array element for the linked list in the array of the SR metadata collection corresponding to the first position of the reference sequence where the SR is aligned, wherein the linked list comprises a plurality of SR metadata entries after the inserting, wherein SRs corresponding to the plurality of SR metadata entries in the linked list are all aligned at the first position of the reference sequence, and wherein a size of the array is at least a quantity of bp positions in the reference sequence;
determining, by the at least one processing unit, that alignment of all of a plurality of SRs stored in a fastq file against the reference sequence has been performed, wherein corresponding SR metadata entries of aligned SRs of the plurality of SRs have been automatically sorted by aligned positions of the aligned SRs when each of the corresponding SR metadata entries was inserted to a corresponding linked list of the array, the corresponding linked list indexed at a corresponding array index based on a corresponding alignment position; and
generating, by the at least one processing unit, a binary alignment map (BAM) file based on the array of linked lists in the SR metadata collection, the BAM file being a binary presentation of the aligned SRs sorted by the aligned positions, and the BAM file being an input directly fed into a Mark Duplicates sub-system.
2. The method of claim 1 , wherein the SR metadata entry comprises an identifier (ID) of the SR.
3. The method of claim 2 , further comprising:
before the performing the alignment of the SR, receiving, by the at least one processing unit, the SR from the plurality of SRs stored in the fastq file, wherein the ID of the SR comprises an index of the SR in the plurality of SRs.
4. The method of claim 2 , the SR metadata entry further comprises alignment results generated by the performing the alignment of the SR.
5. The method of claim 4 , wherein the alignment results comprise a concise idiosyncratic gapped alignment report (CIGAR), a Flag, and a mark duplicate (MD) notation.
6. The method of claim 1 , wherein each of the plurality of SRs includes at least 100 bps.
7. A computer device comprising:
at least one processor unit;
a non-transitory computer readable storage medium storing programming for execution by the at least one processor unit, the programming including instructions to cause the computer device to perform operations including:
performing alignment of a short read (SR) against a reference sequence, wherein the reference sequence comprises a first sequence of base pairs (bps), and the SR comprises a second sequence of bps;
in response to determination that the SR is aligned with the reference sequence at a first position in the reference sequence:
generating an SR metadata entry corresponding to the SR; and
inserting the SR metadata entry to a linked list in a SR metadata collection, wherein the SR metadata collection includes an array of linked lists, an array index of an array element for the linked list in the array of the SR metadata collection corresponding to the first position of the reference sequence where the SR is aligned, wherein the linked list comprises a plurality of SR metadata entries after inserting the SR metadata entry, wherein SRs corresponding to the plurality of SR metadata entries in the linked list are all aligned at the first position of the reference sequence, and wherein a size of the array is at least a quantity of bp positions in the reference sequence;
determining that alignment of all of a plurality of SRs stored in a fastq file against the reference sequence has been performed, wherein corresponding SR metadata entries of aligned SRs of the plurality of SRs have been automatically sorted by aligned positions of the aligned SRs when each of the corresponding SR metadata entries was inserted to a corresponding linked list of the array, the corresponding linked list indexed at a corresponding array index based on a corresponding alignment position; and
generating a binary alignment map (BAM) file based on the array of linked lists in the SR metadata collection, the BAM file being a binary presentation of the aligned SRs sorted by the aligned positions, and the BAM file being an input directly fed into a Mark Duplicates sub-system.
8. The computer device of claim 7 , wherein the SR metadata entry comprises an identifier (ID) of the SR.
9. The computer device of claim 8 , operations further comprising:
before the alignment of the SR is performed, receiving the SR from the plurality of SRs stored in the fastq file, wherein the ID of the SR comprises an index of the SR in the plurality of SRs.
10. The computer device of claim 8 , SR metadata entry further comprises alignment results generated by performing the alignment of the SR.
11. The computer device of claim 10 , wherein the alignment results comprise a concise idiosyncratic gapped alignment report (CIGAR), a Flag, and a mark duplicate (MD) notation.
12. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one processor unit, cause the at least one processor unit to perform operations, the operations comprising:
performing alignment of a short read (SR) against a reference sequence, wherein the reference sequence comprises a first sequence of base pairs (bps), and the SR comprises a second sequence of bps;
in response to determination that the SR is aligned with the reference sequence at a first position in the reference sequence:
generating an SR metadata entry corresponding to the SR; and
inserting the SR metadata entry to a linked list in a SR metadata collection, wherein the SR metadata collection includes an array of linked lists, an array index of an array element for the linked list in the array of the SR metadata collection corresponding to the first position of the reference sequence where the SR is aligned, wherein the linked list comprises a plurality of SR metadata entries after the inserting, wherein SRs corresponding to the plurality of SR metadata entries in the linked list are all aligned at the first position of the reference sequence, and wherein a size of the array is at least a quantity of bp positions in the reference sequence;
determining that alignment of all of a plurality of SRs stored in a fastq file against the reference sequence has been performed, wherein corresponding SR metadata entries of aligned SRs of the plurality of SRs have been automatically sorted by aligned positions of the aligned SRs when each of the corresponding SR metadata entries was inserted to a corresponding linked list of the array, the corresponding linked list indexed at a corresponding array index based on a corresponding alignment position; and
generating a binary alignment map (BAM) file based on the array of linked lists in the SR metadata collection, the BAM file being a binary presentation of the aligned SRs sorted by the aligned positions, and the BAM file being an input directly fed into a Mark Duplicates sub-system.
13. The non-transitory computer-readable medium of claim 12 , wherein the SR metadata entry comprises an identifier (ID) of the SR.
14. The non-transitory computer-readable medium of claim 13 , the operations further comprising:
before the performing the alignment of the SR, receiving the SR from the plurality of SRs stored in the fastq file, wherein the ID of the SR comprises an index of the SR in the plurality of SRs.
15. The non-transitory computer-readable medium of claim 13 , the SR metadata entry further comprises alignment results generated by the performing the alignment of the SR.
16. The non-transitory computer-readable medium of claim 15 , wherein the alignment results comprise a concise idiosyncratic gapped alignment report (CIGAR), a Flag, and a mark duplicate (MD) notation.