Packet prioritization and associated bandwidth and buffer management techniques for audio over IP
The present invention is directed to voice communication devices in which an audio stream is divided into a sequence of individual packets, each of which is routed via pathways that can vary depending on the availability of network resources. All embodiments of the invention rely on an acoustic prioritization agent that assigns a priority value to the packets. The priority value is based on factors such as whether the packet contains voice activity and the degree of acoustic similarity between this packet and adjacent packets in the sequence. A confidence level, associated with the priority value, may also be assigned. In one embodiment, network congestion is reduced by deliberately failing to transmit packets that are judged to be acoustically similar to adjacent packets; the expectation is that, under these circumstances, traditional packet loss concealment algorithms in the receiving device will construct an acceptably accurate replica of the missing packet. In another embodiment, the receiving device can reduce the number of packets stored in its jitter buffer, and therefore the latency of the speech signal, by selectively deleting one or more packets within sustained silences or non-varying speech events. In both embodiments, the ability of the system to drop appropriate packets may be enhanced by taking into account the confidence levels associated with the priority assessments.
1. A method comprising:
receiving a voice stream from a user, the voice stream comprising a plurality of temporally distinct segments;
processing at least one selected first segment of the voice stream, wherein the processing step comprises the following substeps:
determining whether or not the contents of the selected first segment are the product of voice activity and, when the contents are determined to be a product of voice activity, a level of confidence that the voice activity determination is accurate;
determining a type of voice activity associated with the contents of the first segment;
comparing the first segment with a second segment of the voice stream to determine a degree of acoustic similarity between the first and second segments, wherein the processing of the first segment is based on at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity;
converting at least the first segment into a packet;
providing a receive buffer;
receiving the packet into the receive buffer;
determining a level of importance associated with the packet, wherein the level of importance is based on the processing of the first segment; and
based on the level of importance associated with the packet, removing the packet from the receive buffer.
2. The method of claim 1 , further comprising:
based on the at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity, assigning an importance to the first segment.
3. The method of claim 2 , wherein the importance is a value marker and further comprising:
incorporating the value marker into a first packet comprising the first segment.
4. The method of claim 2 , wherein the importance is a service class assigned to a first packet comprising the first segment.
5. The method of claim 2 , wherein the importance is a transmission priority assigned to a first packet comprising the first segment.
6. A non-transitory computer readable medium having stored thereon computer-executable instructions to perform a method, the computer-executable instructions comprising:
instructions to receive a voice stream from a user, the voice stream comprising a plurality of temporally distinct segments;
instructions to process at least one selected first segment of the voice stream, wherein the processing step comprises the following instructions:
instructions to determine whether or not the contents of the selected first segment are the product of voice activity and, when the contents are determined to be a product of voice activity, a level of confidence that the voice activity determination is accurate;
instructions to determine a type of voice activity associated with the contents of the first segment; and
instructions to compare the first segment with a second segment of the voice stream to determine a degree of acoustic similarity between the first and second segments, wherein the processing of the first segment is based on at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity;
instructions to convert at least the first segment into a packet;
instructions to provide a receive buffer;
instructions to receive the packet into the receive buffer;
instructions to determine a level of importance associated with the packet, wherein the level of importance is based on the processing of the first segment; and
based on the level of importance associated with the packet, instructions to remove the packet from the receive buffer.
7. The computer readable medium of claim 6 , further comprising: based on the at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity, instructions to assign an importance to the first segment.
8. The computer readable medium of claim 7 , wherein the importance is a value marker and further comprising instructions to incorporate the value marker into a first packet comprising the first segment.
9. The computer readable medium of claim 7 , wherein the importance is a service class assigned to a first packet comprising the first segment.
10. The computer readable medium of claim 7 , wherein the importance is a transmission priority assigned to a first packet comprising the first segment.
11. A computer system comprising:
a memory;
a processor in communication with the memory, the processor operable to:
receive a voice stream from a user, the voice stream comprising a plurality of temporally distinct segments;
process at least one selected first segment of the voice stream, wherein the processing step comprises:
determine whether or not the contents of the selected first segment are the product of voice activity and, when the contents are determined to be a product of voice activity, a level of confidence that the voice activity determination is accurate;
determine a type of voice activity associated with the contents of the first segment; and
compare the first segment with a second segment of the voice stream to determine a degree of acoustic similarity between the first and second segments, wherein the processing of the first segment is based on at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity;
convert at least the first segment into a packet;
provide a receive buffer;
receive the packet into the receive buffer;
determine a level of importance associated with the packet, wherein the level of importance is based on the processing of the first segment; and
based on the level of importance associated with the packet, remove the packet from the receive buffer.
12. The computer system of claim 11 , further comprising: based on the at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity, the processor operable to assign an importance to the first segment.
13. The computer system of claim 12 , wherein the importance is a value marker and further comprising instructions to incorporate the value marker into a first packet comprising the first segment.
14. The computer system of claim 12 , wherein the importance is a service class assigned to a first packet comprising the first segment.
15. The computer system claim 12 , wherein the importance is a transmission priority assigned to a first packet comprising the first segment.