IP Library Granted Patent US 10,354,668
Granted Patent B2
US 10,354,668 · App. 15/786,451 · Granted Jul 16, 2019

System and method for processing audio data

Inventors: James David Johnston (Redmond, WA); Stephen Daniel White (Woodinville, WA); King Wei Hor (Redmond, WA); Barry M. Genova (Bothell, WA)
Assignee: IMMERSION NETWORKS, INC.
G10L19/167G10L19/0017G10L19/0204G10L19/025G10L19/032H03H17/0272H04B1/667H04B3/20H04B3/21H04L65/601H04L65/607H04N19/45H04N19/625H04N21/233G10L19/008G10L19/0212
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,354,668
App. No.
15/786,451
Granted
Jul 16, 2019
Kind
B2
Abstract

A codec operable to process audio data and related data. The codec further operable to receive at least one of an audio, audio auxiliary, program configuration, and data signals from a program source, the audio signals including at least one of single channel audio and multi-channel audio signals, audio auxiliary signals including spatial and motion data and environmental characteristics, the data signals including program related data. The codec further operable to generate a non-transitory encoded bitstream, wherein the bitstream includes at least one of synchronization command data and at least one of a program command data, audio channel data, audio auxiliary data, program content data, and an end of stream data, wherein the encoded bitstream includes an identifier for defining packet type for each data component. The synchronization command data includes a stream start flag defining an entry point for decoding the bitstream and further provides sample rate for the encoded bitstream.

Claims (91)

1. A codec for processing audio data comprising:

a computer-usable non-transitory storage resource; and

a processor communicatively coupled to the storage resource, wherein the processor executes application code instructions that are stored in the storage resource to cause the codec to:

receive an audio, audio auxiliary, program configuration, and data signals from a program source, the audio signals including single channel audio and multi-channel audio signals, audio auxiliary signals including spatial and motion data and environmental characteristics, the data signals including program related data; and

generate an encoded bitstream, wherein the bitstream includes a synchronization command data, zero or more of a program command data, audio channel data, audio auxiliary data, program content data, and an end of stream data, wherein the encoded bitstream includes an identifier for defining packet type for each data component;

wherein the synchronization command data includes a stream start flag defining an entry point for decoding the bitstream and information at first sample location within a block, and further provides sample rate for the encoded bitstream.

2. The codec for processing audio data according to claim 1 wherein the audio auxiliary data includes spatial audio information and environmental characteristics and effects.

3. The codec for processing audio data according to claim 1 wherein the application code instructions that generate the encoded bitstream further includes application code instructions to cause the codec to:

filter sampled audio signals into a plurality of frequency band components;

generate quantized digital components for each band of the plurality of bands by adaptive pulse code modulating the frequency band components;

identify a potential for pre-echo events for at least one band exceeding threshold within the generated quantized digital components;

generate an approximate signal of the filtered audio samples by decoding zero or more quantized digital components using inverse adaptive pulse code modulation;

generate an error signal by comparing the approximate signal with the sampled audio signal; and

process the error signal and quantized digital components.

4. The codec for processing audio data of claim 3 wherein the application code instructions that process the error signal and quantized digital components further comprises application code instruction that cause the codec to:

process delayed audio signals and Q band values;

determine the potential for pre-echo events from the Q band values; and

determine scale factors and Modified Discrete Cosine Transform (MDCT) block sizes.

5. The codec for processing audio data of claim 4 wherein the application code instructions further comprises application code instruction that causes the encoder to:

determine if an absolute Q band value exceeds a pre-determined threshold value;

determine the MDCT block sizes based on the magnitude of a Q band value and the band that contained the Q band value; and

determine scale factors based on the MDCT block sizes and the delayed audio signals.

6. The codec for processing audio data of claim 4 wherein the application code instructions that process the error signal and quantized digital components further comprises application code that cause the codec to:

transform, using Modified Discrete Cosine Transform (MDCT), the error signal into higher resolution frequency components using an MDCT of the previously determined block sizes;

quantize the scale factors and frequency components; and

encode the quantized frequency components, block sizes, and quantized scale factors for inclusion in the bitstream.

7. The codec for processing audio data of claim 3 wherein the application code instruction that filter sampled audio signals, generate quantized digital components, and identify pre-echo events from the generated quantized digital components further comprises application code instructions to cause the codec to:

filter the audio samples into a plurality of frequency band components using at least one Pseudo-Quadrature Mirror Filter (PQMF);

modulate the plurality of frequency band components into a plurality of quantized band values using an adaptive pulse code modulation technique; and

decode the plurality of quantized band values into an approximate signal using an inverse adaptive pulse code modulation technique and at least one Inverse Pseudo-Quadrature Mirror Filter (IPQMF).

8. A non-transitory computer readable storage medium comprising a set of computer instructions executable by a processor for instructing a computer to encode and decode audio data, the computer instructions configured to:

receive an audio, audio auxiliary, program configuration, and data signals from a program source, the audio signals including single channel audio and multi-channel audio signals, audio auxiliary signals including spatial and motion data and environmental characteristics, the data signals including program related data; and

generate an encoded bitstream, wherein the bitstream includes a synchronization command data, zero or more of a program command data, audio channel data, audio auxiliary data, program content data, and an end of stream data, wherein the encoded bitstream includes an identifier for defining packet type for each data component;

wherein the synchronization command data includes a stream start flag defining an entry point for decoding the bitstream and information at first sample location within a block, and further provides sample rate for the encoded bitstream.

9. The non-transitory computer readable storage medium of claim 8 comprising a set of computer instructions executable by a processor for instructing a computer to encode and decode audio data, the computer instructions configured to:

receive the encoded bitstream that includes bitstream synchronization command data and program command data;

process the encoded bitstream and identify within the bitstream the synchronization command data; and

decode the program command packet and at least one program related channel data using information provided in the synchronization command data and decode program related channel data using information provided in the program command data.

10. The non-transitory computer readable storage medium of claim 9 comprising a set of computer instructions executable by a processor for instructing a computer to encode and decode audio data, the computer instructions configured to:

receive encoded quantized bands and encoded Modified Discrete Cosine Transform (MDCT) data; and

decode the received encoded quantized bands and encoded MDCT data.

11. The non-transitory computer readable storage medium of claim 10 comprising a set of computer instructions executable by a processor for instructing a computer to encode and decode audio data, the computer instructions configured to:

decode the MDCT data; and

inverse modified discrete cosine transform line values from the decoded MDCT data to generate a coded error signal;

wherein the coded error signal was generated by an encoder.

12. The non-transitory computer readable storage medium of claim 11 comprising a set of computer instructions executable by a processor for instructing a computer to decode audio data, the computer instructions configured to:

inverse adaptive pulse code modulate individual quantized bands;

inverse pseudo quadrature mirror filter each inverse modulated band; and

add each inverse mirror filtered band together to generate an approximate signal.

13. The non-transitory computer readable storage medium of claim 12 comprising a set of computer instructions executable by a processor for instructing a computer to decode audio data, the computer instructions configured to cause the decoder to add the approximate signal and error signal together.

14. A computer-aided method of encoding and decoding audio data, the method comprising:

receiving an audio, audio auxiliary, program configuration, and data signals from a program source, the audio signals including signal channel audio and multi-channel audio signals, audio auxiliary signals including spatial and motion data and environmental characteristics, the data signals including program related data; and

generating an encoded bitstream, wherein the bitstream includes a synchronization command data, zero or more of a program command data, audio channel data, audio auxiliary data, program content data, and an end of stream data, wherein the encoded bitstream includes an identifier for defining packet type for each data component;

wherein the synchronization command data includes a stream start flag defining an entry point for decoding the bitstream and information at first sample location within a block, and further provides sample rate for the encoded bitstream.

15. The computer-aided method of claim 14 wherein the step of generating further comprising:

filtering sampled audio signals into a plurality of frequency band components;

generating quantized digital components for each band of the plurality of bands by adaptive pulse code modulating the frequency band components;

identifying a potential for pre-echo events for at least one band exceeding threshold within the generated quantized digital components;

generating an approximate signal of the filtered audio samples by decoding zero or more quantized digital components using inverse adaptive pulse code modulation;

generating an error signal by comparing the approximate signal with the sampled audio signal; and

processing the error signal and quantized digital components.

16. The computer-aided method of claim 15 wherein the step of processing the error signal and quantized digital components further comprising:

processing delayed audio signals and Q band values;

determining the potential for pre-echo events from the Q band values; and

determining scale factors and Modified Discrete Cosine Transform (MDCT) block sizes.

17. The computer-aided method of claim 16 further comprising:

determining if an absolute Q band value exceeds a pre-determined threshold value;

determining the MDCT block sizes based on a magnitude of a Q band value and a band that contained the Q band value; and

determining scale factors based on the MDCT block sizes and the delayed audio signals.

18. The computer-aided method of claim 16 wherein the step of processing the error further comprising:

transforming, using Modified Discrete Cosine Transform (MDCT) the error signal into higher resolution frequency components using the MDCT block sizes;

quantizing the scale factors and frequency components; and

encoding the quantized frequency components, block sizes and quantized scale factors for inclusion in the bitstream.

19. The computer-aided method of claim 15 wherein the step of filtering sampled audio signals, generating quantized digital components, and identifying pre-echo events from the generated quantized digital components further comprising:

filtering the audio samples into a plurality of frequency band components using at least one Pseudo-Quadrature Mirror Filter (PQMF);

modulating the plurality of frequency band components into a plurality of quantized band values using an adaptive pulse code modulation technique; and

decoding the plurality of quantized band values into an approximate signal using an inverse adaptive pulse code modulation technique and at least one Inverse Pseudo-Quadrature Mirror Filter (IPQMF).

20. The computer-aided method of claim 14 further comprising:

receiving the encoded bitstream that includes bitstream synchronization command data and program command data;

processing the encoded bitstream and identify within the bitstream the synchronization command data; and

decoding the program command packet and at least one program related channel data using information provided in the synchronization command data and decode program related channel data using information provided in the program command data.

21. The computer-aided method of claim 20 further comprising:

receiving encoded quantized bands and encoded MDCT data; and

decoding the received encoded quantized bands and encoded MDCT data;

decoding the MDCT data; and

generating a coded error signal by inverse modified discrete cosine transforming line values from the decoded MDCT data to generate a coded error signal;

inverse adaptive pulse code modulating individual quantized bands;

inverse pseudo quadrature mirror filtering each inverse modulated band;

adding each inverse mirror filtered band together to generate an approximate signal; and

adding the approximate signal and error signal together;

wherein the coded error signal was generated by an encoder.

Assignments (5)
ORDER GRANTING RECEIVER’S MOTION TO APPROVE SALE OF IP ASSETS, BID PROCEDURES, AND SALE DOCUMENTS Recorded Oct 23, 2025
From: SUPERIOR COURT FOR THE STATE OF WASHINGTON IN AND FOR THE COUNTY OF KING
To: TURNINGPOINTE, LLC
Reel/Frame 073238/0447 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2025
From: TURNINGPOINTE, LLC
To: SPOTIFY AB
Reel/Frame 072663/0038 →
ORDER APPOINTING GENERAL RECEIVER Recorded Oct 22, 2025
From: SUPERIOR COURT FOR THE STATE OF WASHINGTON IN AND FOR THE COUNTY OF KING
To: TURNINGPOINTE, LLC
Reel/Frame 073170/0289 →
MERGER AND CHANGE OF NAME Recorded Jan 21, 2019
From: IMMERSION SERVICES LLC; IMMERSION NETWORKS, INC.
To: IMMERSION NETWORKS, INC.
Reel/Frame 048104/0179 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2017
From: JOHNSTON, JAMES DAVID; WHITE, STEPHEN DANIEL; HOR, KING WEI; GENOVA, BARRY M.
To: IMMERSION SERVICES LLC DBA IMMERSION NETWORKS
Reel/Frame 043892/0302 →
Continuity (2)
Provisional Application 62475186 · Mar 22, 2017
Related Publication 20180277130A1 · Sep 27, 2018