System and method for likelihood computation in multi-stream HMM based speech recognition

1. A method for speech recognition, comprising the steps of:

determining active Gaussians related to a first feature stream and a second feature stream by labeling at least one of the first and second streams;

determining active Gaussians co-occurring in the first stream and the second stream based upon joint probability;

reducing a number of Gaussians computed for the second stream based upon Gaussians already computed for the first stream and a number of Gaussians co-occurring in the second stream; and

using a computer, decoding speech based on the Gaussians computed for the first and second streams.

2. The method as recited in claim 1 , wherein the step of labeling includes hierarchically labeling by surveying Gaussians in multiple resolutions.

3. The method as recited in claim 1 , wherein the step of labeling includes employing a search tree.

4. The method as recited in claim 1 , wherein the step of determining a distribution of Gaussians co-occurring includes providing a Gaussian co-occurrence map.

5. The method as recited in claim 1 , wherein the first stream includes an audio stream and the second stream includes a video stream and the step of decoding speech includes employing multi-stream hidden Markov models.

6. The method as recited in claim 1 , further comprising a plurality of feature streams wherein the step of decoding speech includes employing multi-stream hidden Markov models.

7. The method as recited in claim 1 , further comprising a plurality of feature streams wherein the step of decoding speech includes employing multi-stream hidden Markov models.

8. A method for speech recognition based upon a plurality of feature streams, comprising the steps of:

determining active Gaussians related to a first feature stream by hierarchically labeling the first feature stream;

determining active Gaussians co-occurring in the feature streams other than the first feature stream based upon joint probability using co-occurrence statistics such that a number of Gaussians computed for subsequent feature streams are reduced based upon co-occurring Gaussians already computed for at least one other feature stream; and

using a computer, decoding speech based on the Gaussians computed for the plurality of feature streams.

9. The method as recited in claim 8 , wherein the step of hierarchically labeling includes surveying Gaussians in multiple resolutions.

10. The method as recited in claim 8 , wherein the step of hierarchically labeling includes employing a search tree.

11. The method as recited in claim 8 , wherein the step of determining active Gaussians includes providing a Gaussian co-occurrence map.

12. The method as recited in claim 8 , wherein the first feature stream includes an audio stream and at least one other stream includes a video stream and the step of decoding speech includes employing multi-stream hidden Markov models.

13. A non-transitory computer-readable program storage device encoded with a program of instructions that, when executed by at least one computer, performs a method for speech recognition comprising:

determining active Gaussians related to a first feature stream and a second feature stream by labeling at least one of the first and second streams;

determining active Gaussians co-occurring in the first stream and the second stream based upon joint probability;

reducing a number of Gaussians computed for the second stream based upon Gaussians already computed for the first stream and a number of Gaussians co-occurring in the second stream; and

decoding speech based on the Gaussians computed for the first and second streams.

14. A non-transitory computer-readable program storage device encoded with a program of instructions that, when executed by at least one computer, performs a method for speech recognition comprising:

determining active Gaussians related to a first feature stream by hierarchically labeling the first feature stream;

determining active Gaussians co-occurring in the feature streams other than the first feature stream based upon joint probability using co-occurrence statistics such that a number of Gaussians computed for subsequent feature streams are reduced based upon co-occurring Gaussians already computed for at least one other feature stream; and

decoding speech based on the Gaussians computed for the plurality of feature streams.

15. A speech recognition system, comprising:

at least one memory to store processor-executable instructions; and

at least one processor coupled to the at least one memory. wherein upon execution of the processor-executable instructions by the at least one processor, the at least one processor:

A) extracts features from a first stream to generate likelihoods of the features of the first stream;

B) extracts features from a second stream associated with the first stream for generating likelihoods of the features of the second stream;

C) determines active Gaussians used to compute the likelihoods of the features of the first stream and finds active Gaussians co-occurring in the second stream to generate the likelihoods of the features of the second stream such that a number of Gaussians computed for the second stream is reduced based upon Gaussians already computed for the first stream; and

D) decodes speech based on the Gaussians computed for the first and second streams.

16. The speech recognition system as recited in claim 15 , wherein the at least one processor further fuses the features associated with the first and second streams to provide a third stream for decoding speech.

17. The speech recognition system as recited in claim 15 , wherein in D) the at least one processor employs multi-stream hidden Markov models.

18. The speech recognition system as recited in claim 15 , wherein in D) the at least one processor employs a co-occurrence map to generate joint probability statistics for the likelihoods associated with the first and second streams.

19. The speech recognition system as recited in claim 15 , wherein the first stream includes an acoustic stream and the second stream includes a video stream.

20. The speech recognition system as recited in claim 19 , wherein the video stream includes a human image as a region of interest for decoding speech.

21. The speech recognition system as recited in claim 15 , wherein in D) the at least one processor decodes speech in accordance with a plurality of streams.

22. The speech recognition system as recited in claim 15 , wherein the at least one processor further determines a set of available Gaussians for at least one stream.