MINIMUM DIVERGENCE BASED DISCRIMINATIVE TRAINING FOR PATTERN RECOGNITION
A method of providing discriminative training of a speech recognition unit is discussed. The method includes receiving an acoustic indication of an utterance having a hypothesis space and comparing the hypothesis space against a reference. The method measures the Kullback-Leibler Divergence (KLD) between the reference and the hypothesis space to adjust the reference and stores the adjusted reference on a tangible storage medium.
1 . A method of providing discriminative training of a speech recognition unit, comprising:
receiving an acoustic indication of an utterance having a hypothesis space;
comparing the hypothesis space against a reference;
measuring the Kullback-Leibler Divergence (KLD) between the reference and the hypothesis space to adjust the reference; and
storing the adjusted reference on a tangible storage medium.
2 . The method of claim 1 , and further comprising:
smoothing the minimum divergence based discriminative training by interpolating between the minimum divergence and a maximum likelihood calculation.
3 . The method of claim 2 , wherein interpolating between the divergence and a maximum likelihood includes applying a smoothing constant.
4 . The method of claim 1 , wherein measuring the KLD includes employing a forward-backward algorithm.
5 . The method of claim 1 , wherein comparing the hypothesis space against a reference comprises:
calculating a posterior probability.
6 . The method of claim 1 , wherein comparing the hypothesis space against a reference comprises:
calculating a gain function indicative of an accuracy measure of the hypothesis space given the reference.
7 . The method of claim 6 wherein calculating the gain function includes calculating an indication of the acoustic similarity of the hypothesis space given the reference.
8 . The method of claim it wherein adjusting the reference includes adopting an Extended Baum-Welch algorithm to update a parameter.
9 . The method of claim 1 , wherein receiving the acoustic indication includes receiving a plurality of Hidden Markov Models.
10 . A method of automatically recognizing a pattern, comprising:
receiving pattern training data configured to train a pattern recognition model;
aligning the acoustic training data with a portion of the pattern recognition model;
calculating a gain indicative of a similarity between the pattern training data and the pattern recognition model;
adjusting the pattern recognition model to account for the pattern training data; and
providing the adjusted pattern recognition model to a pattern recognition application stored on a tangible computer medium
11 . The method of claim 10 , wherein receiving pattern data includes receiving speech pattern data configured to train an acoustic speech recognition model.
12 . The method of claim 10 , wherein calculating a gain includes calculating a Kullback-Leibler Divergence (KLD) between a portion of pattern training data and the recognition model.
13 . The method of claim 10 , wherein calculating a gain includes employing a forward-backward algorithm over a portion of the pattern training data.
14 . The method of claim 10 and further comprising:
employing a smoothing algorithm by applying a constant indicative of a maximum likelihood statistic to adjust the calculated gain.
15 . The method of claim 14 , wherein employing the smoothing algorithm includes interpolating between the maximum likelihood statistic and the gain.
16 . A pattern recognition system configured to train a model having a plurality of parameters, comprising:
a data store located on a tangible computer medium and configured to accept pattern training data;
a discriminative training engine configured to receive an observation and compare the observation with a portion of the pattern training data; and
wherein the discriminative training engine is configured to employ a minimum divergence based discriminative training algorithm to modify the pattern training data.
17 . The system of claim 16 , wherein the discriminative training engine is configured to calculated a KLD between a portion of the pattern training data and the observation.
18 . The system of claim 16 and further comprising:
an application module configured to access the pattern training data.
19 . The system of claim 16 , wherein the pattern training data includes a plurality of Hidden Markov Models.
20 . The system of claim 16 , wherein the discriminative training engine is configured to apply a smoothing algorithm to the pattern training data.