Combining generative aligners and transition-based parsers
Systems, computer-implemented methods, and computer program products to facilitate reducing error propagation when combining generative aligners and transition-based aligners are provided. According to an embodiment, a system can comprise a processor that executed components stored in memory. The computer executable components comprise a generative alignment component, an error propagation component, a discriminative parser, and a stochastic oracle policy component. The error propagation component can compute a posterior distribution over one or more hard alignments of parts given a pair of the generative alignment component. The discriminative parser can be trained via the stochastic oracle policy component to reduce error propagation when combining the generative alignment component with the discriminative parser.
1 . A system, comprising:
a processor that executes computer-executable components stored in a non-transitory computer-readable memory, wherein the computer-executable components comprise:
a generative aligner approximates samples from a posterior distribution, wherein the processor computes the posterior distribution over one or more hard alignments of parts given a pair within a generative aligner; and wherein the system comprises
a discriminative parser trained via a stochastic oracle policy component to reduce error propagation when combining the generative alignment component with the discriminative parser.
2 . The system of claim 1 , wherein the stochastic oracle policy component samples the one or more hard alignments of parts from the posterior distribution and runs a deterministic oracle policy on one or more samples of the one or more hard alignments of parts.
3 . The system of claim 2 , wherein the stochastic oracle policy component performs a learning step on the discriminative parser over one or more epochs.
4 . The system of claim 3 , wherein the learning step includes learning via gradient descent over one or more epochs.
5 . The system of claim 3 , wherein the processor further:
applies a scaling factor to average over the one or more samples.
6 . The system of claim 1 , wherein the discriminative parser is exposed to 6 . one or more explanatory models.
7 . The system of claim 1 , wherein the discriminative parser and the generative alignment component are end-to-end systems.
8 . A computer implemented method for reducing error propagation when combining generative aligners and transition-based aligners, the computer implemented method comprising:
computing, by a device operatively coupled to a processor, a posterior distribution over one or more hard alignments of parts given a pair within a generative alignment; and
running, by the device, a stochastic oracle policy to train a discriminative parser, further comprising:
sampling, by the device, the one or more hard alignments of parts from the posterior distribution;
running, by the device, a deterministic oracle policy on one or more samples of the one or more hard alignments of parts;
performing, by the device, a learning step on the discriminative parser; and
applying, by the device, a scaling factor to average over the one or more samples.
9 . The computer implemented method of claim 8 , wherein an uncertainty of the one or more hard alignments of parts is incorporated into training the discriminative parser.
10 . The computer implemented method of claim 8 , further comprising:
exposing the discriminative parser to one or more explanatory models.
11 . The computer implemented method of claim 8 , wherein running the stochastic oracle policy prevents over-fitting the discriminative parser to bad alignments.
12 . The computer implemented method of claim 8 , wherein the learning step is performed via gradient descent.
13 . The computer implemented method of claim 8 , further comprising:
transferring embedding-level knowledge from the generative alignment into speech embeddings.
14 . The computer implemented method of claim 8 , wherein the pair includes a natural language portion and a formal representation portion associated with an input sentence.
15 . A computer program product for reducing error propagation when combining generative aligners and transition-based aligners, the computer program product comprising a non-transitory computer readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
compute a posterior distribution over one or more hard alignments of parts given a pair within a generative alignment, wherein the generative alignment is relative to approximate samples from a posterior distribution; and
run a stochastic oracle policy to train a discriminative parser,
wherein an uncertainty of alignment of the generative alignment is incorporated into training the discriminative parser.
16 . The computer program product of claim 15 , wherein the program instructions are further executable to cause the processor to:
sample the one or more hard alignments of parts from the posterior distribution;
run a deterministic oracle policy on one or more samples of the one or more hard alignments of parts;
perform a learning step on the discriminative parser; and
apply a scaling factor to average over the one or more samples.
17 . The computer program product of claim 16 , wherein the program instructions are further executable to cause the processor to:
expose the discriminative parser to one or more explanatory models.
18 . The computer program product of claim 16 , wherein the program instructions are further executable to cause the processor to:
prevent over-fitting of the discriminative parser to bad alignments via the stochastic oracle policy.
19 . The computer program product of claim 16 , wherein the program instructions are further executable to cause the processor to:
perform the learning step via gradient descent.
20 . The computer program product of claim 16 , wherein the program instructions are further executable to cause the processor to:
transfer embedding-level knowledge from the generative alignment into speech embeddings.