On-device streaming inverse text normalization (ITN)
Solutions for on-device streaming inverse text normalization (ITN) include: receiving a stream of tokens, each token representing an element of human speech; tagging, by a tagger that can work in a streaming manner (e.g., a neural network), the stream of tokens with one or more tags of a plurality of tags to produce a tagged stream of tokens, each tag of the plurality of tags representing a different normalization category of a plurality of normalization categories; based on at least a first tag representing a first normalization category, converting, by a first language converter of a plurality of category-specific natural language converters (e.g., weighted finite state transducers, WFSTs), at least one token of the tagged stream of tokens, from a first lexical language form, to a first natural language form; and based on at least the first natural language form, outputting a natural language representation of the stream of tokens.
1. A system comprising:
a processor; and
a computer-readable medium storing instructions that are operative upon execution by the processor to:
receive a stream of tokens, each token representing an element of human speech;
chunk the stream of tokens;
tag, by a tagger, the stream of tokens with one or more tags of a plurality of tags to produce a tagged stream of tokens by chunks in a streaming manner, each tag of the plurality of tags representing a different normalization category of a plurality of normalization categories, wherein the tagger comprises a neural network using self-attention to compute representations of input and output;
detect, by each category-specific natural language converter of a plurality of category-specific natural language converters, each of the plurality of category-specific natural language converters comprising a weighted finite state transducer (WFST), from the tagged stream of tokens, a tag representing a normalization category of the plurality of normalization categories upon which each category-specific natural language converter is trained to operate, wherein each category-specific natural language converter is trained for a single normalization category of the plurality of normalization categories by each respective trainer of a plurality of trainers;
upon detecting a first tag representing a first normalization category, convert, by a first language converter of the plurality of category-specific natural language converters, a first token of the tagged stream of tokens, from a first lexical language form to a first natural language form, wherein the first language converter is trained to operate upon the first normalization category, and wherein the first token is associated with the first tag;
upon detecting a second tag representing a second normalization category, convert, in parallel with converting by the first language converter, by a second language converter of the plurality of category-specific natural language converters, a second token of the tagged stream of tokens from a second lexical language form to a second natural language form, wherein the second language converter is trained to operate upon the second normalization category, and wherein the second token is associated with the second tag; and
based on at least the first natural language form, output a natural language representation of the stream of tokens.
2. The system of claim 1 , wherein the instructions are further operative to:
determine a chunk size, wherein the stream of tokens is chunked according to the chunk size.
3. The system of claim 1 , wherein the stream of tokens is obtained from a live speech, and wherein outputting the natural language representation comprises outputting the natural language representation as a streaming output.
4. The system of claim 1 , wherein tagging and converting are performed locally without transmitting data to a remote node.
5. The system of claim 1 , wherein the plurality of category-specific natural language converters comprises at least two natural language converters selected from a category list consisting of:
address, location, time, date, currency, decimal, fraction, email, internet address, and math.
6. The system of claim 1 , wherein chunking the stream of tokens comprises dynamically determining a number of tokens per chunk.
7. A method comprising:
receiving a stream of tokens, each token representing an element of human speech;
chunking the stream of tokens;
tagging, by a tagger, the stream of tokens with one or more tags of a plurality of tags to produce a tagged stream of tokens by chunks in a streaming manner, each tag of the plurality of tags representing a different normalization category of a plurality of normalization categories, wherein the tagger comprises a neural network using self-attention to compute representations of input and output;
detecting, by each category-specific natural language converter of a plurality of category-specific natural language converters, each of the plurality of category-specific natural language converters comprising a weighted finite state transducer (WFST), from the tagged stream of tokens, a tag representing a normalization category of the plurality of normalization categories upon which each category-specific natural language converter is trained to operate, wherein each category-specific natural language converter is trained for a single normalization category of the plurality of normalization categories by each respective trainer of a plurality of trainers;
upon detecting a first tag representing a first normalization category, converting, by a first language converter of the plurality of category-specific natural language converters, a first token of the tagged stream of tokens, from a first lexical language form to a first natural language form, wherein the first language converter is trained to operate upon the first normalization category, and wherein the first token is associated with the first tag;
upon detecting a second tag representing a second normalization category, converting, in parallel with converting by the first language converter, by a second language converter of the plurality of category-specific natural language converters, a second token of the tagged stream of tokens from a second lexical language form to a second natural language form, wherein the second language converter is trained to operate upon the second normalization category, and wherein the second token is associated with the second tag; and
based on at least the first natural language form and the second natural language form, outputting a natural language representation of the stream of tokens.
8. The method of claim 7 , further comprising:
determining a chunk size, wherein the stream of tokens is chunked according to the chunk size.
9. The method of claim 7 , wherein the stream of tokens is obtained from a live speech, and wherein outputting the natural language representation comprises outputting the natural language representation as a streaming output.
10. The method of claim 7 , wherein tagging and converting are performed locally without transmitting data to a remote node.
11. The method of claim 7 , wherein the plurality of category-specific natural language converters comprises at least two natural language converters selected from a category list consisting of:
address, location, time, date, currency, decimal, fraction, email, internet address, and math.
12. The method of claim 7 , wherein chunking the stream of tokens comprises dynamically determining a number of tokens per chunk.
13. One or more computer storage devices having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:
receiving a stream of tokens, each token representing an element of human speech;
chunking the stream of tokens;
tagging, by a tagger, the stream of tokens with one or more tags of a plurality of tags to produce a tagged stream of tokens by chunks in a streaming manner, each tag of the plurality of tags representing a different normalization category of a plurality of normalization categories, wherein the tagger comprises a neural network using self-attention to compute representations of input and output;
detecting, by each category-specific natural language converter of a plurality of category-specific natural language converters, each of the plurality of category-specific natural language converters comprising a weighted finite state transducer (WFST), from the tagged stream of tokens, a tag representing a normalization category of the plurality of normalization categories upon which each category-specific natural language converter is trained to operate, wherein each category-specific natural language converter is trained for a single normalization category of the plurality of normalization categories by each respective trainer of a plurality of trainers;
upon detecting a first tag representing a first normalization category, converting, by a first language converter of the plurality of category-specific natural language converters, a first token of the tagged stream of tokens, from a first lexical language form to a first natural language form, wherein the first language converter is trained to operate upon the first normalization category, and wherein the first token is associated with the first tag;
upon detecting a second tag representing a second normalization category, convert, in parallel with converting by the first language converter, by a second language converter of the plurality of category-specific natural language converters, a second token of the tagged stream of tokens from a second lexical language form to a second natural language form, wherein the second language converter is trained to operate upon the second normalization category, and wherein the second token is associated with the second tag; and
based on at least the first natural language form and the second natural language form, outputting a natural language representation of the stream of tokens.
14. The one or more computer storage devices of claim 13 , wherein the operations further comprise:
determining a chunk size, wherein the stream of tokens is chunked according to the chunk size.
15. The one or more computer storage devices of claim 13 , wherein the stream of tokens is obtained from a live speech, and wherein outputting the natural language representation comprises outputting the natural language representation as a streaming output.
16. The one or more computer storage devices of claim 13 , wherein tagging and converting are performed locally without transmitting data to a remote node.
17. The one or more computer storage devices of claim 13 , wherein the plurality of category-specific natural language converters comprises at least two natural language converters selected from a category list consisting of:
address, location, time, date, currency, decimal, fraction, email, internet address, and math.
18. The one or more computer storage devices of claim 13 , wherein chunking the stream of tokens comprises dynamically determining a number of tokens per chunk.