Methods, mediums, and systems for providing a model for an end-user device
Exemplary embodiments relate to methods, mediums, and systems for moving language models from a server to the client device. Such embodiments may be deployed in an environment where the server is not able to provide modeling services to the clients, such as an end-to-end encrypted (E2EE) environment. Several different techniques are described to address issues of size and complexity reduction, model architecture optimization, model training, battery power reduction, and latency reduction.
1. A method comprising:
receiving an input at a first end-user device, the input comprising natural language from a communication associated with a second end-user device and transmitted over a communications service;
converting the input into a byte-level embedding;
providing the byte-level embedding to a natural language understanding model located on the first end-user device, the natural language understanding model configured to operate on byte-level embeddings;
generating an output from the natural language understanding model;
selecting a recommendation based on the output;
presenting the recommendation on an interface of the first end-user device;
receiving a selection of the recommendation; and
transmitting a message incorporating the recommendation to the second end-user device.
2. The method of claim 1 , wherein the communications service transmits the communication in an end-to-end encrypted environment in which content of the communication is not visible to an intermediate server of the communications service.
3. The method of claim 1 , wherein the natural language understanding model makes use of an operator, and further comprising selecting an operator compatible with the first end-user device from an operator library.
4. The method of claim 3 , wherein multiple operators are available in the operator library, and the selected operator is selected based on an effect of the selected operator on a latency in generating the output.
5. The method of claim 1 , wherein the natural language understanding model is trained on training data represented as integers, and further comprising converting the input to a sequence of integers.
6. The method of claim 1 , wherein a size of the input is constrained by a maximum value, the maximum value selected based on an effect of the size of the input on a latency in generating the output.
7. The method of claim 6 , wherein the maximum value is 200 characters or less.
8. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:
receive an input at a first end-user device, the input comprising natural language from a communication associated with a second end-user device and transmitted over a communications service;
convert the input into a byte-level embedding;
provide the byte-level embedding to a natural language understanding model located on the first end-user device, the natural language understanding model configured to operate on byte-level embeddings;
generate an output from the natural language understanding model;
select a recommendation based on the output;
present the recommendation on an interface of the first end-user device;
receive a selection of the recommendation; and
transmit a message incorporating the recommendation to the second end-user device.
9. The medium of claim 8 , wherein the communications service transmits the communication in an end-to-end encrypted environment in which content of the communication is not visible to an intermediate server of the communications service.
10. The medium of claim 8 , wherein the natural language understanding model makes use of an operator, and the instructions, when executed by the processor, cause the processor to select an operator compatible with the first end-user device from an operator library.
11. The medium of claim 10 , wherein multiple operators are available in the operator library, and the selected operator is selected based on an effect of the selected operator on a latency in generating the output.
12. The medium of claim 8 , wherein the natural language understanding model is trained on training data represented as integers, and the instructions, when executed by the processor, cause the processor to convert the input to a sequence of integers.
13. The medium of claim 8 , wherein a size of the input is constrained by a maximum value, the maximum value selected based on an effect of the size of the input on a latency in generating the output.
14. The medium of claim 13 , wherein the maximum value is 200 characters or less.
15. An end-user device comprising:
a hardware interface configured to receive an input at a first end-user device, the input comprising natural language from a communication associated with a second end-user device and transmitted over a communications service;
a non-transitory device-readable medium configured to store a natural language understanding model;
a hardware processor configured to:
convert the input into a byte-level embedding;
provide the byte-level embedding to the natural language understanding model, the natural language understanding model configured to operate on byte-level embeddings;
generate an output from the natural language understanding model; and
select a recommendation based on the output;
a display configured to present the recommendation on an interface, wherein the processor is further configured to receive a selection of the recommendation;
a network transmitter configured to transmit a message incorporating the recommendation to the second end-user device.
16. The device of claim 15 , wherein the communications service transmits the communication in an end-to-end encrypted environment in which content of the communication is not visible to an intermediate server of the communications service.
17. The device of claim 15 , wherein the natural language understanding model makes use of an operator, and the hardware processor is configured to select an operator compatible with the first end-user device from an operator library.
18. The device of claim 17 , wherein multiple operators are available in the operator library, and the selected operator is selected based on an effect of the selected operator on a latency in generating the output.
19. The device of claim 15 , wherein the natural language understanding model is trained on training data represented as integers, and the hardware processor is configured to convert the input to a sequence of integers.
20. The device of claim 15 , wherein a size of the input is constrained by a maximum value, the maximum value selected based on an effect of the size of the input on a latency in generating the output.