Data security for machine learning systems
The technology described herein provides a system and method for securely managing information provided to a machine-learning system. In particular, the machine-learning system may determine that additional user data will improve the accuracy of a task being performed for a user. Security is improved by only requesting access to additional user data after determining that already available data may produce a task response that does not meet quality criteria. Further, the technology determines and requests a limited amount of user data and/or access needed to complete a task successfully. Several methods of determining whether additional user information will improve the task response are contemplated.
1 . A computer system, comprising:
a plurality of processors; and
computer memory having computer-readable instructions embodied thereon, that, when executed by the plurality of processors, perform operations comprising:
receiving, at a machine-learning system, an initiation request for a task associated with a user;
determining, by the machine-learning system, that additional user data has above a threshold probability of improving a response to the task, wherein the response is to be generated by the machine-learning system;
determining that the additional user data satisfies a sensitivity criteria, wherein the sensitivity criteria is defined in a sensitivity policy and applies to a subset of user data;
outputting, to the user, a request to access the additional user data;
receiving, from the user, permission to access the additional user data;
accessing the additional user data; and
generating, by the machine-learning system, the response to the task using the additional user data.
2 . The system of claim 1 , wherein the determining, by the machine-learning system, that the additional user data has above the threshold probability of improving the response to the task comprises determining that a quality measure associated with an initial task response generated by the machine-learning system is outside of a threshold range.
3 . The system of claim 1 , wherein the determining, by the machine-learning system, that the additional user data has above the threshold probability of improving the response to the task comprises:
automatically generating a prompt asking whether additional user data would improve the response to the task;
communicating the prompt to the machine-learning system; and
receiving an answer to the prompt indicating that additional user data would improve the response to the task.
4 . The system of claim 1 , wherein the determining, by the machine-learning system, that the additional user data has above the threshold probability of improving the response to the task comprises:
communicating the request to a second machine-learning model trained to determine whether additional user data improves task responses; and
receiving a second response from the second machine-learning model indicating that the additional user data would improve the response to the task.
5 . The system of claim 1 , wherein the machine-learning system is able to access the additional user data without receiving the permission.
6 . The system of claim 1 , wherein the task is a query.
7 . The system of claim 1 , wherein the machine-learning system comprises a large language model.
8 . A computer-implemented method, comprising:
receiving, at a machine-learning system, an initiation request for a task associated with a user, wherein the task is to be completed by the machine-learning system;
generating, by the machine-learning system, an initial response to the task using a first plurality of data;
assigning, by the machine-learning system, a quality measure to the initial response;
determining, by the machine-learning system, that the quality measure assigned to the initial response generated by the machine-learning system is below a threshold quality;
identifying, by the machine-learning system, a second plurality of data that is relevant to the initiation request, wherein the second plurality of data is identified by:
automatically generating a prompt asking whether additional user data would improve the response to the task;
communicating the prompt to the machine-learning system; and
receiving an answer to the prompt indicating the second plurality of data would improve the response to the task;
outputting, to the user, a request to access the second plurality of data;
receiving, from the user, permission to access the second plurality of data;
accessing, by the machine-learning system, the second plurality of data; and
generating, by the machine-learning system, a final response to the task using the second plurality of data.
9 . The computer-implemented method of claim 8 , wherein the task uses content from an application that is not associated with the machine-learning system as input.
10 . The computer-implemented method of claim 9 , wherein the machine-learning system is associated with an operating system on which the application is running.
11 . The computer-implemented method of claim 8 , wherein the initiation request is a prompt for the machine-learning system.
12 . The computer-implemented method of claim 8 , wherein the quality measure is generated by a machine-classifier.
13 . The computer-implemented method of claim 8 , wherein the task is a sentiment analysis of electronic communications.
14 . The computer-implemented method of claim 8 , wherein the machine-learning system comprises a large language model.
15 . Computer storage media having computer-executable instructions embodied thereon, that, when executed by at least one computer processor, cause computing operations to be performed, the operations comprising:
receiving, at a machine-learning system, an initiation request for a task associated with a user, wherein the task uses content from an application that is not accessible by the machine-learning system as input, and wherein the task is to be completed by the machine-learning system;
outputting, to the user, a request to access the content from the application wherein the request identifies the content as additional user data and the additional user data will improve a response to the task;
receiving, from the user, permission to access the content;
accessing the content; and
generating, at the machine-learning system, the response to the task using the content.
16 . The computer storage media of claim 15 , wherein the machine-learning system is associated with an operating system on which the application is running.
17 . The computer storage media of claim 15 , wherein the operations further comprise:
automatically generating a prompt asking whether the additional user data would improve the response to the task;
communicating the prompt to the machine-learning system; and
receiving an answer to the prompt indicating that the additional user data would improve the response to the task.
18 . The computer storage media of claim 17 , wherein the machine-learning system comprises a large language model.
19 . The computer storage media of claim 17 , wherein the task is a query.