Training a network to inhibit performance of a secondary task
The present disclosure describes techniques for training a neural network such that the trained network can be implemented to perform a utility task (e.g., a classification task) while inhibiting performance of a secondary task (e.g., a privacy-violating task). In some embodiments, the techniques include training a neural network using a first loss associated with a first task and a second loss associated with a second task different from the first task. In some embodiments, this includes performing a first training operation associated with the first loss, and performing a second training operation associated with the second loss, wherein the second training operation includes providing, to the neural network, a plurality of input items associated with the second task.
1 . An electronic device, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
training a neural network using a first loss associated with a first task and a second loss associated with a second task different from the first task, including:
performing a first training operation associated with the first loss; and
performing a second training operation associated with the second loss, wherein the second training operation includes providing, to the neural network, a plurality of input items associated with the second task, wherein the plurality of input items associated with the second task include one or more sets of images having a set of distinct elements associated with the second task, and wherein the one or more sets of images further include a set of similar elements associated with the first task.
2 . The electronic device of claim 1 , wherein training the neural network further includes:
after performing the second training operation, performing the first training operation associated with the first loss.
3 . The electronic device of claim 1 , wherein the plurality of input items associated with the second task include audio samples having different audio characteristics or images having different facial image data.
4 . The electronic device of claim 1 , wherein the first training operation includes providing, to the neural network, a plurality of input items associated with the first task.
5 . The electronic device of claim 4 , wherein the input items associated with the first task include audio samples associated with the first task or images associated with the first task.
6 . The electronic device of claim 1 , wherein performing the second training operation comprises training the neural network to extract features from the plurality of input items associated with the second task such that performance of the second task using the extracted features is inhibited.
7 . The electronic device of claim 1 , wherein:
the second training operation further includes training the neural network to extract features from input items having dissimilar elements present in one or more of the plurality of input items associated with the second task to a common location in a feature space, and
the second loss is representative of a distance metric between the extracted features from the input items having dissimilar elements.
8 . The electronic device of claim 1 , wherein the second training operation further includes providing a first subset of the sets of images to a first portion of the neural network, and providing a second subset of the sets of images to a second portion of the neural network.
9 . The electronic device of claim 8 , wherein the second loss is a distance metric between features extracted from the first subset of the sets of images by the first portion of the neural network and features extracted from the second subset of the sets of images by the second portion of the neural network.
10 . The electronic device of claim 1 , wherein the second training operation further includes:
prior to providing, to the neural network, the plurality of input items associated with the second task:
in accordance with a determination that the plurality of input items associated with the second task includes a respective set of images that exclude a respective element associated with the second task, modifying the respective set of images to include the respective element associated with the second task.
11 . The electronic device of claim 10 , wherein modifying the respective set of images to include the respective element associated with the second task includes:
modifying a first image of the respective set of images to include a first version of the respective element; and
modifying a second image of the respective set of images to include a second version of the respective element different from the first version of the respective element,
wherein the first version of the respective element is positioned in the first image at a first location, and the second version of the respective element is positioned in the second image at the first location.
12 . The electronic device of claim 1 , wherein the second loss is associated with inhibiting performance of the second task.
13 . The electronic device of claim 1 , wherein the second loss has a first weight applied during the first training operation and a second weight different from the first weight applied during the second training operation, wherein training the neural network further includes:
after performing the second training operation a number of instances, adjusting the second weight.
14 . The electronic device of claim 1 , wherein the first task is a utility task of the neural network and the second task is a privacy-violating task.
15 . The electronic device of claim 1 , the one or more programs further including instructions for:
after training the neural network:
receiving an input image for performing the first task, wherein the first task includes a visual search and the input image includes image data of a first type;
extracting features using the input image; and
performing the first task using the extracted features, while inhibiting use of the image data of the first type to perform the second task.
16 . The electronic device of claim 1 , the one or more programs further including instructions for:
after training the neural network:
receiving an input audio for performing the first task, wherein the first task includes speech recognition and the input audio includes audio data of a first type;
extracting features using the input audio; and
performing the first task using the extracted features, while inhibiting use of the audio data of the first type to perform the second task.
17 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:
training a neural network using a first loss associated with a first task and a second loss associated with a second task different from the first task, including:
performing a first training operation associated with the first loss; and
performing a second training operation associated with the second loss, wherein the second training operation includes providing, to the neural network, a plurality of input items associated with the second task, wherein the plurality of input items associated with the second task include one or more sets of images having a set of distinct elements associated with the second task, and wherein the one or more sets of images further include a set of similar elements associated with the first task.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein performing the second training operation comprises training the neural network to extract features from the plurality of input items associated with the second task such that performance of the second task using the extracted features is inhibited.
19 . The non-transitory computer-readable storage medium of claim 17 , wherein:
the second training operation further includes training the neural network to extract features from input items having dissimilar elements present in one or more of the plurality of input items associated with the second task to a common location in a feature space, and
the second loss is representative of a distance metric between the extracted features from the input items having dissimilar elements.
20 . The non-transitory computer-readable storage medium of claim 17 , wherein the second loss is associated with inhibiting performance of the second task.
21 . A method, comprising:
at an electronic device:
training a neural network using a first loss associated with a first task and a second loss associated with a second task different from the first task, including:
performing a first training operation associated with the first loss; and
performing a second training operation associated with the second loss, wherein the second training operation includes providing, to the neural network, a plurality of input items associated with the second task, wherein the plurality of input items associated with the second task include one or more sets of images having a set of distinct elements associated with the second task, and wherein the one or more sets of images further include a set of similar elements associated with the first task.
22 . The method of claim 21 , wherein performing the second training operation comprises training the neural network to extract features from the plurality of input items associated with the second task such that performance of the second task using the extracted features is inhibited.
23 . The method of claim 21 , wherein:
the second training operation further includes training the neural network to extract features from input items having dissimilar elements present in one or more of the plurality of input items associated with the second task to a common location in a feature space, and
the second loss is representative of a distance metric between the extracted features from the input items having dissimilar elements.
24 . The method of claim 21 , wherein the second loss is associated with inhibiting performance of the second task.