Key update using comparison of transformed feature sets for extended reality privacy
Systems and methods are provided herein for providing privacy while allowing interactions with virtual representations of sensitive objects. This may be accomplished by an extended reality (XR) system receiving information associated with a real-world environment of a user. The XR system may transmit the received information to one or more devices (e.g., server, XR device, etc.) for processing. Before transmitting the received information, the XR system can use the received information to identify and categorize objects in the real-world environment to identify sensitive objects. To protect the privacy of the sensitive objects, the XR system transforms features of the sensitive objects using one or more transform keys prior to transmitting the information to the one or more devices. The XR system may also update the transform keys after a first time period to further protect the privacy of the sensitive objects.
1 . A method comprising:
receiving a first image comprising a plurality of objects;
extracting a first plurality of features from the first image, wherein the first plurality of features are associated with a first object of the plurality of objects;
determining that the first plurality of features are to be transformed;
in response to determining that the first plurality of features are to be transformed, transmitting a request for one or more transform keys;
receiving a first transform key corresponding to a first feature transform algorithm and a second transform key corresponding to a second feature transform algorithm;
transforming the first plurality of features into a first transformed feature set using the first feature transform algorithm;
transforming the first plurality of features into a second transformed feature set using the second feature transform algorithm, wherein the second transformed feature set is different than the first transformed feature set; and
transmitting, by a first device, the first transformed feature set and the second transformed feature set to a second device, wherein:
the transmitting causes the second device to update a first entry of a plurality of entries to include the first transformed feature set and the second transformed feature set;
the first entry is associated with a first avatar; and the transmitting causes the second device to display an updated extended reality (XR) environment comprising the first avatar, wherein a characteristic of the first avatar is changed within the updated XR environment based, at least in part, on the first transformed feature set.
2 . The method of claim 1 , wherein determining that the first plurality of features are to be transformed comprises:
determining that the first plurality of features correspond to the first object of the plurality of objects; and
determining that the first object is a first type of object, wherein the first type of object corresponds to a private object.
3 . The method of claim 2 , wherein the request indicates the first type of object.
4 . The method of claim 1 , further comprising:
extracting a second plurality of features related to a second object of the plurality of objects from the first image;
determining that the second object is a second type of object; and
transmitting the first transformed feature set, the second transformed feature set, and the second plurality of features to the second device.
5 . The method of claim 1 , further comprising, receiving, by the first device, a notification indicating that first transform key can be deleted.
6 . The method of claim 1 , wherein prior to receiving the first transformed feature set and the second transformed feature set, the plurality of entries only comprised transformed features sets generated using the first transform key.
7 . The method of claim 1 , wherein:
each entry of the plurality of entries is associated with one or more avatars of a plurality of avatars;
the first entry is associated with the first avatar; and
the second transformed feature set is used to update the first avatar.
8 . The method of claim 1 , wherein the second device is a head-mounted device comprising a display and the updated XR environment is displayed on the display of the second device.
9 . A method comprising:
transmitting, by a first device, a first transformed feature set to a second device, wherein:
the first transformed feature set is generated using a first feature transform algorithm;
the transmitting causes the second device to update a first entry of a plurality of entries to include the first transformed feature set;
the first entry is associated with a first avatar; and
the transmitting causes the second device to display an extended reality (XR) environment comprising the first avatar, wherein the first avatar is displayed within the XR environment according to the first transformed feature set;
receiving a first image comprising a plurality of objects;
extracting a first plurality of features from the first image, wherein the first plurality of features are associated with a first object of the plurality of objects;
determining that the first plurality of features are to be transformed;
in response to determining that the first plurality of features are to be transformed, transmitting a request for one or more transform keys;
receiving a first transform key corresponding to the first feature transform algorithm and a second transform key corresponding to a second feature transform algorithm;
transforming the first plurality of features into a second transformed feature set using the first feature transform algorithm;
transforming the first plurality of features into a third transformed feature set using the second feature transform algorithm, wherein the second transformed feature set is different than the third transformed feature set; and
transmitting, by the first device, the second transformed feature set and the third transformed feature set to the second device, wherein:
the transmitting causes the second device to determine that the second transformed feature set and the third transformed feature set correspond to the first entry based, at least in part, on the first transformed feature set and the second transformed feature set;
the transmitting causes the second device to update the first entry of the plurality of entries to include the second transformed feature set and the third transformed feature set; and
the transmitting causes the second device to display an updated XR environment comprising the first avatar, wherein a characteristic of the first avatar is changed within the updated XR environment based, at least in part, on the second transformed feature set.
10 . The method of claim 9 , wherein the second device determines that the second transformed feature set and the third transformed feature set correspond to the first entry based, at least in part, on a portion of the first transformed feature set being the same as a portion of the second transformed feature set.
11 . The method of claim 10 , wherein:
each entry of the plurality of entries is associated with one or more avatars of a plurality of avatars.
12 . The method of claim 9 , wherein the second device is a head-mounted device comprising a display and the updated XR environment is displayed on the display of the second device.
13 . The method of claim 12 , wherein the second device determines that the second transformed feature set and the third transformed feature set correspond to the first entry based, at least in part, on a portion of the first transformed feature set being within a difference threshold as a portion of the second transformed feature set.
14 . A method comprising:
receiving, by a first device, a first transformed feature set, wherein the first transformed feature set is generated using a first feature transform algorithm;
generating, by the first device, a virtual environment for display, wherein:
the virtual environment comprises a first avatar;
the first avatar corresponds to a first object; and
the first avatar is generated within the virtual environment using the first transformed feature set;
accessing, by the first device, a database comprising a plurality of entries, wherein each entry links at least one avatar to at least one feature set;
generating, by the first device, a first entry associated with the first avatar, wherein the first entry comprises the first transformed feature set;
displaying, by the first device, the virtual environment comprising the first avatar;
receiving a first image comprising a plurality of objects, wherein the plurality of objects comprise the first object;
extracting a first plurality of features from the first image, wherein the first plurality of features are associated with the first object;
determining that the first plurality of features are to be transformed;
in response to determining that the first plurality of features are to be transformed, transmitting a request for one or more transform keys;
receiving a first transform key corresponding to the first feature transform algorithm;
receiving a second transform key corresponding to a second feature transform algorithm;
transforming, by a second device, the first plurality of features into a second transformed feature set using the second feature transform algorithm;
transforming, by the second device, the first plurality of features into a third transformed feature set using the first feature transform algorithm, wherein the second transformed feature set is different than the third transformed feature set;
transmitting, by the second device, the second transformed feature set and the third transformed feature set to the first device;
determining, by the first device, that the second transformed feature set corresponds to the first entry based, at least in part, on the first transformed feature set and the second transformed feature set;
updating, by the first device, the first entry to comprises the second transformed feature set and the third transformed feature set;
generating, by the first device, an updated virtual environment, wherein the first avatar is updated within the updated virtual environment using the second transformed feature set; and
displaying, by the first device, the updated virtual environment.
15 . The method of claim 14 , wherein the first entry is associated with the first object.
16 . The method of claim 15 , wherein prior to receiving the second transformed feature set and the third transformed feature set, the plurality of entries only comprised transformed features sets generated using the first transform key.
17 . The method of claim 14 , wherein the first device is a head-mounted device comprising a display and the updated XR environment is displayed on the display of the first device.
18 . The method of claim 17 , wherein determining that the second transformed feature set corresponds to the first entry comprises determining a portion of the first transformed feature set is the same as a portion of the second transformed feature set.
19 . The method of claim 17 , wherein determining that the second transformed feature set corresponds to the first entry comprises determining a portion of the first transformed feature set is within a difference threshold of a portion of the second transformed feature set.