Method, device, and system for generating affordances linked to a representation of an item
In one implementation, a method of generating an affordance linked to an SR representation of an item is performed in a device including one or more processors, a non-transitory memory, and one or more displays. The method includes identifying an item; generating an affordance-item pair that corresponds to the item; detecting an input selecting the affordance-item pair; and displaying, on the one or more displays, a simulated reality (SR) environment that includes an SR representation of the item in response to detecting the user input selecting the affordance-item pair. In some implementations, the method includes obtaining an image data frame that includes the item.
1. A method comprising:
at a device including one or more processors, non-transitory memory, and one or more displays:
displaying, on the one or more displays, an environment that includes a representation of a first physical item being used by a first person;
obtaining an indication of a real-world interaction of a second person with a second physical item that is different from the first physical item; and
overlaying, on the representation of the first physical item, a virtual object that indicates the real-world interaction of the second person with the second physical item.
2. The method of claim 1 , wherein overlaying the virtual object comprises synthesizing the virtual object.
3. The method of claim 1 , wherein the virtual object comprises a set of one or more annotations that are overlaid on the representation of the first physical item.
4. The method of claim 1 , wherein the virtual object comprises a virtual representation of the second physical item.
5. The method of claim 1 , wherein the first person is situated in a first location and the second person is situated in a second location that is different than the first location.
6. The method of claim 1 , wherein the first person is situated in a first location and the second person is situated at the first location.
7. The method of claim 1 , wherein the device includes a forward-facing image sensor; and
wherein obtaining the indication of the real-world interaction of the second person with the second physical item comprises:
capturing images by the forward-facing image sensor; and
identifying the real-world interaction of the second person with the second physical item based on the images captured by the forward-facing image sensor.
8. The method of claim 1 , wherein obtaining the indication of the real-world interaction of the second person with the second physical item comprises:
receiving images captured by another device being used by the second person; and
identifying the real-world interaction of the second person with the second physical item based on the images captured by the other device being used by the second person.
9. The method of claim 1 , wherein overlaying the virtual object comprises:
determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item; and
displaying the virtual object indicating the real-world interaction of the second person with the second physical item in response to determining that the first person is unable to directly view the real-world interaction of the second person with the second physical item.
10. The method of claim 9 , wherein determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item comprises:
determining whether there is an obstruction in a line-of-sight between the first person and the real-world interaction of the second person with the second physical item.
11. The method of claim 9 , wherein determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item comprises:
determining a distance between the first person and the second physical item; and
determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item based on the distance between the first person and the second physical item.
12. The method of claim 9 , wherein determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item comprises:
determining a viewing angle between the first person and the second physical item; and
determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item based on the viewing angle between the first person and the second physical item.
13. A device comprising:
one or more processors;
a non-transitory memory;
one or more displays; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
display, on the one or more displays, an environment that includes a representation of a first physical item being used by a first person;
obtain an indication of a real-world interaction of a second person with a second physical item that is different from the first physical item; and
overlay, on the representation of the first physical item, a virtual object that indicates the real-world interaction of the second person with the second physical item.
14. The device of claim 13 , wherein overlaying the virtual object comprises:
determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item; and
displaying the virtual object indicating the real-world interaction of the second person with the second physical item in response to determining that the first person is unable to directly view the real-world interaction of the second person with the second physical item.
15. The device of claim 14 , wherein determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item comprises:
determining whether there is an obstruction in a line-of-sight between the first person and the real-world interaction of the second person with the second physical item.
16. The device of claim 14 , wherein determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item comprises:
determining a distance between the first person and the second physical item; and
determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item based on the distance between the first person and the second physical item.
17. The device of claim 14 , wherein determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item comprises:
determining a viewing angle between the first person and the second physical item; and
determining whether the first person is able to directly view the real-world interaction of the second person with the second physical item based on the viewing angle between the first person and the second physical item.
18. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with one or more displays, cause the device to:
display, on the one or more displays, an environment that includes a representation of a first physical item being used by a first person;
obtain an indication of a real-world interaction of a second person with a second physical item that is different from the first physical item; and
overlay, on the representation of the first physical item, a virtual object that indicates the real-world interaction of the second person with the second physical item.
19. The non-transitory memory of claim 18 , wherein the virtual object comprises a set of one or more annotations that are overlaid on the representation of the first physical item.
20. The non-transitory memory of claim 18 , wherein the virtual object comprises a virtual representation of the second physical item.
21. The non-transitory memory of claim 18 , wherein the first person is situated in a first location and the second person is situated in a second location that is different than the first location.
22. The non-transitory memory of claim 18 , wherein obtaining the indication of the real-world interaction of the second person with the second physical item comprises:
receiving images captured by another device being used by the second person; and
identifying the real-world interaction of the second person with the second physical item based on the images captured by the other device being used by the second person.