Displaying content based on state information
A head-mounted device may determine contextual information by analyzing sensor data. The head-mounted device may use computer vision analysis to determine contextual information from images of the physical environment around the head-mounted device. Instead or in addition, the head-mounted device may determine contextual information by receiving state information directly from external equipment within the physical environment. Based on the received state information, the head-mounted device may display content, play audio, change a device setting on the head-mounted device, the external equipment, and/or additional external equipment, and/or may open an application. The head-mounted device may receive the state information in accordance with identifying the external equipment in images of the physical environment. The head-mounted device may receive the state information and then obtain and analyze sensor data to determine contextual information in response to the received state information being insufficient.
1. An electronic device comprising:
one or more sensors comprising one or more cameras;
communication circuitry;
one or more processors; and
memory storing instructions configured to be executed by the one or more processors, the instructions for:
capturing, via the one or more cameras, one or more images;
analyzing the one or more images, wherein analyzing the one or more images comprises detecting an external electronic device in the one or more images;
obtaining, via the communication circuitry, state information from the external electronic device; and
in accordance with a determination that the state information does not satisfy a criterion:
capturing, via the one or more cameras, one or more additional images;
estimating state information for the external electronic device based on the one or more additional images; and
presenting content that is based on the estimated state information for the external electronic device.
2. The electronic device defined in claim 1 , wherein the instructions further comprise instructions for:
ceasing to analyze the one or more images in response to obtaining, via the communication circuitry, the state information from the external electronic device.
3. The electronic device defined in claim 1 , wherein analyzing the one or more images comprises determining a first position of the external electronic device and wherein the instructions further comprise instructions for:
determining a second position at which to present the content based on the first position of the external electronic device, wherein presenting the content comprises presenting the content at the second position.
4. The electronic device defined in claim 3 , wherein the external electronic device is positioned in a physical environment, wherein analyzing the one or more images to determine the first position of the external electronic device comprises determining the first position of the external electronic device within the physical environment, and wherein the electronic device further comprises:
one or more displays configured to emit light from a transparent structure towards an eye box, wherein the second position overlaps the external electronic device when viewed from the eye box.
5. The electronic device defined in claim 1 , wherein the instructions further comprise instructions for:
in response to detecting the external electronic device in the one or more images:
wirelessly pairing with the external electronic device; and
transmitting, via the communication circuitry, a request for state information to the external electronic device.
6. The electronic device defined in claim 1 , wherein the state information comprises an application running on the external electronic device, comprises application information for an application running on the external electronic device, identifies content being displayed on a display in the external electronic device, or identifies audio being played on a speaker in the external electronic device.
7. The electronic device defined in claim 1 , wherein the instructions further comprise instructions for:
changing one or more settings in the electronic device, the external electronic device, or an additional external electronic device based on the state information received from the external electronic device.
8. A method of operating an electronic device that comprises one or more sensors and communication circuitry, the one or more sensors comprising one or more cameras and the method comprising:
capturing, via the one or more cameras, one or more images;
analyzing the one or more images, wherein analyzing the one or more images comprises detecting an external electronic device in the one or more images;
obtaining, via the communication circuitry, state information from the external electronic device; and
in accordance with a determination that the state information does not satisfy a criterion:
capturing, via the one or more cameras, one or more additional images;
estimating state information for the external electronic device based on the one or more additional images; and
presenting content that is based on the estimated state information for the external electronic device.
9. The method defined in claim 8 , further comprising:
ceasing to analyze the one or more images in response to obtaining, via the communication circuitry, the state information from the external electronic device.
10. The method defined in claim 8 , wherein analyzing the one or more images comprises determining a first position of the external electronic device and wherein the method further comprises:
determining a second position at which to present the content based on the first position of the external electronic device, wherein presenting the content comprises presenting the content at the second position.
11. The method defined in claim 10 , wherein the external electronic device is positioned in a physical environment, wherein analyzing the one or more images to determine the first position of the external electronic device comprises determining the first position of the external electronic device within the physical environment, wherein the electronic device further comprises one or more displays configured to emit light from a transparent structure towards an eye box, and wherein the second position overlaps the external electronic device when viewed from the eye box.
12. The method defined in claim 8 , further comprising:
in response to detecting the external electronic device in the one or more images:
wirelessly pairing with the external electronic device; and
transmitting, via the communication circuitry, a request for state information to the external electronic device.
13. The method defined in claim 8 , wherein the state information comprises an application running on the external electronic device, comprises application information for an application running on the external electronic device, identifies content being displayed on a display in the external electronic device, or identifies audio being played on a speaker in the external electronic device.
14. The method defined in claim 8 , further comprising:
changing one or more settings in the electronic device, the external electronic device, or an additional external electronic device based on the state information received from the external electronic device.
15. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device that comprises one or more sensors and communication circuitry, the one or more sensors comprising one or more cameras and the one or more programs including instructions for:
capturing, via the one or more cameras, one or more images;
analyzing the one or more images, wherein analyzing the one or more images comprises detecting an external electronic device in the one or more images;
obtaining, via the communication circuitry, state information from the external electronic device; and
in accordance with a determination that the state information does not satisfy a criterion:
capturing, via the one or more cameras, one or more additional images;
estimating state information for the external electronic device based on the one or more additional images; and
presenting content that is based on the estimated state information for the external electronic device.
16. The non-transitory computer-readable storage medium defined in claim 15 , wherein the instructions further comprise instructions for:
ceasing to analyze the one or more images in response to obtaining, via the communication circuitry, the state information from the external electronic device.
17. The non-transitory computer-readable storage medium defined in claim 15 , wherein analyzing the one or more images comprises determining a first position of the external electronic device and wherein the instructions further comprise instructions for:
determining a second position at which to present the content based on the first position of the external electronic device, wherein presenting the content comprises presenting the content at the second position.
18. The non-transitory computer-readable storage medium defined in claim 17 , wherein the external electronic device is positioned in a physical environment, wherein analyzing the one or more images to determine the first position of the external electronic device comprises determining the first position of the external electronic device within the physical environment, wherein the electronic device further comprises one or more displays configured to emit light from a transparent structure towards an eye box, and wherein the second position overlaps the external electronic device when viewed from the eye box.
19. The non-transitory computer-readable storage medium defined in claim 15 , wherein the instructions further comprise instructions for:
in response to detecting the external electronic device in the one or more images:
wirelessly pairing with the external electronic device; and
transmitting, via the communication circuitry, a request for state information to the external electronic device.
20. The non-transitory computer-readable storage medium defined in claim 15 , wherein the state information comprises an application running on the external electronic device, comprises application information for an application running on the external electronic device, identifies content being displayed on a display in the external electronic device, or identifies audio being played on a speaker in the external electronic device.
21. The non-transitory computer-readable storage medium defined in claim 15 , wherein the instructions further comprise instructions for:
changing one or more settings in the electronic device, the external electronic device, or an additional external electronic device based on the state information received from the external electronic device.