Remote multiparticipant augmented reality
Provided is a method of establishing a remote multiparticipant augmented reality session such that participants in the remote multiparticipant augmented reality session are at different locations and are in different physical environments. Augmented reality objects may be populated in a shared virtual space that has components of both physical environments and some of those augmented reality objects are controlled by the remote participant by exchanging participant state information.
1. A non-transitory, machine-readable medium storing instructions that, when executed by one or more processors, effectuate operations comprising:
establishing, by a computer system, a remote multiparticipant augmented reality session via a network, wherein establishing the remote multiparticipant augmented reality session includes establishing a peer-to-peer connection between a first augmented reality device and a second augmented reality device;
providing, by the computer system and via the peer-to-peer connection, first state information for a participant controlled augmented reality object that is controlled by the first augmented reality device located in a first physical environment and that is viewed in the first physical environment via the first augmented reality device, to the second augmented reality device located in a second physical environment that is different than the first physical environment, wherein the first augmented reality device and the second augmented reality device are included in the remote multiparticipant augmented reality session;
receiving, by the computer system and via the peer-to-peer connection, second state information for a coparticipant controlled augmented reality object associated with the second augmented reality device;
providing, by the computer system, first physical environment information associated with the first physical environment that includes the first augmented reality device to the second augmented reality device that is included in the second physical environment;
receiving, by the computer system, second physical environment information associated with the second physical environment from the second augmented reality device included in the remote multiparticipant augmented reality session;
generating, by the computer system and using the first physical environment information and the second physical environment information, a shared virtual space, wherein the shared virtual space includes a plurality of virtual constraints determined from both the first physical environment information and the second physical environment information;
overlaying, by the computer system, the shared virtual space in the first physical environment viewed by the first augmented reality device such that objects in the first physical environment are still visible via the first augmented reality device; and
populating, by the computer system, the shared virtual space with augmented reality objects, wherein at least a portion of the augmented reality objects includes the coparticipant controlled augmented reality object controlled by the second augmented reality device according to the second state information and the participant controlled augmented reality object that is controlled and viewed in the first physical environment via the first augmented reality device, and wherein a virtual constraint of the plurality of virtual constraints that is generated based on the second physical environment information prevents movement of the participant controlled augmented reality object into a volume of the first physical environment even when a physical boundary of the first physical environment would permit movement of the participant controlled augmented reality object into the volume.
2. The medium of claim 1 , wherein the operations further include:
obtaining, by the computer system, the first physical environment information of the first physical environment for the remote multiparticipant augmented reality session that includes the first augmented reality device and the second augmented reality device.
3. The medium of claim 1 , wherein the first physical environment information includes a first reference point selected within the first physical environment and the second physical environment information includes a second reference point selected within the second physical environment.
4. The medium of claim 3 , wherein the operations further include:
obtaining, by the computer system, the first reference point selected within the first physical environment captured by a first imaging sensor and displayed on a first display coupled to the first augmented reality device and position information of the first imaging sensor when the first reference point was captured.
5. The medium of claim 3 , wherein the operations further include:
aligning, by the computer system, the first reference point and the second reference point when generating the shared virtual space.
6. The medium of claim 1 , wherein the first physical environment information includes a first virtual model of the first physical environment and the second physical environment information includes a second virtual model of the second physical environment.
7. The medium of claim 1 , wherein the operations further comprise:
receiving, by the computer system, updated second state information for the coparticipant controlled augmented reality object; and
updating, by the computer system and based on the updated second state information for the coparticipant controlled augmented reality object, the coparticipant controlled augmented reality object with at least one visual change in a presentation of the coparticipant controlled augmented reality object on a display of at least one of the first augmented reality device or the second augmented reality device.
8. The medium of claim 1 , wherein the computer system includes at least one of the first augmented reality device or a server computing device.
9. The medium of claim 1 , wherein the operations further comprise:
displaying, by the computer system, the shared virtual space with the populated augmented reality objects on a display of at least one of the first augmented reality device or the second augmented reality device.
10. The medium of claim 1 , wherein the second physical environment information includes a virtual representation of a physical object in the second physical environment associated with the second augmented reality device, and wherein the virtual representation is mapped to the shared virtual space, and wherein the operations further comprise:
populating, by the computer system, the virtual representation of the physical object with an augmented reality object representing that physical object; and
displaying, by the computer system, the shared virtual space with the populated augmented reality objects including the augmented reality object representing the physical object on a first display of the first augmented reality device.
11. The medium of claim 10 , wherein the augmented reality object representing the physical object provides constraints to the participant controlled augmented reality object controlled by the first augmented reality device.
12. The medium of claim 1 , wherein at least a second portion of the augmented reality objects includes a permissionless augmented reality object that is controllable by all participants of the remote multiparticipant augmented reality session.
13. The medium of claim 12 , wherein the operations further comprise:
changing, by the computer system and based on an update to the first state information of the participant controlled augmented reality object, the permissionless augmented reality object to a permissioned augmented reality object controlled by a portion of the participants in the remote multiparticipant augmented reality session.
14. The medium of claim 1 , wherein at least a second portion of the augmented reality objects includes the participant controlled augmented reality object controlled by the participant of the first augmented reality device.
15. The medium of claim 1 , wherein at least a second portion of the augmented reality objects includes an augmented reality application permissioned augmented reality object controlled by an augmented reality application.
16. The medium of claim 1 , wherein the operations comprise steps for generating the shared virtual space.
17. The medium of claim 1 , wherein the operations further comprise:
updating, by the computer system, the second state information for the coparticipant controlled augmented reality object, based on a change in the first state information for the participant controlled augmented reality object.
18. The medium of claim 1 , wherein the operations further comprise:
overlaying, by the computer system, the shared virtual space in the second physical environment viewed by the second augmented reality device such that objects in the second physical environment are still visible via the second augmented reality device; and
populating, by the computer system, the shared virtual space with the augmented reality objects, wherein at least a portion of the augmented reality objects includes the participant controlled augmented reality object controlled by the first augmented reality device according to the first state information and the coparticipant controlled augmented reality object that is controlled and viewed in the second physical environment via the second augmented reality device.
19. A method, comprising:
establishing, by a computer system, a remote multiparticipant augmented reality session via a network, wherein establishing the remote multiparticipant augmented reality session includes establishing a peer-to-peer connection between a first augmented reality device and a second augmented reality device;
providing, by the computer system and via the peer-to-peer connection, first state information for a participant controlled augmented reality object that is controlled by the first augmented reality device located in a first physical environment and that is viewed in the first physical environment via the first augmented reality device, to the second augmented reality device located in a second physical environment that is different than the first physical environment, wherein the first augmented reality device and the second augmented reality device are included in the remote multiparticipant augmented reality session;
receiving, by the computer system and via the peer-to-peer connection, second state information for a coparticipant controlled augmented reality object associated with the second augmented reality device;
providing, by the computer system, first physical environment information associated with the first physical environment that includes the first augmented reality device to the second augmented reality device that is included in the second physical environment;
receiving, by the computer system, second physical environment information associated with the second physical environment from the second augmented reality device included in the remote multiparticipant augmented reality session;
generating, by the computer system and using the first physical environment information and the second physical environment information, a shared virtual space, wherein the shared virtual space includes a plurality of virtual constraints determined from both the first physical environment information and the second physical environment information;
overlaying, by the computer system, the shared virtual space in the first physical environment viewed by the first augmented reality device such that objects in the first physical environment are still visible via the first augmented reality device; and
populating, by the computer system, the shared virtual space with augmented reality objects, wherein at least a portion of the augmented reality objects includes the coparticipant controlled augmented reality object controlled by the second augmented reality device according to the second state information and the participant controlled augmented reality object that is controlled and viewed in the first physical environment via the first augmented reality device, and wherein a virtual constraint of the plurality of virtual constraints that is generated based on the second physical environment information prevents movement of the participant controlled augmented reality object into a volume of the first physical environment even when a physical boundary of the first physical environment would permit movement of the participant controlled augmented reality object into the volume.