System and method for augmented reality multi-view telepresence

1. A method comprising:

in a multi-location telepresence session that includes a first-location participant at a first location and a second-location participant at a second location, the first location having an array of first-location cameras and a plurality of first-location display segments, the second location having an array of second-location cameras and a plurality of second-location display segments, detecting a first-location position of the first-location participant at the first location with respect to a first-location augmented-reality (AR) marker, wherein the first-location AR marker is located in a predefined relationship to the array of first-location cameras at the first location;

obtaining a session geometry of the multi-location telepresence session, the session geometry determined based at least in part on the detected first-location position of the first-location participant at the first location with respect to the first-location AR marker;

selecting, according to the session geometry, a second-to-first-viewpoint first-location camera from the array of first-location cameras;

selecting, according to the session geometry, a first-to-second-viewpoint second-location camera from the array of second-location cameras;

selecting, according to the session geometry, a first-to-second-viewpoint first-location display segment from the plurality of first-location display segments;

receiving a first-to-second-viewpoint video stream captured by the selected first-to-second-viewpoint second-location camera; and

generating a line-of-sight augmented-reality experience for the multi-location telepresence session at least in part by rendering the received first-to-second-viewpoint video stream via the selected first-to-second-viewpoint first-location display segment.

2. The method of claim 1 , further comprising:

detecting a movement of the first-location participant at the first location from the detected first-location position to a new first-location position with respect to the first-location AR marker; and

updating the session geometry based on the detected new first-location position of the first-location participant.

3. The method of claim 1 , further comprising:

detecting a movement of the second-location participant at the second location from a second-location position to a new second-location position with respect to a second-location AR marker, wherein the second-location AR marker is located in a predefined relationship to the array of second-location cameras at the second location.

4. The method of claim 1 , wherein at least one of the cameras in the combined arrays of first-location cameras and second-location cameras is a monoscopic camera.

5. The method of claim 1 , wherein at least one of the cameras in the combined arrays of first-location cameras and second-location cameras is a stereoscopic camera.

6. The method of claim 1 , wherein a first-location head-mounted augmented-reality device comprises the selected first-to-second-viewpoint first-location display segment.

7. The method of claim 1 , wherein at least two of the first-location display segments in the plurality of first-location display segments are on two separate respective display devices at the first location.

8. The method of claim 1 , further comprising generating a virtual meeting space that reflects the session geometry, wherein rendering the received first-to-second-viewpoint video stream comprises rendering the received first-to-second-viewpoint video stream as appearing in the virtual meeting space.

9. The method of claim 1 , further comprising:

identifying, according to the session geometry, a first-to-second viewing orientation of the first-location participant with respect to the second-location participant, wherein:

selecting the first-to-second-viewpoint second-location camera from the array of second-location cameras according to the session geometry comprises:

determining that, among the second-location cameras in the array of second-location cameras, a given second-location camera most closely bears the identified first-to-second viewing orientation to the second participant; and

designating the given second-location camera as being the selected first-to-second-viewpoint second-location camera; and

selecting the first-to-second-viewpoint first-location display segment from the plurality of first-location display segments according to the session geometry comprises:

determining that, among the first-location display segments in the plurality of first-location display segments, a given first-location display segment is the one to which the first participant most closely bears the identified first-to-second-viewing orientation; and

designating the given first-location display segment as being the selected first-to-second-viewpoint first-location display segment.

10. The method of claim 1 , wherein the selected second-to-first-viewpoint first-location camera bears substantially the same orientation to the first-location participant as the selected first-to-second-viewpoint first-location display segment bears to the first-location participant; and

wherein the method further comprises:

capturing a second-to-first-viewpoint video stream of the first participant using the selected second-to-first-viewpoint first-location camera,

wherein generating the line-of-sight augmented-reality experience for the multi-location telepresence session further comprises transmitting the captured second-to-first-viewpoint video stream to the second location for rendering via a second-location display segment that bears substantially the same orientation to the second-location participant as the selected first-to-second-viewpoint second-location camera bears to the second-location participant.

11. The method of claim 10 , further comprising cropping the captured second-to-first-viewpoint video stream to substantially display a head of the first participant prior to transmitting the captured second-to-first-viewpoint video stream to the second location, wherein the array of first-location cameras comprises an array of wide-angle cameras.

12. The method of claim 1 , further comprising:

selecting, according to the session geometry and from the array of first-location cameras, a respective other-to-first-viewpoint first-location camera for each of one or more other-location participants at respective other locations; and

capturing a respective other-to-first-viewpoint video stream of the first participant using each of the one or more selected other-to-first-viewpoint first-location cameras,

wherein generating the line-of-sight augmented-reality experience for the multi-location telepresence session further comprises transmitting each respective captured other-to-first-viewpoint video stream to the corresponding other location for rendering via a respective other-location display segment that bears substantially the same orientation to the respective other-location participant as the first participant bears to the corresponding respective other-to-first-viewpoint first-location camera.

13. The method of claim 1 , further comprising:

identifying, according to the session geometry, a first-to-second viewing orientation of the first-location participant with respect to the second-location participant, wherein:

generating the line-of-sight augmented-reality experience for the multi-location telepresence session at least in part by rendering the received first-to-second-viewpoint video stream via the selected first-to-second-viewpoint first-location display segment comprises:

aligning the rendered received first-to-second-viewpoint video stream on the selected first-to-second-viewpoint first-location display segment such that vertical distortion of the second participant from the first-to-second viewing orientation of the first-location participant is prevented.

14. The method of claim 1 , wherein the first location comprises a wall on which the first-location cameras in the array of first-location cameras are mounted and on which the first-location AR marker is displayed and integrated together with the array of first-location cameras.

15. The method of claim 1 , wherein selecting the second-to-first-viewpoint first-location camera further comprises selecting the second-to-first-viewpoint first-location camera based on an orientation of the array of first-location cameras with respect to the first-location participant, wherein the orientation of the array of first-location cameras with respect to the first-location participant is known due to the first-location AR marker being located in the predefined relationship to the array of first-location cameras.

16. The method of claim 2 , further comprising:

selecting, according to the updated session geometry, a different second-to-first-viewpoint location camera from the array of first-location cameras.

17. A system comprising:

an array of first-location cameras at a first location;

a first-location augmented-reality (AR) marker, wherein the first-location AR marker is located in a predefined relationship to the array of first-location cameras at the first location;

a head-mounted augmented-reality (AR) device, the head-mounted augmented-reality device comprising:

a processor; and

data storage containing instructions executable by the processor for causing the system to carry out functions including:

in a multi-location telepresence session that includes a first-location participant wearing the head-mounted AR device at the first location, the multi-location telepresence session further including a second-location participant at a second location, the second location having an array of second-location cameras and a plurality of second-location display segments, detecting a first-location position of the first-location participant at the first location with respect to the first-location AR marker;

obtaining a session geometry of the multi-location telepresence session, the session geometry determined based at least in part on the detected first-location position of the first-location participant at the first location with respect to the first-location AR marker;

selecting, according to the session geometry, a second-to-first-viewpoint first-location camera from the array of first-location cameras;

selecting, according to the session geometry, a first-to-second-viewpoint second-location camera from the array of second-location cameras;

selecting, according to the session geometry, a first-to-second-viewpoint first-location display segment from a plurality of first-location display segments at the first location;

receiving a first-to-second-viewpoint video stream captured by the selected first-to-second-viewpoint second-location camera; and

generating a line-of-sight augmented-reality experience for the multi-location telepresence session at least in part by rendering the received first-to-second-viewpoint video stream via the selected first-to-second-viewpoint first-location display segment.

18. The system of claim 17 , wherein the instructions are further executable by the processor for causing the system to carry out functions including:

detecting a movement of the first-location participant at the first location from the detected first-location position to a new first-location position with respect to the first-location AR marker; and

updating the session geometry based on the detected new first-location position of the first-location participant.

19. The system of claim 17 , wherein the instructions are further executable by the processor for causing the system to carry out functions including:

detecting a movement of the second-location participant at the second location from a second-location position to a new second-location position with respect to a second-location AR marker, wherein the second-location AR marker is located in a predefined relationship to the array of second-location cameras at the second location.

20. The system of claim 17 , wherein the instructions are further executable by the processor for causing the system to carry out functions including:

identifying, according to the session geometry, a first-to-second viewing orientation of the first-location participant with respect to the second-location participant, wherein:

generating the line-of-sight augmented-reality experience for the multi-location telepresence session at least in part by rendering the received first-to-second-viewpoint video stream via the selected first-to-second-viewpoint first-location display segment comprises:

aligning the rendered received first-to-second-viewpoint video stream on the selected first-to-second-viewpoint first-location display segment such that vertical distortion of the second participant from the first-to-second viewing orientation of the first-location participant is prevented.