System and method for synthesizing and preserving consistent relative neighborhood position in multi-perspective multi-point tele-immersive environments

1. An e-learning system comprising:

a classroom having a fixed seating arrangement for a plurality of students and a plurality of first depth-sensing cameras positioned to capture 3D video images of each student from a variety of viewpoints;

a computerized session controller coupled to the first depth-sensing cameras, and executing software from a non-transitory medium;

a presenter station providing a position for a presenter, and a plurality of second depth-sensing cameras positioned to capture video images of a presenter at the presenter station from a variety of viewpoints, the plurality of second depth-sensing cameras coupled to the computerized session controller; and

a plurality of augmented-reality headsets, including video display and audio feedback circuitry, one each associated with each of the seats in the fixed-seating arrangement, and one associated with the presenter station;

wherein the software executing at the computerized session controller operates in one of an instruction mode, wherein a straight-on view and audio of the presenter station is streamed to each of the augmented-reality headsets at each of the seats in the fixed seating arrangement for the plurality of students, an interaction mode, wherein video and audio of interaction between the presenter station and a particular student at one of the seats in the fixed seating arrangement is streamed, such that the particular student views and hears the presenter station head on, and the presenter station views and hears the particular student head-on, and the remainder of the students view and hear the presenter and the student alternately as each participates in an audio exchange, and a discussion mode, wherein video and audio streaming takes place between individual students, the viewpoints and audio feeds to all of the AR headsets controlled such that the students directly interacting see and hear one another directly, and the remainder of the headsets see and here one or the other of the directly-engaged students according to which student is speaking.

2. The e-learning system of claim 1 wherein the classroom and the presenter station are in the same physical, geographic location, and the first and second depth-sensing cameras, and the augmented reality headsets are coupled to the computerized session controller on a local area network.

3. The e-learning system of claim 1 wherein the classroom and the presenter station are in different physical, geographic locations, and the first and second depth-sensing cameras, and the augmented reality headsets are coupled to the computerized session controller through the Internet network.

4. The e-learning system of claim 1 further comprising a plurality of classrooms, each having a fixed seating arrangement for a plurality of students and a plurality of first depth-sensing cameras positioned to capture 3D video images of each student from a variety of viewpoints, a plurality of augmented-reality headsets, including video display and audio feedback circuitry, one each associated with each of the seats in the fixed-seating arrangement at each classroom.

5. The e-learning system of claim 1 wherein the software executing at the computerized session controller provides for switching mode-to-mode based on gesture recognition in video feeds from individual ones of the depth-sensing cameras, or key words in audio feeds.

6. The e-learning system of claim 5 wherein the presenter is enabled to override mode switches software executing at the computerized session controller provides for switching mode-to-mode based on override input from an authorized source.

7. The e-learning system of claim 6 wherein the presenter is enabled to override mode switches.

8. The e-learning system of claim 1 wherein a viewpoint is determined by interpolation of frames from two different depth-sensing cameras.