IP Library Granted Patent US 11,287,653
Granted Patent B2
US 11,287,653 · App. 16/908,592 · Granted Mar 29, 2022

Method and system for selective content processing based on a panoramic camera and a virtual-reality headset

Inventor: Jean Mayrand (Chambly, CA)
Assignee: VANTRIX CORPORATION
G02B27/017G06F3/011G06K9/00335H04N5/23238H04N5/23296G02B2027/0138
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,287,653
App. No.
16/908,592
Granted
Mar 29, 2022
Kind
B2
Abstract

Gaze positions of an operator wearing a virtual-reality headset displaying a video stream define preferred view regions of the display. Starting with a reference gaze position, and for each subsequent distinctly different gaze position, the virtual-reality headset sends control data, including three spatial coordinates and a time coordinate expressed as a cyclical video-frame index, to a view adaptor receiving the video stream. The view adaptor stores contents of a number of most recent video frames of the video stream in a circular content-buffer and control data of a number of most recent gaze positions in a circular control-buffer. A content filter within the view adaptor determines a preferred view region surrounding a gaze position according to control data held in the circular control-buffer and extracts a partial content of a respective frame held in the circular content-buffer according to the preferred view region.

Claims (98)

1. A device for selective video-content dissemination comprising:

an acquisition module configured to receive a modulated carrier and extract a pure video signal;

a virtual-reality headset, communicatively coupled to the acquisition module, configured to provide:

a virtual-reality display of the pure video signal; and

coordinates of gaze positions of an operator wearing said virtual-reality headset and corresponding video-frame indices;

and

a content filter, communicatively coupled to the acquisition module and the virtual-reality headset, and having computer readable instructions stored in a memory device for execution by a processor causing the processor to:

receive the pure video signal, the coordinates, and the corresponding video-frame indices;

generate geometric data defining a view region of said display corresponding to each said gaze position;

extract a content-filtered signal from each frame of said pure video signal according to respective geometric data; and

transmit said content-filtered signal to a communication facility for dissemination.

2. The device of claim 1 wherein the communication facility comprises at least one of:

a broadcasting station; and

a streaming server configured to enable viewer content selection and provide said content-filtered signal as a default selection.

3. The device of claim 1 wherein said acquisition module comprises:

a receiver configured to detect from the modulated carrier:

a source multimedia signal; and

a corresponding signal descriptor;

a set of pure-video-signal generators each tailored to a respective signal descriptor; and

a selector for directing said source multimedia signal to a matching pure-video-signal generator according to said corresponding signal descriptor for generating said pure video signal.

4. The device of claim 1 wherein said content-filtered signal comprises samples of said pure video signal corresponding to points within said view region.

5. The device of claim 1 wherein:

said virtual-reality headset is further configured to provide an indication of a view-region shape of a predefined set of view-region shapes; and

said content filter is further configured to generate said geometric data according to a respective view-region shape.

6. A system for selective video-content dissemination comprising:

a virtual-reality headset configured to:

receive from a source a specific signal as one of:

a video signal; and

a frame-sampled signal of the video signal;

provide a display of the specific signal;

determine gaze positions, at spaced time instants, of an operator wearing the headset; and

communicate said gaze positions and corresponding video-frame indices;

and

a view adaptor comprising computer readable instructions stored in a memory for execution by a processor causing the processor to receive the video signal from the source and receive the gaze positions and corresponding frame indices from the virtual-reality headset, the view adaptor comprising:

a circular content-buffer for storing full-content frame data derived from the video signal;

a circular control-buffer for storing gaze-positions received from the virtual-reality headset;

a content-filter controller configured to determine for each said gaze position a surrounding view region according to a predefined view-region shape;

and

a content filter configured to extract a portion of each full-content frame data read from the circular content-buffer according to a view region of a respective gaze position read from the circular control-buffer for dissemination.

7. The system of claim 6 wherein the content-filter controller is further configured to:

initialize a reference gaze position;

determine a displacement of a current gaze position from the reference gaze position;

update the reference gaze position to equal the current gaze position subject to a determination that the displacement exceeds a predefined threshold;

and

set the current gaze position to equal the reference gaze position subject to a determination that the displacement does not exceed the predefined threshold.

8. The system of claim 6 wherein said circular content-buffer holds full-content of at least a predetermined number of frames where the predetermined number times a frame period exceeds a magnitude of:

sum of signal transfer delay from the source to the virtual-reality headset and signal transfer delay from the virtual-reality headset to the content-filter controller

minus

signal transfer delay from source to the view adaptor.

9. The system of claim 8 wherein:

said spaced time instants correspond to distant video frames with indices of immediately consecutive video frames differing by a predetermined integer Y, Y>1; and

said circular control-buffer holds a number of gaze-positions at least equal to ┌H/Y┐, H being said predetermined number of frames, H>Y.

10. The system of claim 9 wherein said content-filter controller is further configured to:

store frame content of a video frame of cyclical index f*, 0≤f*<L, in a storage division of index f* of the circular content-buffer;

and

store a gaze position corresponding to a cyclical index ϕ*, 0≤ϕ*<L, in a storage division of index └ϕ*/Y┘, L being a predefined cyclical period.

11. The system of claim 6 wherein said frame-sampled signal is produced at a frame-selection module coupled to said source and comprises distant video frames with immediately consecutive video frames separated by a time interval exceeding a duration of a single frame period.

12. The system of claim 7 wherein:

said virtual-reality headset is configured to define each said gaze position as Pan, Tilt, and Zoom coordinates; and

said content-filter controller is further configured to evaluate said displacement as a sum of absolute differences of pan, tilt, and zoom values of a first set of coordinates representing the reference gaze position and a second set of coordinates representing the current gaze position.

13. The system of claim 6 wherein said virtual-reality headset is further configured to enable said operator to select said predefined view-region shape as one of:

a default view-region shape; and

a view-region shape of a set of predefined view-region shapes.

14. A method of selective video-content dissemination comprising:

employing a virtual-reality headset to:

view a display of a video signal;

sense gaze positions, at spaced time instants, of an operator wearing the headset;

and

communicate said gaze positions and corresponding video-frame indices;

executing processor-readable instructions causing a hardware processor to initialize a reference gaze position and a corresponding view-region definition, and continually perform processes of:

receiving the video signal;

receiving the gaze positions and corresponding video-frame indices;

determining a displacement of a current gaze position from the reference gaze position;

subject to a determination that the displacement exceeds a predefined threshold:

updating the reference gaze position to equal the current gaze position; and

generating view-region definition data according to the reference gaze position and a predefined contour shape;

extracting a content-filtered signal from the video signal according to the view-region definition data; and

transmitting the content-filtered signal to a broadcasting facility.

15. The method of claim 14 further comprising selecting the spaced time instants so that a time interval between each time instant and an immediately succeeding time instant is an integer multiple, Y, of a video-frame time period, Y>1.

16. The method of claim 15 further comprising the instructions causing the processor to:

store full-content frame data of a predetermined number, H, of most-recent video frames of the video signal in a circular content-buffer, H>Y;

and

store a number, h, of most-recent gaze-positions received from the virtual-reality headset in a circular control-buffer, h=┌H/Y┐.

17. The method of claim 16 wherein said extracting comprises processes of:

determining for each video frame present in the circular content-buffer a respective gaze position present in the circular control buffer; and

deriving a content-filtered frame from full-content frame data of said each video frame.

18. The method of claim 14 wherein said determining comprises processes of:

representing each gaze position of the succession of gaze positions as a set of coordinates;

and

evaluating the displacement as a sum of absolute differences of corresponding coordinate values of a first set of coordinates representing the reference gaze position and a second set of coordinates representing the current gaze position.

19. The method of claim 14 wherein the virtual-reality headset receives the video signal and said display covers all video frames of the video signal.

20. The method of claim 14 wherein:

the virtual-reality headset receives only a frame-sampled signal of the video signal produced at a frame-selection module coupled to a source of the video signal, said frame-sampled signal comprising distant video frames with immediately consecutive video frames separated by a time interval exceeding a duration of a single frame period;

and

said display covers said distant video frames.

21. The method of claim 14 further comprising selecting said predefined contour shape as one of:

a default contour shape; and

a contour shape of a set of predefined contour shapes.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2022
From: VANTRIX CORPORATION
To: 3649954 CANADA INC.
Reel/Frame 059151/0316 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2020
From: MAYRAND, JEAN
To: VANTRIX CORPORATION
Reel/Frame 053285/0691 →
Continuity (7)
Continuation In Part 15452841 · Mar 8, 2017
Continuation In Part 15340193 · Nov 1, 2016
Continuation In Part 15259962 · Sep 8, 2016
Provisional Application 62361627 · Jul 13, 2016
Provisional Application 62249599 · Nov 2, 2015
Provisional Application 62216326 · Sep 9, 2015
Related Publication 20200348517A1 · Nov 5, 2020