IP Library Granted Patent US 11,681,145
Granted Patent B2
US 11,681,145 · App. 17/701,547 · Granted Jun 20, 2023

Method and system for filtering a panoramic video signal

Inventor: Jean Mayrand (Chambly, CA)
Assignee: 3649954 Canada Inc.
G02B27/017G06F3/011G06V40/20H04N23/69H04N23/698G02B2027/0138
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,681,145
App. No.
17/701,547
Granted
Jun 20, 2023
Kind
B2
Abstract

Gaze positions of an operator wearing a virtual-reality headset displaying a video stream define preferred view regions of the display. Starting with a reference gaze position, and for each subsequent distinctly different gaze position, the virtual-reality headset sends control data, including three spatial coordinates and a time coordinate expressed as a cyclical video-frame index, to a view adaptor receiving the video stream. The view adaptor stores contents of a number of most recent video frames of the video stream in a circular content-buffer and control data of a number of most recent gaze positions in a circular control-buffer. A content filter within the view adaptor determines a preferred view region surrounding a gaze position according to control data held in the circular control-buffer and extracts a partial content of a respective frame held in the circular content-buffer according to the preferred view region.

Claims (109)

1. A system for selective video-content dissemination, comprising:

a content controller;

a routing apparatus; and

a panoramic multimedia source configured to transmit a carrier of a panoramic video signal indicating video-frame indices to said content controller and said routing apparatus;

said content controller comprising:

a first acquisition module for deriving a pure video signal from said carrier;

a virtual-reality headset configured to indicate gaze positions of an operator viewing a display of said pure video signal wearing the virtual-reality headset;

a frame-index extraction module for extracting said video-frame indices; and

a refresh module for combining said video-frame indices with respective gaze positions and transmitting outcome to said routing apparatus;

said routing apparatus comprising:

a repeater, for amplifying said carrier, directing output to a streaming server;

a second acquisition module for deriving a replica pure video signal from said carrier;

a view adaptor for extracting from said replica pure video signal an operator-defined content-filtered signal corresponding to said gaze positions; and

a transmitter.

2. The system of claim 1 further comprising a first channel from the transmitter to a broadcasting station and a second channel from the transmitter to a network for establishing paths to a plurality of streaming servers.

3. The system of claim 1 wherein said refresh module is configured to:

receive a current gaze position from said VR headset;

determine a displacement of the current gaze position from a reference gaze position, said reference gaze position being initialized as an arbitrary position;

update the reference gaze position to equal the current gaze position subject to a determination that the displacement exceeds a predefined threshold; and

send to said routing apparatus a control message comprising the reference gaze position together with a corresponding video-frame index.

4. The system of claim 3 wherein said view adaptor is configured to:

generate view-region definition data according to the reference gaze position and a predefined contour shape; and

extract said operator-defined content-filtered signal according to the view-region definition data.

5. The system of claim 1 wherein said view adaptor comprises:

a circular content-buffer for storing full-content frame data derived from the replica pure video signal;

a circular control-buffer for storing, at addresses corresponding to said video-frame indices, said gaze-positions received from the content controller;

a content-filter controller for determining for each said gaze position a surrounding view region according to a predefined view-region shape; and

a content filter for extracting a portion of each full-content frame data read from the circular content-buffer according to a view region of a respective gaze position read from the circular control-buffer for dissemination.

6. The system of claim 5 wherein:

the circular content-buffer is organized into a number, L, of segments, L>1, each for holding video-frame data of a respective video frame;

said video-frame indices are cyclical, indexed as 0 to (Γ−1), Γ being a positive integer multiple of L; and

video-frame data of a video-frame of index M, 0≤M<Γ, is stored in a segment of index (M modulo L ) of the circular content-buffer.

7. The system of claim 1 wherein said first acquisition module is configured to demodulate said carrier and derive said pure video signal to be directed to said frame-index extraction module and said virtual-reality headset, said pure video signal being de-warped and uncompressed.

8. The system of claim 1 wherein said second acquisition module is configured to demodulate said carrier and derive said replica pure video signal directed to said view adaptor, said replica pure video signal being de-warped and uncompressed.

9. The system of claim 1 wherein said panoramic multimedia source comprises:

a panoramic camera configured to produce a raw video signal representing a field of view of a solid angle of up to 4π Steradians;

a source-processing unit configured to perform processes of de-warping and compression of said raw video-signal;

a frame-identifying module configured to insert in each video-frame data a respective cyclic frame index to produce said panoramic video signal indicating video-frame indices;

a transmitter for modulating said carrier with said panoramic video signal.

10. The system of claim 2 wherein at least one streaming server of the plurality of streaming servers receives from the routing apparatus:

said panoramic video signal; and

said operator-defined content-filtered signal produced at said view adapter;

said at least one streaming server being configured to:

provide, to a plurality of viewers of said at least one streaming server, said operator-defined content-filtered signal; and

derive, from said panoramic video signal, individual viewer-defined content-filtered signals.

11. A method for selective video-content dissemination, implemented at a system employing at least one processor, comprising:

acquiring a pure video signal indicating video-frame indices;

storing contents of successive video frames of the pure video signal in a circular content buffer;

receiving successive gaze positions, and respective video-frame indices, of an operator viewing a display of said pure video signal wearing a virtual-reality headset;

storing selected gaze-positions of the successive gaze positions in a circular control-buffer at cyclic addresses corresponding to said respective video-frame indices;

defining for each said selected gaze position a surrounding view region according to a predefined contour;

extracting a portion of content of each video-frame stored in the circular content-buffer according to a view region of a respective gaze position stored in the circular control-buffer;

and

transmitting said portion of content to at least one of:

a broadcasting station; and

a streaming server configured to enable viewer content selection and provide a content-filtered signal comprising said portion of content as a default selection.

12. The method of claim 11 wherein said acquiring comprises:

receiving a carrier of a panoramic video signal, said panoramic video signal including description of signal processing performed at source;

detecting said panoramic video signal;

deriving said pure video signal from said panoramic video signal according to said description.

13. The method of claim 11 , further comprising:

determining a displacement of a current gaze position from a reference gaze position initialized as an arbitrary value;

subject to a determination that a magnitude of the displacement exceeds a predefined threshold:

storing the current gaze position, as a selected gaze position, in said circular control-buffer; and

updating the reference gaze position to equal the current gaze position.

14. The method of claim 11 further comprising:

organizing the circular content-buffer into a number, L, of segments, L>1, each for holding video-frame data of a respective video frame;

selecting the video-frame indices as cyclical indices of 0 to (Γ−1), Γ being a positive integer multiple of L; and

storing video-frame data of a video-frame of index M, 0≤M<Γ, in a segment of index (M modulo L ) of the circular content-buffer.

15. A method for selective video-content dissemination, implemented at a system employing at least one processor, comprising:

transmitting from a panoramic multimedia source a carrier of a panoramic video signal, indicating video-frame indices, to a content controller and a routing apparatus;

at said content controller:

deriving a pure video signal from said carrier;

determining reference gaze positions of an operator viewing a display of said pure video signal wearing a virtual-reality headset;

identifying said video-frame indices; and

combining said video-frame indices with respective reference gaze positions and transmitting outcome to said routing apparatus;

at said routing apparatus:

amplifying said carrier and directing output to a streaming server;

deriving a replica pure video signal from said carrier;

extracting from said replica pure video signal an operator-defined content-filtered signal corresponding to said reference gaze positions; and

disseminating said content filter to a broadcasting station and a plurality of streaming servers.

16. The method of claim 15 wherein said determining comprises:

receiving a succession of candidate gaze position from said content controller;

determining a displacement of each candidate gaze position from a reference gaze position, said reference gaze position being initialized as an arbitrary position; and

updating the reference gaze position to equal the candidate gaze position subject to a determination that the displacement exceeds a predefined threshold.

17. The method of claim 16 wherein said extracting comprises:

for each reference gaze position:

generating a view-region contour according to a predefined contour shape; and

generating said operator-defined content-filtered signal from the video signal according to the view-region contour.

18. The method of claim 15 wherein:

said panoramic video signal comprises: pixels; said video-frame indices, and description of signal processes performed at source; and

said deriving the pure video signal comprises:

detecting from said carrier said panoramic video signal;

generating said pure video signal from said panoramic video signal according to said description, said pure video signal being de-warped and uncompressed;

said deriving the replica pure video signal comprises:

detecting from said carrier said panoramic video signal;

generating said replica pure video signal from said panoramic video signal according to said description, said pure video signal being de-warped and uncompressed.

19. The method of claim 15 wherein said extracting comprises:

storing full-content video-frame data derived from the pure video signal in a circular content-buffer;

storing said gaze-positions received from the content controller in a circular control-buffer, at addresses corresponding said video-frame indices;

determining for each said gaze position a surrounding view region according to a predefined view-region shape; and

obtaining a portion of each full-content frame data read from the circular content-buffer according to a view region of a respective gaze position read from the circular control-buffer.

20. The method of claim 15 further comprising:

said routing apparatus sending to at least one streaming server of the plurality of streaming servers:

said panoramic video signal; and

said operator-defined content-filtered signal produced at said view adapter;

said at least one streaming server:

providing, to a plurality of viewers of said at least one streaming server, said operator-defined content-filtered signal; and

deriving, from said panoramic video signal, individual viewer-defined content-filtered signals.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2022
From: MAYRAND, JEAN
To: 3649954 CANADA INC.
Reel/Frame 060039/0379 →
Continuity (8)
Continuation 16908592 · Jun 22, 2020
Continuation In Part 15452841 · Mar 8, 2017
Continuation In Part 15340193 · Nov 1, 2016
Continuation In Part 15259962 · Sep 8, 2016
Provisional Application 62361627 · Jul 13, 2016
Provisional Application 62249599 · Nov 2, 2015
Provisional Application 62216326 · Sep 9, 2015
Related Publication 20220214545A1 · Jul 7, 2022
Cited By (1)
US 12,332,439