Viewport dependent video streaming events
View Patent ↗Systems and methods described herein provide for rendering and quality monitoring of rendering of a 360-degree video, where the video has a plurality of representations with different levels of quality in different regions. In an exemplary method, a client device tracks a position of a viewport with respect to the 360-degree video and renders to the viewport a selected set of the representations. The client adaptively adds and removes representations from the selected set based on the viewport position. The client also measures and reports a viewport switching latency. In some embodiments, the latency for a viewport switch is a comparable-quality viewport switch latency that represents the time it takes after a viewport switch to return to a quality comparable to the pre-switch viewport quality.
1. A method comprising:
rendering a portion of an omnidirectional video to a viewport;
logging a plurality of viewport quality measurements of the viewport and associated measurement times;
detecting a viewport switch event;
from among the viewport quality measurements, selecting a pre-switch measurement time before the viewport switch event;
from among the viewport quality measurements, selecting a post-switch measurement time such that the selected post-switch measurement time has an associated viewport quality measurement comparable to a viewport quality measurement associated with the selected pre-switch measurement time; and
reporting a latency value, where the latency value represents an interval between the selected pre-switch measurement time and the selected post-switch measurement time.
2. The method of claim 1 , wherein selecting the post-switch measurement time comprises selecting a post-switch measurement time having an associated viewport quality measurement at least as great as the viewport quality measurement associated with the selected pre-switch measurement time.
3. The method of claim 1 , wherein the omnidirectional video comprises a plurality of sub-pictures, and wherein rendering a portion of an omnidirectional video to a viewport comprises rendering an active set of at least one of the sub-pictures.
4. The method of claim 3 , wherein detecting a viewport switch event comprises detecting an addition of a new sub-picture to the active set.
5. The method of claim 3 , wherein each sub-picture in the active set is associated with a sub-picture quality level, and wherein the viewport quality measurement is determined based on the sub-picture quality levels of sub-pictures in the active set.
6. The method of claim 5 , further comprising calculating the viewport quality measurement using a weighted average of quality levels of sub-pictures in the active set, wherein the quality levels are weighted by the area of the viewport being covered by the respective sub-picture.
7. The method of claim 1 , wherein the logging of the plurality of viewport quality measurements and associated measurement times is performed on a periodic basis.
8. The method of claim 1 , further comprising reporting at least one the viewport quality measurement associated with the selected pre-switch measurement time and the viewport quality measurement associated with the selected post-switch measurement time.
9. The method of claim 1 , wherein the selected post-switch measurement time is selected to have an associated viewport quality measurement within a predetermined percentage of the viewport quality measurement associated with the selected pre-switch measurement time.
10. The method of claim 1 , wherein the selected pre-switch measurement time is within a predetermined pre-switch interval before the viewport switch event.
11. The method of claim 1 , wherein the selected pre-switch measurement time is a time of the last viewport quality measurement before the viewport switch event.
12. The method of claim 1 , wherein the selected pre-switch measurement time is selected such that (i) the selected pre-switch measurement time is within a predetermined pre-switch interval and (ii) the viewport quality measurement associated with the selected pre-switch measurement time is a maximum viewport quality measurement within the pre-switch interval.
13. The method of claim 1 , wherein the selected post-switch measurement time is selected within a predetermined interval ending after the viewport switch event.
14. A system comprising a processor, a display, and a non-transitory computer-readable storage medium storing instructions operative when executed on the processor to perform a method comprising:
rendering a portion of an omnidirectional video to a viewport;
logging a plurality of viewport quality measurements of the viewport and associated measurement times;
detecting a viewport switch event;
from among the viewport quality measurements, selecting a pre-switch measurement time before the viewport switch event;
from among the viewport quality measurements, selecting a post-switch measurement time such that the selected post-switch measurement time has an associated viewport quality measurement comparable to the viewport quality measurement associated with the selected pre-switch measurement time; and
reporting a latency value, where the latency value represents an interval between the selected pre-switch measurement time and the selected post-switch measurement time.
15. The system of claim 14 , wherein the omnidirectional video comprises a plurality of sub-pictures, wherein rendering a portion of an omnidirectional video to a viewport comprises rendering an active set of at least one of the sub-pictures, and wherein detecting a viewport switch event comprises detecting an addition of a new sub-picture to the active set.
16. The system of claim 14 , wherein the selected post-switch measurement time is selected to have an associated viewport quality measurement within a predetermined percentage of the viewport quality measurement associated with the selected pre-switch measurement time.
17. The system of claim 14 , wherein the selected pre-switch measurement time is within a predetermined pre-switch interval before the viewport switch event.
18. The system of claim 14 , wherein the selected pre-switch measurement time is a time of the last viewport quality measurement before the viewport switch event.
19. The system of claim 14 , wherein the selected pre-switch measurement time is selected such that (i) the selected pre-switch measurement time is within a predetermined pre-switch interval and (ii) the viewport quality measurement associated with the selected pre-switch measurement time is a maximum viewport quality measurement within the pre-switch interval.
20. The system of claim 14 , wherein the selected post-switch measurement time is selected within a predetermined interval ending after the viewport switch event.