Handheld controller gestures for virtual reality video playback
Embodiments disclosed herein facilitate virtual reality (VR) video playback using handheld controller gestures. More specifically, jog and shuttle gestures are associated with controller rotations that can be tracked once a triggering event is detected (e.g., pressing and holding a controller play button). A corresponding jog or shuttle command can be initialized when the VR controller rotates more than a defined angular threshold in an associated rotational direction (e.g., yaw, pitch, roll). For example, the jog gesture can be associated with changes in controller yaw, and the shuttle gesture can be associated with changes in controller pitch. Subsequent controller rotations can be mapped to playback adjustments for a VR video, such as a frame adjustment for a jog gesture and a playback speed adjustment for the shuttle gesture. Corresponding visualizations of available gestures and progress bars can be generated or otherwise triggered to facilitate efficient VR video playback control.
1. A method for controlling VR video playback, the method comprising:
detecting a triggering event based on state information of a handheld controller capable of detecting 3D rotations of the handheld controller;
upon detecting the triggering event, detecting a 3D gesture made with the handheld controller by determining a 1D rotational direction of a 3D controller rotation of the handheld controller;
initializing a command associated with changes in the determined 1D rotational direction, wherein the determined 1D rotational direction of the 3D controller rotation is yaw or pitch, wherein the initialized command associated with changes in the determined 1D rotational direction is a jog command of the VR video or a shuttle command of the VR video;
detecting a subsequent 3D controller rotation of the handheld controller in an initialized rotational direction matching the determined 1D rotational direction; and
determining a playback adjustment of a VR video based on the initialized command and the subsequent 3D controller rotation.
2. The method of claim 1 , wherein the determining of the 1D rotational direction comprises, upon detecting the triggering event:
periodically accessing a controller orientation of the handheld controller;
tracking a plurality of rotational directions of the handheld controller by comparing the controller orientation at a first time with the controller orientation at a second time; and
determining the 1D rotational direction as one of the plurality of rotational directions that first exceeds a defined initialization threshold.
3. The method of claim 1 , wherein the determined 1D rotational direction of the 3D controller rotation is yaw, and wherein the initialized command associated with changes in yaw is a jog command of the VR video.
4. The method of claim 1 , wherein the initialized command is the jog command, and wherein the playback adjustment is an adjustment of a defined number of frames of the VR video for each movement of the subsequent 3D controller rotation through a defined angular increment.
5. The method of claim 1 , wherein the initialized command is the shuttle command, and wherein the playback adjustment is an adjustment of a factor change in playback speed of the VR video for each movement of the subsequent 3D controller rotation through a defined angular increment.
6. The method of claim 1 , additionally comprising defining a frame of reference, wherein the determining of the 1D rotational direction is with respect to the defined frame of reference.
7. The method of claim 6 , wherein the defined frame of reference is based on a controller orientation of the handheld controller at a time associated with the triggering event.
8. The method of claim 6 , wherein the defined frame of reference is based on a controller calibration.
9. The method of claim 1 , additionally comprising:
triggering a visualization of available gestures once the triggering event is detected;
replacing the visualization of available gestures with a visualization of a progress bar corresponding to the initialized command, when the command is initialized; and
triggering an update of the visualization of the progress bar based on the subsequent 3D controller rotation.
10. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
determining a 1D rotational direction of a 3D controller rotation of a handheld controller capable of detecting 3D rotations of the handheld controller;
initializing a command associated with the determined 1D rotational direction, wherein the command comprises a jog command or a shuttle command;
replacing, based on initializing the command, a first visualization of available gestures with a second visualization of a progress bar corresponding to the initialized command;
during a defined duration, detecting a subsequent 3D controller rotation of the handheld controller considering only an initialized direction associated with the initialized command;
generating a control signal for the initialized command, wherein the control signal indicates the subsequent 3D controller rotation of the handheld controller; and
providing the control signal to facilitate a playback adjustment of a VR video.
11. The media of claim 10 , wherein the determining of the 1D rotational direction comprises:
periodically accessing a controller orientation of the handheld controller;
tracking a plurality of rotational directions of the handheld controller by comparing the controller orientation at a first time with the controller orientation at a second time; and
determining the 1D rotational direction as one of the plurality of rotational directions that first exceeds a defined initialization threshold.
12. The media of claim 10 , wherein the determined 1D rotational direction of the 3D controller rotation is pitch, and wherein the initialized command associated with changes in pitch is a shuttle command of the VR video.
13. The media of claim 10 :
wherein the operations additionally comprise defining a frame of reference, and
wherein the determining of the 1D rotational direction is with respect to the defined frame of reference.
14. The media of claim 10 , wherein the initialized direction matches the determined 1D rotational direction.
15. The media of claim 10 , wherein the operations additionally comprise:
detecting a triggering event based on state information of the handheld controller, wherein the determining of the 1D rotational direction of the 3D controller rotation is based on the detecting of the triggering event;
triggering the first visualization of available gestures once the triggering event is detected;
after the replacing of the first visualization of available gestures with the second visualization of the progress bar, triggering an update of the second visualization of the progress bar based on the subsequent 3D controller rotation.
16. A computer system comprising:
one or more hardware processors and memory configured to provide computer program instructions to the one or more hardware processors;
a triggering component configured to utilize the one or more hardware processors to detect a triggering event based on state information of a handheld controller capable of detecting 3D rotations of the handheld controller;
a means for initializing a command associated with changes in a determined 1D rotational direction of a 3D controller rotation of the handheld controller, wherein the initialized command associated with changes in the determined 1D rotational direction is a jog command of the VR video or a shuttle command of the VR video; and
a means for determining a playback adjustment of a VR video based on the initialized command and a subsequent 3D controller rotation of the handheld controller.
17. The computer system of claim 16 , wherein the means for initializing the command is configured to:
track a plurality of rotational directions of the handheld controller to determine a 1D rotational direction of the 3D controller rotation; and
initialize the command based on the determined 1D rotational direction.
18. The computer system of claim 17 , wherein the means for initializing the command is configured to initialize the jog command when the determined 1D rotational direction is yaw, and wherein the means for initializing the command is configured to initialize the shuttle command when the determined 1D rotational direction is pitch.
19. The computer system of claim 17 , wherein the means for initializing the command is configured to determine the 1D rotational direction with respect to a controller orientation of the handheld controller at a time associated with the triggering event.
20. The computer system of claim 16 , wherein the means for determining a playback adjustment is configured to determine the playback adjustment based on defined angular increments of the subsequent 3D controller rotation.