Head-coupled kinematic template matching for predicting 3D ray cursors
An electronic device with a display selects a subset of templates from a library of templates. Each of the templates comprises logged movements of one or more predefined input device types in a 3D space and each template includes one or more movement parameters. The electronic device tracks, for a user, movement parameters of at least one input device of the user. The at least one input device corresponds to one of the predefined input device types. The electronic device compares the tracked movement parameters of the at least one input device of the user to the movement parameters for the subset of the templates selected from the library of templates. The electronic device predicts, based on the comparison, a goal target ray, within the 3D space. In some embodiments, the goal target ray is highlighted or visually emphasized according to the prediction.
1. A method of predicting future positions and directions of one or more input devices in 3D spaces, comprising:
at an electronic device having a display, one or more processors, and memory storing one or more programs having instructions for:
selecting a subset of templates from a library of templates, wherein each template in the library of templates comprises logged movements of one or more predefined input device types in a 3D space and each template includes tracking data for one or more movement parameters;
tracking, for a user, movement parameters of at least one input device of the user, the at least one input device corresponding to one of the predefined input device types;
comparing the tracked movement parameters of the at least one input device of the user to the movement parameters for the subset of the templates; and
predicting, based on the comparison, a goal target ray, within the 3D space.
2. The method of claim 1 , wherein the one or more movement parameters of each template comprise a positional velocity and an angular velocity of the one or more predefined input device types.
3. The method of claim 1 , wherein the subset of templates comprises seven templates from the library of templates, the seven templates selected based on similarities between the seven templates and the tracked movement parameters of the at least one input device of the user.
4. The method of claim 1 , wherein comparing the tracked movement parameters of the at least one input device of the user to the movement parameters for the subset of templates comprises generating a weighted score for each movement parameter of the tracked movement parameters.
5. The method of claim 4 , wherein the weights of the weighted score change dynamically for different portions of the user input.
6. The method of claim 1 , wherein the at least one input device of the user comprises a hand of the user or a handheld controller of the user.
7. The method of claim 1 , wherein the at least one input device of the user comprises a head of the user.
8. The method of claim 1 , wherein:
the at least one input device of the user comprises a hand of the user and a head of the user; and
the tracked movement parameters comprise movement parameters of the hand of the user and movement parameters of the head of the user.
9. The method of claim 1 , wherein the tracked movement parameters of the at least one input device of the user is not a full movement of the user.
10. The method of claim 1 , wherein the prediction is made based on only a first portion of the tracked movement parameters of the at least one input device of the user.
11. The method of claim 1 , further comprising, displaying, on the display, a visualization of the goal target ray based on the prediction.
12. The method of claim 11 , wherein the visualization comprises zooming or highlighting the predicted target.
13. The method of claim 1 , wherein the subset of templates is selected based in part on the user.
14. The method of claim 1 , further comprising, prefetching content corresponding to the goal target ray before the user makes the goal target ray.
15. The method of claim 1 , wherein the movement parameters are tracked using accelerometers or other sensors on the at least one input device of the user.
16. An electronic device comprising:
a display;
one or more processors; and
memory storing one or more programs having instructions for:
selecting a subset of templates from a library of templates, wherein each template in the library of templates tracks movements of one or more predefined input device types in a 3D space and each template includes tracking data for one or more movement parameters;
tracking, for a user, movement parameters of at least one input device of the user, the at least one input device corresponding to one of the predefined input device types;
comparing the tracked movement parameters of the at least one input device of the user to the movement parameters for the subset of the templates; and
predicting, based on the comparison, a goal target ray, within the 3D space, displayed on the display for the user.
17. The electronic device of claim 16 , wherein the one or more movement parameters of each template comprise a positional velocity and an angular velocity of the one or more predefined input device types.
18. The electronic device of claim 16 , wherein the at least one input device comprises a first input device and a second input device, and the prediction is made based on tracked movement parameters for a first portion of user input using the first input device and based on tracked movement parameters for a second portion of the user input using the second input device.
19. The electronic device of claim 16 , further comprising, displaying, on the display, a visualization of the goal target ray based on the prediction, wherein the visualization comprises zooming or highlighting the predicted target.
20. The electronic device of claim 16 , further comprising, prefetching content corresponding to the goal target ray before the user makes the goal target ray.