Dynamic graceful degradation of augmented-reality effects
In one embodiment, a method includes generating, by a device, first tracking data using a first tracking algorithm, based on first video frames associated with a scene. An augmented-reality (AR) effect may be displayed based on the first tracking data. The device may generate a first confidence score associated with the first tracking data and determine that the first confidence score is above a threshold. The device may generate, based on second video frames subsequent to the first video frames, second tracking data using the first tracking algorithm. The device may determine that an associated second confidence score is below a threshold. In response, the device may generate, based on third video frames subsequent to the second video frames, third tracking data using a second tracking algorithm different from the first. The device may then display the AR effect based on the third tracking data.
1. A method, comprising, by a computing device:
obtaining a first set of video frames associated with a scene;
generating, based on the first set of video frames, first tracking data using a first tracking algorithm;
generating, based on the first tracking data, a first confidence score associated with the first tracking algorithm, wherein the first confidence score is indicative of a confidence level of the first tracking algorithm in tracking objects in the first set of video frames;
displaying an augmented-reality effect based on the first tracking data;
generating a performance score based on a number of frames of the augmented-reality effect displayed based on the first tracking data;
selecting, in response to a determination that the first confidence score and the performance score fail to satisfy one or more criteria, a second tracking algorithm to be used for tracking objects within a second set of video frames subsequent to the first set of video frames;
switching from the first tracking algorithm to the second tracking algorithm based on the selecting;
obtaining the second set of video frames associated with the scene;
generating, based on the second set of video frames, second tracking data using the second tracking algorithm; and
displaying the augmented-reality effect based on the second tracking data.
2. The method of claim 1 , further comprising:
generating a second confidence score associated with the second tracking data;
determining that the second confidence score fails to satisfy a second criteria;
obtaining a third set of video frames associated with the scene, the third set of video frames being subsequent to the second set of video frames; and
displaying the augmented-reality effect with the third set of video frames based on gyroscope data from a gyroscope of the computing device.
3. The method of claim 1 , wherein the first tracking data and the second tracking data are associated with a first object in the scene, the method further comprising:
generating, using the first tracking algorithm, the second tracking algorithm, or a third tracking algorithm, third tracking data associated with a second object in the scene, the third tracking data being based on the first set of video frames or the second set of video frames; and
displaying a second augmented-reality effect based on the third tracking data.
4. The method of claim 1 , further comprising:
transforming the first tracking data to a format compatible with the second tracking algorithm;
wherein the generating of the second tracking data is further based on the transformed first tracking data.
5. The method of claim 1 , further comprising:
determining that one or more capabilities of the computing device are below one or more predetermined capability standards; and
configuring the first tracking algorithm or the second tracking algorithm to reduce tracking granularity.
6. The method of claim 1 , further comprising:
determining that one or more capabilities of the computing device satisfy one or more predetermined capability standards; and
downloading software associated with the first tracking algorithm based on the determining that the one or more capabilities satisfy the one or more predetermined capability standards.
7. The method of claim 1 , wherein the second tracking algorithm is selected from a predetermined hierarchy of tracking algorithms comprising the first tracking algorithm and the second tracking algorithm.
8. The method of claim 1 , wherein the first tracking algorithm is a SLAM algorithm or facial-recognition algorithm, and wherein the second tracking algorithm is a region tracking algorithm.
9. A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to:
obtain a first set of video frames associated with a scene;
generate, based on the first set of video frames, first tracking data using a first tracking algorithm;
generate, based on the first tracking data, a first confidence score associated with the first tracking algorithm, wherein the first confidence score is indicative of a confidence level of the first tracking algorithm in tracking objects in the first set of video frames;
display an augmented-reality effect based on the first tracking data;
generate a performance score based on a number of frames of the augmented-reality effect displayed based on the first tracking data;
select, in response to a determination that the first confidence score and the performance score fail to satisfy one or more criteria, a second tracking algorithm to be used for tracking objects within a second set of video frames subsequent to the first set of video frames;
switch from the first tracking algorithm to the second tracking algorithm based on the selection;
obtain the second set of video frames associated with the scene;
generate, based on the second set of video frames, second tracking data using the second tracking algorithm; and
display the augmented-reality effect based on the second tracking data.
10. The system of claim 9 , wherein the processors are further operable when executing the instructions to:
generate a second confidence score associated with the second tracking data;
determine that the second confidence score fails to satisfy a second criteria;
obtain a third set of video frames associated with the scene, the third set of video frames being subsequent to the second set of video frames; and
display the augmented-reality effect with the third set of video frames based on gyroscope data from a gyroscope of the system.
11. The system of claim 9 , wherein the first tracking data and the second tracking data are associated with a first object in the scene, wherein the processors are further operable when executing the instructions to:
generate, using the first tracking algorithm, the second tracking algorithm, or a third tracking algorithm, third tracking data associated with a second object in the scene, the third tracking data being based on the first set of video frames or the second set of video frames; and
display a second augmented-reality effect based on the third tracking data.
12. The system of claim 9 , wherein the processors are further operable when executing the instructions to:
transform the first tracking data to a format compatible with the second tracking algorithm;
wherein the generation of the second tracking data is further based on the transformed first tracking data.
13. One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to:
obtain a first set of video frames associated with a scene;
generate, based on the first set of video frames, first tracking data using a first tracking algorithm;
generate, based on the first tracking data, a first confidence score associated with the first tracking algorithm, wherein the first confidence score is indicative of a confidence level of the first tracking algorithm in tracking objects in the first set of video frames;
display an augmented-reality effect based on the first tracking data;
generate a performance score based on a number of frames of the augmented-reality effect displayed based on the first tracking data;
select, in response to a determination that the first confidence score and the performance score fail to satisfy one or more criteria, a second tracking algorithm to be used for tracking objects within a second set of video frames subsequent to the first set of video frames;
switch from the first tracking algorithm to the second tracking algorithm based on the selection;
obtain the second set of video frames associated with the scene;
generate, based on the second set of video frames, second tracking data using the second tracking algorithm; and
display the augmented-reality effect based on the second tracking data.
14. The media of claim 13 , wherein the software is further operable when executed to cause the one or more processors to:
generate a second confidence score associated with the second tracking data;
determine that the second confidence score fails to satisfy a second criteria;
obtain a third set of video frames associated with the scene, the third set of video frames being subsequent to the second set of video frames; and
display the augmented-reality effect with the third set of video frames based on gyroscope data from a gyroscope.
15. The media of claim 13 , wherein the first tracking data and the second tracking data are associated with a first object in the scene, wherein the software is further operable when executed to cause the one or more processors to:
generate, using the first tracking algorithm, the second tracking algorithm, or a third tracking algorithm, third tracking data associated with a second object in the scene, the third tracking data being based on the first set of video frames or the second set of video frames; and
display a second augmented-reality effect based on the third tracking data.
16. The media of claim 13 , wherein the software is further operable when executed to cause the one or more processors to:
transform the first tracking data to a format compatible with the second tracking algorithm;
wherein the generation of the second tracking data is further based on the transformed first tracking data.
17. The system of claim 9 , wherein the second tracking algorithm is selected from a predetermined hierarchy of tracking algorithms comprising the first tracking algorithm and the second tracking algorithm.
18. The system of claim 9 , wherein the first tracking algorithm is a SLAM algorithm or facial-recognition algorithm, and wherein the second tracking algorithm is a region tracking algorithm.
19. The media of claim 13 , wherein the second tracking algorithm is selected from a predetermined hierarchy of tracking algorithms comprising the first tracking algorithm and the second tracking algorithm.
20. The media of claim 13 , wherein the first tracking algorithm is a SLAM algorithm or facial-recognition algorithm, and wherein the second tracking algorithm is a region tracking algorithm.