Repositioning method and apparatus in camera pose tracking process, device, and storage medium

1. A repositioning method in a camera pose tracking process, applied to an electronic device having a camera, the electronic device being applied to a process of sequentially performing camera pose tracking on a plurality of anchor images, the method comprising:

obtaining a current image acquired by the camera after an ith anchor image in the plurality of anchor images, i being greater than 1;

in accordance with a determination that the current image satisfies a repositioning condition, wherein the repositioning condition indicates that a tracking process of the current image relative to the ith anchor image fails:

selecting a target keyframe from a keyframe database according to Hash index information, the keyframe database storing Hash index information corresponding to at least one keyframe, the target keyframe being an image that is cached in a camera pose tracking process and has a successful first repositioning relative to another image, wherein first feature point descriptors in the keyframe are classified according to a Hash classification rule and stored in first entry storage locations, and selecting a target keyframe from the keyframe database according to Hash index information includes:

obtaining the second feature point descriptors in the current image;

for each second feature point descriptor, determining a second entry storage location corresponding to the second feature point descriptor; and

determining a target first feature point descriptor with the highest similarity to the second feature point descriptor from the first feature point descriptors stored in the second entry storage location;

performing a second repositioning on the current image relative to the target keyframe; and

calculating a camera pose parameter of the camera used by the camera during acquisition of the current image according to a positioning result of the first repositioning and a positioning result of the second repositioning, the positioning result of the first repositioning and the positioning result of the second repositioning indicate a movement amount of the camera in a real world, and the camera pose parameter includes a displacement vector and a rotation matrix.

2. The method according to claim 1 , wherein the Hash index information comprising m columns*n rows of entry storage locations, and the first entry storage locations corresponds to the m columns*n rows of entry storage locations;

the selecting a target keyframe from a keyframe database according to Hash index information further comprises:

recording a similarity score of the target first feature point descriptor and a keyframe to which the target first feature point descriptor belongs; and

accumulating at least one similarity score belonging to the same keyframe, and determining a keyframe with the highest accumulated similarity score sum as the target keyframe, wherein the second entry storage location is determined from the m columns*n rows of entry storage locations according to the Hash classification rule.

3. The method according to claim 2 , wherein the determining a target entry storage location corresponding to the second feature point descriptor according to the Hash classification rule in the m columns*n rows of entry storage locations comprises:

dividing the second feature point descriptor into m descriptor segments;

obtaining a second Hash value of the ith descriptor segment for an ith descriptor segment in the m descriptor segments; and

determining an entry storage location located at an ith column and a jth row from the m columns*n rows of entry storage locations as a second entry storage location,

the entry storage location at the ith column and the jth row being an entry storage location matching the second Hash value in the n entry storage locations located in the ith column.

4. The method according to claim 1 , further comprising:

obtaining a latest candidate image with successful repositioning;

determining whether the candidate image satisfies an addition condition, the addition condition comprising: a first distance between the candidate image and the first anchor image is greater than a first threshold, and/or, a second distance between the candidate image and a keyframe added last time is greater than a second threshold; and

adding the candidate image as the first keyframe to the keyframe database in a case that the candidate image satisfies the addition condition, wherein the first distance represents an image similarity or a camera pose similarity between the candidate image and the first anchor image and the second distance represents an image similarity or a camera pose similarity between the candidate image and the keyframe added last time.

5. The method according to claim 4 , wherein the adding the candidate image as the first keyframe to the keyframe database comprises:

obtaining first feature point descriptors in the first keyframe;

for each first feature point descriptor, determining a first entry storage location corresponding to the first feature point descriptor from the m columns*n rows of entry storage locations according to the Hash classification rule; and

storing the first feature point descriptor and an identifier of the first keyframe in the first entry storage location.

6. The method according to claim 5 , wherein the determining a first entry storage location corresponding to the first feature point descriptor from the m columns*n rows of entry storage locations according to the Hash classification rule comprises:

dividing the first feature point descriptor into m descriptor segments;

for an ith descriptor segment in the m descriptor segments, obtaining a first Hash value of the ith descriptor segment; and

determining an entry storage location located at an ith column and a jth row from the m columns*n rows of entry storage locations as a first entry storage location,

the entry storage location at the ith column and the jth row being an entry storage location matching the first Hash value in the n entry storage locations located in the ith column.

7. The method according to claim 4 , further comprising:

randomly selecting a second keyframe from the keyframe database in a case that a quantity of keyframes in the keyframe database reaches a maximum value; and

deleting the second keyframe from the keyframe database.

8. The method according to claim 7 , wherein the deleting the keyframe to be deleted from the keyframe database comprises:

obtaining first feature point descriptors in the second keyframe;

for each first feature point descriptor, determining a first entry storage location corresponding to the first feature point descriptor from the m columns*n rows of entry storage locations according to the Hash classification rule; and

deleting the first feature point descriptor and an identifier of the second keyframe from the first entry storage location.

9. The method according to claim 1 , wherein the selecting a target keyframe from a keyframe database according to Hash index information in a case that the current image satisfies a repositioning condition comprises:

performing third repositioning on the current image relative to the first anchor image in a case that the current image satisfies the repositioning condition; and

performing the operation of selecting the target keyframe from the keyframe database according to the Hash index information in a case that the third repositioning fails.

10. An electronic device, comprising a memory and a processor, a camera, and a plurality of programs stored in the memory, wherein the plurality of programs, when executed by the processor, cause the electronic device to sequentially perform a process of camera pose tracking on a plurality of anchor images, the process including:

obtaining a current image acquired by the camera after an ith anchor image in the plurality of anchor images, i being greater than 1;

in accordance with a determination that the current image satisfies a repositioning condition, wherein the repositioning condition indicates that a tracking process of the current image relative to the ith anchor image fails:

selecting a target keyframe from a keyframe database according to Hash index information, the keyframe database storing Hash index information corresponding to at least one keyframe, the target keyframe being an image that is cached in a camera pose tracking process and has a successful first repositioning relative to another image, wherein first feature point descriptors in the keyframe are classified according to a Hash classification rule and stored in first entry storage locations, and selecting a target keyframe from the keyframe database according to Hash index information includes:

obtaining the second feature point descriptors in the current image;

for each second feature point descriptor, determining a second entry storage location corresponding to the second feature point descriptor; and

determining a target first feature point descriptor with the highest similarity to the second feature point descriptor from the first feature point descriptors stored in the second entry storage location;

performing a second repositioning on the current image relative to the target keyframe; and

calculating a camera pose parameter of the camera used by the camera during acquisition of the current image according to a positioning result of the first repositioning and a positioning result of the second repositioning, the positioning result of the first repositioning and the positioning result of the second repositioning indicate a movement amount of the camera in a real world, and the camera pose parameter includes a displacement vector and a rotation matrix.

11. The electronic device according to claim 10 , wherein the Hash index information comprising m columns*n rows of entry storage locations, and the first entry storage locations corresponds to the m columns*n rows of entry storage locations;

the selecting a target keyframe from a keyframe database according to Hash index information further comprises:

recording a similarity score of the target first feature point descriptor and a keyframe to which the target first feature point descriptor belongs; and

accumulating at least one similarity score belonging to the same keyframe, and determining a keyframe with the highest accumulated similarity score sum as the target keyframe, wherein the second entry storage location is determined from the m columns*n rows of entry storage locations according to the Hash classification rule.

12. The electronic device according to claim 11 , wherein the determining a target entry storage location corresponding to the second feature point descriptor according to the Hash classification rule in the m columns*n rows of entry storage locations comprises:

dividing the second feature point descriptor into m descriptor segments;

obtaining a second Hash value of the ith descriptor segment for an ith descriptor segment in the m descriptor segments; and

determining an entry storage location located at an ith column and a jth row from the m columns*n rows of entry storage locations as a second entry storage location,

the entry storage location at the ith column and the jth row being an entry storage location matching the second Hash value in the n entry storage locations located in the ith column.

13. The electronic device according to claim 10 , wherein the process further comprises:

obtaining a latest candidate image with successful repositioning;

determining whether the candidate image satisfies an addition condition, the addition condition comprising: a first distance between the candidate image and the first anchor image is greater than a first threshold, and/or, a second distance between the candidate image and a keyframe added last time is greater than a second threshold; and

adding the candidate image as the first keyframe to the keyframe database in a case that the candidate image satisfies the addition condition, wherein the first distance represents an image similarity or a camera pose similarity between the candidate image and the first anchor image and the second distance represents an image similarity or a camera pose similarity between the candidate image and the keyframe added last time.

14. The electronic device according to claim 13 , wherein the adding the candidate image as the first keyframe to the keyframe database comprises:

obtaining first feature point descriptors in the first keyframe;

for each first feature point descriptor, determining a first entry storage location corresponding to the first feature point descriptor from the m columns*n rows of entry storage locations according to the Hash classification rule; and

storing the first feature point descriptor and an identifier of the first keyframe in the first entry storage location.

15. The electronic device according to claim 14 , wherein the determining a first entry storage location corresponding to the first feature point descriptor from the m columns*n rows of entry storage locations according to the Hash classification rule comprises:

dividing the first feature point descriptor into m descriptor segments;

for an ith descriptor segment in the m descriptor segments, obtaining a first Hash value of the ith descriptor segment; and

determining an entry storage location located at an ith column and a jth row from the m columns*n rows of entry storage locations as a first entry storage location,

the entry storage location at the ith column and the jth row being an entry storage location matching the first Hash value in the n entry storage locations located in the ith column.

16. The electronic device according to claim 13 , wherein the process further comprises:

randomly selecting a second keyframe from the keyframe database in a case that a quantity of keyframes in the keyframe database reaches a maximum value; and

deleting the second keyframe from the keyframe database.

17. The electronic device according to claim 16 , wherein the deleting the keyframe to be deleted from the keyframe database comprises:

obtaining first feature point descriptors in the second keyframe;

for each first feature point descriptor, determining a first entry storage location corresponding to the first feature point descriptor from the m columns*n rows of entry storage locations according to the Hash classification rule; and

deleting the first feature point descriptor and an identifier of the second keyframe from the first entry storage location.

18. The electronic device according to claim 10 , wherein the selecting a target keyframe from a keyframe database according to Hash index information in a case that the current image satisfies a repositioning condition comprises:

performing third repositioning on the current image relative to the first anchor image in a case that the current image satisfies the repositioning condition; and

performing the operation of selecting the target keyframe from the keyframe database according to the Hash index information in a case that the third repositioning fails.

19. A non-transitory computer-readable storage medium storing instructions, the instructions, when executed by a processor of an electronic device having a camera, cause the electronic device to sequentially perform a process of camera pose tracking on a plurality of anchor images, the process including:

obtaining a current image acquired by the camera after an ith anchor image in the plurality of anchor images, i being greater than 1;

in accordance with a determination that the current image satisfies a repositioning condition, wherein the repositioning condition indicates that a tracking process of the current image relative to the ith anchor image fails:

selecting a target keyframe from a keyframe database according to Hash index information, the keyframe database storing Hash index information corresponding to at least one keyframe, the target keyframe being an image that is cached in a camera pose tracking process and has a successful first repositioning relative to another image, wherein first feature point descriptors in the keyframe are classified according to a Hash classification rule and stored in first entry storage locations, and selecting a target keyframe from the keyframe database according to Hash index information includes:

obtaining the second feature point descriptors in the current image;

for each second feature point descriptor, determining a second entry storage location corresponding to the second feature point descriptor; and

determining a target first feature point descriptor with the highest similarity to the second feature point descriptor from the first feature point descriptors stored in the second entry storage location;

performing a second repositioning on the current image relative to the target keyframe; and

calculating a camera pose parameter of the camera used by the camera during acquisition of the current image according to a positioning result of the first repositioning and a positioning result of the second repositioning, the positioning result of the first repositioning and the positioning result of the second repositioning indicate a movement amount of the camera in a real world, and the camera pose parameter includes a displacement vector and a rotation matrix.

20. The non-transitory computer-readable storage medium according to claim 19 , wherein the Hash index information comprising m columns*n rows of entry storage locations, and the first entry storage locations corresponds to the m columns*n rows of entry storage locations;

the selecting a target keyframe from a keyframe database according to Hash index information further comprises:

recording a similarity score of the target first feature point descriptor and a keyframe to which the target first feature point descriptor belongs; and

accumulating at least one similarity score belonging to the same keyframe, and determining a keyframe with the highest accumulated similarity score sum as the target keyframe, wherein the second entry storage location is determined from the m columns*n rows of entry storage locations according to the Hash classification rule.