Granted Patent B2

US 12,085,722 · App. 18/475,415 · Granted Sep 10, 2024

Inventors: Yunlong Li (Beijing, CN); Qinglin Chen (Beijing, CN); Cheng Chen (Beijing, CN); Junliang Shan (Beijing, CN); Tao Wu (Beijing, CN)

Assignee: BEIJING ZITIAO NETWORK TECHNOLOGY CO., LTD.

G02B27/0172G02B27/0093G02B2027/0138G02B2027/0178

Abstract

Claims (79)

1. A positioning method, comprising:

extracting multiple light spots from a current image frame, wherein the current image frame is an image currently collected by an extended reality device;

determining an interference light spot in the multiple light spots according to information about a historical interference light spot, and removing the interference light spot in the multiple light spots, wherein the historical interference light spot is obtained by recognizing a light spot of a historical image frame;

recognizing whether there is a light spot of a luminous light source in remaining light spots after removing the interference light spot, wherein the luminous light source is a light source set on a human-computer interaction apparatus, and the human-computer interaction apparatus is used for a user to interact with the extended reality device;

determining a current pose of the human-computer interaction apparatus according to the light spot of the luminous light source recognized in the remaining light spots.

2. The method according to claim 1 , wherein determining the interference light spot in the multiple light spots according to the information about the historical interference light spot, comprises:

mapping an interference light spot in a result of historical light spot recognition to the current image frame;

recognizing the interference light spot from the multiple current light spots according to a result of the mapping.

3. The method according to claim 2 , further comprising:

if the light spot of the luminous light source is not recognized in the remaining light spots, taking the remaining light spots as interference light spots corresponding to the current image frame, and updating the information about the historical interference light spot according to information about the interference light spots corresponding to the current image frame.

4. The method according to claim 1 , wherein recognizing whether there is the light spot of the luminous light source in the remaining light spots after removing the interference light spot, comprises:

recognizing, based on an arrangement characteristics of luminous light sources set on the human-computer interaction apparatus, whether there is the light spot of the luminous light source from the remaining light spots;

wherein determining the current pose of the human-computer interaction apparatus according to the light spot of the luminous light source recognized in the remaining light spots, comprises:

determining a serial number corresponding to a respective light spot of a luminous light source according to serial numbers of multiple luminous light sources forming the arrangement characteristics;

determining the pose of the human-computer interaction apparatus according to position information of the light spot of the luminous light source whose serial number is determined.

5. The method according to claim 4 , wherein a number of luminous light sources set on the human-computer interaction apparatus is more than two, the more than two luminous light sources form at least one light source combination;

wherein recognizing, based on the arrangement characteristics of the luminous light sources set on the human-computer interaction apparatus, whether there is the light spot of the luminous light source from multiple light spots with the interference light spot removed, comprises:

determining at least one light spot combination of composed of the remaining light spots;

for each light spot combination:

determining a distribution characteristics of light spots in the light spot combination, and performing matching between the distribution characteristics of the light spots in the light spot combination and a light source arrangement characteristics corresponding to the at least one light source combination respectively;

if the matching is successful, taking a light spot in the light spot combination as the light spot of the luminous light source;

wherein determining the current pose of the human-computer interaction apparatus according to the light spot of the luminous light source recognized in the remaining light spots, comprises:

determining a serial number of a respective light spot in the light spot combination according to a serial number of a respective light source in a light source combination successfully matched with the light spot combination;

determining the current pose of the human-computer interaction apparatus according to position information of the respective light spot in the light spot combination for which the serial number of the respective light spot is determined.

6. The method according to claim 5 , wherein there are more than two light spot combinations successfully matched with the light source arrangement characteristics corresponding to the at least one light source combination respectively;

wherein determining the current pose of the human-computer interaction apparatus according to the light spot of the luminous light source recognized in the remaining light spots, comprises:

determining a target light spot combination from the more than two light spot combinations according to a preset matching threshold;

determining a serial number of a respective light spot in the target light spot combination according to a serial number of a light source combination corresponding to the target light spot combination;

determining the current pose of the human-computer interaction apparatus according to position information of the respective light spot in the target light spot combination for which the serial number of the respective light spot is determined.

7. The method according to claim 6 , further comprising:

for a next image frame, determining a pose of the human-computer interaction apparatus corresponding to the next image frame using position information of the respective light spot in the target light spot combination.

8. The method according to claim 7 , further comprising:

if the light spot of the luminous light source is not recognized in the remaining light spots, taking the remaining light spots as interference light spots corresponding to the current image frame, and updating the information about the historical interference light spot according to information about the interference light spots corresponding to the current image frame.

9. The method according to claim 4 , further comprising:

if the light spot of the luminous light source is not recognized in the remaining light spots, taking the remaining light spots as interference light spots corresponding to the current image frame, and updating the information about the historical interference light spot according to information about the interference light spots corresponding to the current image frame.

10. The method according to claim 5 , further comprising:

if the light spot of the luminous light source is not recognized in the remaining light spots, taking the remaining light spots as interference light spots corresponding to the current image frame, and updating the information about the historical interference light spot according to information about the interference light spots corresponding to the current image frame.

11. The method according to claim 6 , further comprising:

if the light spot of the luminous light source is not recognized in the remaining light spots, taking the remaining light spots as interference light spots corresponding to the current image frame, and updating the information about the historical interference light spot according to information about the interference light spots corresponding to the current image frame.

12. The method according to claim 1 , further comprising:

if the light spot of the luminous light source is not recognized in the remaining light spots, taking the remaining light spots as interference light spots corresponding to the current image frame, and updating the information about the historical interference light spot according to information about the interference light spots corresponding to the current image frame.

13. An electronic device, comprising a processor and a memory;

the memory stores computer execution instructions;

the processor executes the computer execution instructions stored in the memory; wherein the computer execution instructions, when executed by the processor, cause the processor to:

extract multiple light spots from a current image frame, wherein the current image frame is an image currently collected by an extended reality device;

determine an interference light spot in the multiple light spots according to information about a historical interference light spot, and remove the interference light spot in the multiple light spots, wherein the historical interference light spot is obtained by recognizing a light spot of a historical image frame;

recognize whether there is a light spot of a luminous light source in remaining light spots after removing the interference light spot, wherein the luminous light source is a light source set on a human-computer interaction apparatus, and the human-computer interaction apparatus is used for a user to interact with the extended reality device;

determine a current pose of the human-computer interaction apparatus according to the light spot of the luminous light source recognized in the remaining light spots.

14. The electronic device according to claim 13 , wherein the processor is caused to:

map an interference light spot in a result of historical light spot recognition to the current image frame;

recognize the interference light spot from the multiple current light spots according to a result of the mapping.

15. The electronic device according to claim 13 , wherein the processor is caused to:

recognize, based on an arrangement characteristics of luminous light sources set on the human-computer interaction apparatus, whether there is the light spot of the luminous light source from the remaining light spots;

wherein the processor is caused to:

determine a serial number corresponding to a respective light spot of a luminous light source according to serial numbers of multiple luminous light sources forming the arrangement characteristics;

determine the pose of the human-computer interaction apparatus according to position information of the light spot of the luminous light source whose serial number is determined.

16. The electronic device according to claim 15 , wherein a number of luminous light sources set on the human-computer interaction apparatus is more than two, the more than two luminous light sources form at least one light source combination;

wherein the processor is caused to:

determine at least one light spot combination of composed of the remaining light spots;

for each light spot combination:

determine a distribution characteristics of light spots in the light spot combination, and perform matching between the distribution characteristics of the light spots in the light spot combination and a light source arrangement characteristics corresponding to the at least one light source combination respectively;

if the matching is successful, taking a light spot in the light spot combination as the light spot of the luminous light source;

wherein the processor is caused to:

determine a serial number of a respective light spot in the light spot combination according to a serial number of a respective light source in a light source combination successfully matched with the light spot combination;

determine the current pose of the human-computer interaction apparatus according to position information of the respective light spot in the light spot combination for which the serial number of the respective light spot is determined.

17. The electronic device according to claim 16 , wherein there are more than two light spot combinations successfully matched with the light source arrangement characteristics corresponding to the at least one light source combination respectively;

wherein the processor is caused to:

determine a target light spot combination from the more than two light spot combinations according to a preset matching threshold;

determine a serial number of a respective light spot in the target light spot combination according to a serial number of a light source combination corresponding to the target light spot combination;

determine the current pose of the human-computer interaction apparatus according to position information of the respective light spot in the target light spot combination for which the serial number of the respective light spot is determined.

18. The electronic device according to claim 17 , wherein the processor is further caused to:

for a next image frame, determine a pose of the human-computer interaction apparatus corresponding to the next image frame using position information of the respective light spot in the target light spot combination.

19. The electronic device according to claim 13 , wherein the processor is caused to:

if the light spot of the luminous light source is not recognized in the remaining light spots, take the remaining light spots as interference light spots corresponding to the current image frame, and update the information about the historical interference light spot according to information about the interference light spots corresponding to the current image frame.

20. A non-transitory computer-readable storage medium in which computer execution instructions are stored, when a processor executes the computer execution instructions, the positioning method comprising the following steps is implemented:

extracting multiple light spots from a current image frame, wherein the current image frame is an image currently collected by an extended reality device;

determining an interference light spot in the multiple light spots according to information about a historical interference light spot, and removing the interference light spot in the multiple light spots, wherein the historical interference light spot is obtained by recognizing a light spot of a historical image frame;

recognizing whether there is a light spot of a luminous light source in remaining light spots after removing the interference light spot, wherein the luminous light source is a light source set on a human-computer interaction apparatus, and the human-computer interaction apparatus is used for a user to interact with the extended reality device;

determining a current pose of the human-computer interaction apparatus according to the light spot of the luminous light source recognized in the remaining light spots.

Assignments (10)

Priority Claims (1)

Continuity (2)