Granted Patent B2

US 12,387,371 · App. 17/705,754 · Granted Aug 12, 2025

Inventors: Mi Tian (Beijing, CN); Qiong Nie (Beijing, CN); Hao Shen (Beijing, CN); Huaxia Xia (Beijing, CN)

Assignee: BEIJING SANKUAI ONLINE TECHNOLOGY CO., LTD

G06T7/74G06T2207/30252

Abstract

Claims (72)

1. A pose determining method, comprising:

obtaining environment image data acquired by an unmanned driving device;

determining, for each piece of reference image data matching the environment image data and according to an actual pose corresponding to the piece of reference image data, a pose of the unmanned driving device when acquiring the environment image data as a predicted pose of the unmanned driving device in the piece of reference image data;

determining pose deviation representation information of the piece of reference image data according to the predicted pose and actual poses corresponding to other pieces of reference image data, wherein the pose deviation representation information is used for representing degrees of deviation between poses of the unmanned driving device when acquiring the other pieces of reference image data and the actual poses corresponding to the other pieces of reference image data using the predicted pose of the unmanned driving device in the piece of reference image data as a condition; and

selecting target image data from respective pieces of reference image data according to the pose deviation representation information corresponding to the piece of reference image data, and determining, according to the target image data, a pose of the unmanned driving device when acquiring the environment image data.

2. The method according to claim 1 , wherein the determining, for each piece of reference image data matching the environment image data and according to the actual pose corresponding to the piece of reference image data, the pose of the unmanned driving device when acquiring the environment image data as the predicted pose of the unmanned driving device in the piece of reference image data comprises:

predicting, for each piece of reference image data matching the environment image data, a relative pose between the unmanned driving device when the environment image data is acquired and the unmanned driving device when the piece of reference image data is acquired; and

determining, according to the relative pose and the actual pose corresponding to the piece of reference image data, the predicted pose of the unmanned driving device when acquiring the environment image data.

3. The method according to claim 2 , wherein the predicting, for each piece of reference image data matching the environment image data, the relative pose between the unmanned driving device when the environment image data is acquired and the unmanned driving device when the piece of reference image data is acquired comprises:

inputting, for each piece of reference image data matching the environment image data, the environment image data and the piece of reference image data into a relative pose prediction model, and predicting the relative pose between the unmanned driving device when the environment image data is acquired and the unmanned driving device when the piece of reference image data is acquired.

4. The method according to claim 1 , wherein determining each piece of reference image data matching the environment image data comprises:

predicting, according to the environment image data, a basic pose of the unmanned driving device when acquiring the environment image data;

determining, from a database, candidate image data of which actual poses fall within a set range of the basic pose; and

determining, from the candidate image data, reference image data of which an image similarity with the environment image data is not less than a set first similarity.

5. The method according to claim 4 , wherein the predicting, according to the environment image data, the basic pose of the unmanned driving device when acquiring the environment image data comprises:

inputting the environment image data into a global pose prediction model, to predict the basic pose of the unmanned driving device when acquiring the environment image data.

6. The method according to claim 5 , wherein training the global pose prediction model comprises:

obtaining first historical environment image data acquired by a first specified device;

inputting, for each piece of first historical environment image data, the first historical environment image data into a to-be-trained global pose prediction model, to predict a pose of the first specified device when acquiring the first historical environment image data; and

training the global pose prediction model by using minimization of a deviation between the predicted pose and an actual pose of the first specified device when acquiring the first historical environment image data as an optimization target.

7. The method according to claim 1 , wherein the selecting target image data from the respective pieces of reference image data according to the pose deviation representation information corresponding to the piece of reference image data comprises:

determining, for each piece of reference image data and according to the pose deviation representation information corresponding to the piece of reference image data, a quantity of other pieces of reference image data meeting a preset condition in a case that the unmanned driving device determines the predicted pose according to the piece of reference image data, wherein the preset condition is that the degrees of deviation between the actual poses corresponding to the other pieces of reference image data and the poses of the unmanned driving device when acquiring the other pieces of reference image data are less than a set threshold; and

selecting the target image data from the respective pieces of reference image data according to the quantity.

8. The method according to claim 3 , wherein training the relative pose prediction model comprises:

obtaining second historical environment image data acquired by a second specified device;

determining, for each piece of second historical environment image data, associated historical environment image data of which a similarity with the second historical environment image data is not less than a set second similarity from other pieces of second historical environment image data;

determining an actual relative pose between the second specified device when the second historical environment image data is acquired and the second specified device when the associated historical environment image data is acquired;

inputting the second historical environment image data and the associated historical environment image data into a to-be-trained relative pose prediction model, to obtain a predicted relative pose between the second specified device when the second historical environment image data is acquired and the second specified device when the associated historical environment image data is acquired; and

training the relative pose prediction model by using minimization of a deviation between the predicted relative pose and the actual relative pose as an optimization target.

9. A non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when being executed by a processor, implements the following steps:

obtaining environment image data acquired by an unmanned driving device;

determining, for each piece of reference image data matching the environment image data and according to an actual pose corresponding to the piece of reference image data, a pose of the unmanned driving device when acquiring the environment image data as a predicted pose of the unmanned driving device in the piece of reference image data;

determining pose deviation representation information of the piece of reference image data according to the predicted pose and actual poses corresponding to other pieces of reference image data, wherein the pose deviation representation information is used for representing degrees of deviation between poses of the unmanned driving device when acquiring the other pieces of reference image data and the actual poses corresponding to the other pieces of reference image data using the predicted pose of the unmanned driving device in the piece of reference image data as a condition; and

selecting target image data from respective pieces of reference image data according to the pose deviation representation information corresponding to the piece of reference image data, and determining, according to the target image data, a pose of the unmanned driving device when acquiring the environment image data.

10. The non-transitory computer-readable storage medium according to claim 9 , wherein the computer program, when being executed by the processor, implements the following steps:

predicting, for each piece of reference image data matching the environment image data, a relative pose between the unmanned driving device when the environment image data is acquired and the unmanned driving device when the piece of reference image data is acquired; and

determining, according to the relative pose and the actual pose corresponding to the piece of reference image data, the predicted pose of the unmanned driving device when acquiring the environment image data.

11. The non-transitory computer-readable storage medium according to claim 10 , wherein the computer program, when being executed by the processor, implements the following steps:

inputting, for each piece of reference image data matching the environment image data, the environment image data and the piece of reference image data into a relative pose prediction model, and predicting the relative pose between the unmanned driving device when the environment image data is acquired and the unmanned driving device when the piece of reference image data is acquired.

12. The non-transitory computer-readable storage medium according to claim 9 , wherein the computer program, when being executed by the processor, implements the following steps:

predicting, according to the environment image data, a basic pose of the unmanned driving device when acquiring the environment image data;

determining, from a database, candidate image data of which actual poses fall within a set range of the basic pose; and

determining, from the candidate image data, reference image data of which an image similarity with the environment image data is not less than a set first similarity.

13. The non-transitory computer-readable storage medium according to claim 12 , wherein the computer program, when being executed by the processor, implements the following steps:

inputting the environment image data into a global pose prediction model, to predict a basic pose of the unmanned driving device when acquiring the environment image data.

14. The non-transitory computer-readable storage medium according to claim 13 , wherein the computer program, when being executed by the processor, implements the following steps:

obtaining first historical environment image data acquired by a first specified device;

inputting, for each piece of first historical environment image data, the first historical environment image data into a to-be-trained global pose prediction model, to predict a pose of the first specified device when acquiring the first historical environment image data; and

training the global pose prediction model by using minimization of a deviation between the predicted pose and an actual pose of the first specified device when acquiring the first historical environment image data as an optimization target.

15. The non-transitory computer-readable storage medium according to claim 9 , wherein the computer program, when being executed by the processor, implements the following steps:

determining, for each piece of reference image data and according to the pose deviation representation information corresponding to the piece of reference image data, a quantity of other pieces of reference image data meeting a preset condition in a case that the unmanned driving device determines the predicted pose according to the piece of reference image data, wherein the preset condition is that the degrees of deviation between the actual poses corresponding to the other pieces of reference image data and the poses of the unmanned driving device when acquiring the other pieces of reference image data are less than a set threshold; and

selecting the target image data from the respective pieces of reference image data according to the quantity.

16. The non-transitory computer-readable storage medium according to claim 11 , wherein the computer program, when being executed by the processor, implements the following steps:

obtaining second historical environment image data acquired by a second specified device;

determining, for each piece of second historical environment image data, associated historical environment image data of which a similarity with the second historical environment image data is not less than a set second similarity from other pieces of second historical environment image data;

determining an actual relative pose between the second specified device when the second historical environment image data is acquired and the second specified device when the associated historical environment image data is acquired;

inputting the second historical environment image data and the associated historical environment image data into a to-be-trained relative pose prediction model, to obtain a predicted relative pose between the second specified device when the second historical environment image data is acquired and the second specified device when the associated historical environment image data is acquired; and

training the relative pose prediction model by using minimization of a deviation between the predicted relative pose and the actual relative pose as an optimization target.

17. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of being run on the processor, wherein when executing the program, the processor implements the following steps:

obtaining environment image data acquired by an unmanned driving device;

determining, for each piece of reference image data matching the environment image data and according to an actual pose corresponding to the piece of reference image data, a pose of the unmanned driving device when acquiring the environment image data as a predicted pose of the unmanned driving device in the piece of reference image data;

determining pose deviation representation information of the piece of reference image data according to the predicted pose and actual poses corresponding to other pieces of reference image data, wherein the pose deviation representation information is used for representing degrees of deviation between poses of the unmanned driving device when acquiring the other pieces of reference image data and the actual poses corresponding to the other pieces of reference image data using the predicted pose of the unmanned driving device in the piece of reference image data as a condition; and

selecting target image data from respective pieces of reference image data according to the pose deviation representation information corresponding to the piece of reference image data, and determining, according to the target image data, a pose of the unmanned driving device when acquiring the environment image data.

18. The electronic device according to claim 17 , wherein when executing the program, the processor implements the following steps:

predicting, for each piece of reference image data matching the environment image data, a relative pose between the unmanned driving device when the environment image data is acquired and the unmanned driving device when the piece of reference image data is acquired; and

determining, according to the relative pose and the actual pose corresponding to the piece of reference image data, the predicted pose of the unmanned driving device when acquiring the environment image data.

19. The electronic device according to claim 18 , wherein when executing the program, the processor implements the following steps:

inputting, for each piece of reference image data matching the environment image data, the environment image data and the piece of reference image data into a relative pose prediction model, and predicting the relative pose between the unmanned driving device when the environment image data is acquired and the unmanned driving device when the piece of reference image data is acquired.

20. The electronic device according to claim 17 , wherein when executing the program, the processor implements the following steps:

predicting, according to the environment image data, a basic pose of the unmanned driving device when acquiring the environment image data;

determining, from a database, candidate image data of which actual poses fall within a set range of the basic pose; and

determining, from the candidate image data, reference image data of which an image similarity with the environment image data is not less than a set first similarity.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (12)