Granted Patent B2

US 12,394,156 · App. 18/158,962 · Granted Aug 19, 2025

Inventors: Sheng Tao (Nanjing, CN); Chunli Li (Nanjing, CN); Hongfeng Zhao (Nanjing, CN); Bin Xiao (Nanjing, CN)

Assignee: NANJING SEMIDRIVE TECHNOLOGY LTD.

G06T19/003G06T7/20

Abstract

Claims (80)

1. An object positioning method comprising:

obtaining an environment image collected by a target object through a camera of the target object and a camera parameter of collecting the environment image;

determining candidate position information of the target object and a virtual camera parameter, the virtual camera parameter and the camera parameter of the environmental image having a first correspondence;

rendering, based on a predetermined three-dimensional model, the candidate position information, and the virtual camera parameter, to generate a rendered image corresponding to the candidate position information, the three-dimensional model and a real space having a second correspondence; and

determining current moment position information of the target object based on the environment image, the rendered image, and the candidate position information, including:

comparing the environmental image and the rendered image; and

in response to the environment image and the rendered image being determined to satisfy a predetermined condition, determining the candidate position information to be the current moment position information of the target object.

2. The method according to claim 1 , wherein determining the candidate position information of the target object includes:

obtaining first moment position information of the target object at a first moment, the first moment being a moment before a current moment; and

determining the candidate position information of the target object based on the first moment position information.

3. The method according to claim 2 , wherein determining the candidate position information of the target object based on the first moment position information includes:

obtaining first movement speed information of the target object;

calculating a movement distance of the target object based on the first moving speed information and time difference information between the current moment and the first moment;

predicting the current moment position information of the target object at the current moment based on the first moment position information and the movement distance of the target object as the candidate position information of the target object.

4. The method according to claim 1 , wherein determining the current moment position information of the target object based on the environment image, the rendered image, and the candidate position information includes:

in response to the environment image and the rendered image not satisfying the predetermined condition, calculating position deviation information based on the environment image and the rendered image, updating the candidate position information based on the position deviation information, returning to obtaining the environment image collected by the target object through the camera of the target object and the camera parameter of collecting the environment image until the environment image and the rendered image are determined to satisfy the predetermined condition, and then determining the updated candidate position information to be the current moment position information of the target object.

5. The method according to claim 4 , wherein:

the position deviation information includes time information of the target object collecting a next frame of the environment image;

after updating the candidate position information based on the position deviation information, in response to the time information, the target object re-collects the environment image through the camera of the target object.

6. The method of claim 4 , further comprising, before comparing the environment image and the rendered image:

determining a model parameter corresponding to the candidate position information in the three-dimensional model; and

processing the environmental image based on the model parameter.

7. The method according to claim 1 , further comprising, after determining the current moment position information of the target object based on the environment image, the rendered image, and the candidate position information:

obtaining second movement speed information of the target object;

predicting second moment position information of the target object at a second moment based on the current moment position information of the target object and the second movement speed information as candidate position information of the second moment, the second moment being a moment after the current moment; and

rendering, based on the predetermined three-dimensional model, the candidate position information at the second moment, and the virtual camera parameter, to generate a rendered image corresponding to the candidate position information at the second moment to generate the rendered image corresponding to the candidate position information at the second moment in advance.

8. The method according to claim 1 , further comprising, after determining the current moment position information of the target object based on the environment image, the rendered image, and the candidate position information:

determining difference information of the three-dimensional model relative to the real space based on the environment image and the rendered image; and

updating the three-dimensional model based on the difference information.

9. The method according to claim 1 ,

wherein the target object includes a plurality of cameras arranged at different directions;

the method further comprising, after determining the candidate position information of the target object:

determining a model parameter corresponding to the candidate position information in the three-dimensional model; and

selecting, based on the model parameter, an environment image collected by a camera in a target direction and the camera parameter of collecting the environment image to be used to determine the current moment position information of the target object.

10. An electronic apparatus comprising:

a processor; and

a memory communicatively coupled to the processor and storing instructions that, when executed by the processor, cause the processor to:

obtain an environment image collected by a target object through a camera of the target object and a camera parameter for collecting the environment image;

determine candidate position information of the target object and a virtual camera parameter, the virtual camera parameter and the camera parameter of the environmental image having a first correspondence;

render, based on a predetermined three-dimensional model, the candidate position information, and the virtual camera parameter, to generate a rendered image corresponding to the candidate position information, the three-dimensional model and a real space having a second correspondence; and

determine current moment position information of the target object based on the environment image, the rendered image, and the candidate position information, including:

comparing the environmental image and the rendered image; and

in response to the environment image and the rendered image being determined to satisfy a predetermined condition, determining the candidate position information to be the current moment position information of the target object.

11. The electronic apparatus according to claim 10 , wherein the processor is further configured to:

obtain first moment position information of the target object at a first moment, the first moment being a moment before a current moment; and

determine the candidate position information of the target object based on the current moment position information at the first moment.

12. The electronic apparatus according to claim 11 , wherein the processor is further configured to:

obtain first movement speed information of the target object;

calculate a movement distance of the target object based on the first moving speed information and time difference information between the current moment and the first moment; and

predict the current moment position information of the target object at the current moment based on the first moment position information and the movement distance of the target object as the candidate position information of the target object.

13. The electronic apparatus according to claim 10 , wherein the processor is further configured to:

in response to the environment image and the rendered image not satisfying the predetermined condition, calculate position deviation information based on the environment image and the rendered image, update the candidate position information based on the position deviation information, return to obtaining the environment image collected by the target object through the camera of the target object and the camera parameter of collecting the environment image until the environment image and the rendered image are determined to satisfy the predetermined condition, and then determine the updated candidate position information to be the current moment position information of the target object.

14. The electronic apparatus according to claim 13 , wherein:

the position deviation information includes time information of the target object collecting a next frame of the environment image;

after updating the candidate position information based on the position deviation information, in response to the time information, the target object re-collects the environment image through the camera of the target object.

15. The electronic apparatus of claim 13 , wherein the processor is further configured to:

determine a model parameter corresponding to the candidate position information in the three-dimensional model; and

process the environmental image based on the model parameter.

16. The electronic apparatus according to claim 10 , wherein the processor is further configured to:

obtain second movement speed information of the target object;

predict second moment position information of the target object at a second moment based on the current moment position information of the target object and the second movement speed information as candidate position information of the second moment, the second moment being a moment after the current moment; and

render, based on the predetermined three-dimensional model, the candidate position information at the second moment, and the virtual camera parameter, to generate a rendered image corresponding to the candidate position information at the second moment to generate the rendered image corresponding to the candidate position information at the second moment in advance.

17. The electronic apparatus according to claim 10 , wherein the processor is further configured to:

determine difference information of the three-dimensional model relative to the real space based on the environment image and the rendered image; and

update the three-dimensional model based on the difference information.

18. The electronic apparatus according to claim 10 , wherein:

the target object includes a plurality of cameras arranged at different directions; and

the processor is further configured to:

determine a model parameter corresponding to the candidate position information in the three-dimensional model; and

select, based on the model parameter, an environment image collected by a camera in a target direction and the camera parameter of collecting the environment image to be used to determine the current moment position information of the target object.

19. A non-transitory computer-readable storage medium storing computer instructions that, when executed by a processor, cause the processor to:

obtain an environment image collected by a target object through a camera of the target object and a camera parameter for collecting the environment image;

determine candidate position information of the target object and a virtual camera parameter, the virtual camera parameter and the camera parameter of the environmental image having a first correspondence;

render, based on a predetermined three-dimensional model, the candidate position information, and the virtual camera parameter, to generate a rendered image corresponding to the candidate position information, the three-dimensional model and a real space having a second correspondence; and

determine current moment position information of the target object based on the environment image, the rendered image, and the candidate position information, including:

comparing the environmental image and the rendered image; and

in response to the environment image and the rendered image being determined to satisfy a predetermined condition, determining the candidate position information to be the current moment position information of the target object.

20. The storage medium according to claim 19 , wherein the processor is further configured to:

obtain first moment position information of the target object at a first moment, the first moment being a moment before a current moment; and

determine the candidate position information of the target object based on the first moment position information.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (13)