Method and apparatus for object localization in discontinuous observation scene, and storage medium

1 . A method of object localization in a discontinuous observation scene in computer vision, comprising the following steps:

acquiring an object reconstruction model of an object before object observation is interrupted, by performing model reconstruction based on consecutive images of the object obtained before the object observation is interrupted,

acquiring an object point cloud model of the object after the object observation is resumed from the object observation interruption, based on a benchmark image that is obtained when the object observation is resumed; and

based on the object point cloud model and the object reconstruction model, achieving association between objects before the object observation is interrupted and after the object observation is resumed, for the object localization after the object observation is resumed,

wherein, based on the object point cloud model and the object reconstruction model, achieving the association between the objects before the object observation is interrupted and after the object observation is resumed, further comprises:

determining a similarity between the object point cloud model and the object reconstruction model based on object information, wherein the object information comprises at least one of object geometric feature, object texture feature, or object color feature, and

performing the association between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity.

2 . The method of claim 1 , wherein, the consecutive images comprise consecutive RGB images and depth images of the object, and the object reconstruction model is a surfel-based model.

3 . The method of claim 1 , wherein, the benchmark image is a starting image obtained when the object observation is resumed, and a 2.5D instance point cloud is obtained from the starting image as the object point cloud model.

4 . The method of claim 1 , wherein, the object information comprises both the object geometric feature and the object color feature, and the similarity between the object point cloud model and the object reconstruction model is determined based on both the object geometric feature and the object color feature.

5 . The method of claim 1 , wherein, performing the association between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity further comprises:

determining a one-to-one correspondence between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity, so that the objects before the object observation is interrupted and after the object observation is resumed are associated.

6 . The method of claim 5 , wherein, the one-to-one correspondence between the objects before the object observation is interrupted and after the object observation is resumed is determined based on a maximum total similarity between multiple object information and multiple object models.

7 . The method of claim 1 , wherein, the method further comprises:

aligning the associated objects before the object observation is interrupted and after the object observation is resumed.

8 . The method of claim 7 , wherein, spatial transformation is used to align poses of the object after the object observation is resumed with the object before the object observation is interrupted.

9 . An electronic device, comprising:

a memory; and

a processor coupled to the memory, the memory storing instructions thereon, wherein the instructions, when executed by the processor, cause the electronic device to perform:

acquiring an object reconstruction model of an object before object observation is interrupted, by performing model reconstruction based on consecutive images of the object obtained before the object observation is interrupted,

acquiring an object point cloud model of the object after the object observation is resumed from the object observation interruption, based on a benchmark image that is obtained when the object observation is resumed; and

based on the object point cloud model and the object reconstruction model, achieving association between objects before the object observation is interrupted and after the object observation is resumed, for object localization after the object observation is resumed,

wherein, based on the object point cloud model and the object reconstruction model, achieving the association between the objects before the object observation is interrupted and after the object observation is resumed further comprises:

determining a similarity between the object point cloud model and the object reconstruction model based on object information, wherein the object information comprises at least one of object geometric feature, object texture feature, or object color feature, and

performing the association between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity.

10 . The electronic device of claim 9 , wherein, the consecutive images comprise consecutive RGB images and depth images of the object, and the model reconstructed based on the consecutive images is a surfel-based model, or,

wherein the benchmark image is a starting image obtained when the object observation is resumed, and a 2.5D instance point cloud is obtained from the starting image as the object point cloud model.

11 . The electronic device of claim 9 , wherein, performing the association between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity further comprises:

determining a one-to-one correspondence between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity, so that the objects before the object observation is interrupted and after the object observation is resumed are associated.

12 . The electronic device of claim 9 , wherein, the instructions, when executed by the processor, cause the electronic device to further perform:

aligning the associated objects before the object observation is interrupted and after the object observation is resumed.

13 . A non-transitory computer-readable storage medium with a computer program stored thereon, wherein the program, when executed by a processor, causes the processor to implement:

acquiring an object reconstruction model of an object before object observation is interrupted, by performing model reconstruction based on consecutive images of the object obtained before the object observation is interrupted,

acquiring an object point cloud model of the object after the object observation is resumed from the object observation interruption, based on a benchmark image that is obtained when the object observation is resumed; and

based on the object point cloud model and the object reconstruction model, achieving association between objects before the object observation is interrupted and after the object observation is resumed, for object localization after the object observation is resumed, wherein, based on the object point cloud model and the object reconstruction model, achieving the association between the objects before the object observation is interrupted and after the object observation is resumed further comprises:

determining a similarity between the object point cloud model and the object reconstruction model based on object information, wherein the object information comprises at least one of object geometric feature, object texture feature, or object color feature, and

performing the association between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity.

14 . The non-transitory computer-readable storage medium of claim 13 , wherein, the consecutive images comprise consecutive RGB images and depth images of the object, and the model reconstructed based on the consecutive images is a surfel-based model, or,

wherein the benchmark image is a starting image obtained when the object observation is resumed, and a 2.5D instance point cloud is obtained from the starting image as the object point cloud model.

15 . The non-transitory computer-readable storage medium of claim 13 , wherein, performing the association between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity further comprises:

determining a one-to-one correspondence between the objects before the object observation is interrupted and after the object observation is resumed based on the similarity, so that the objects before the object observation is interrupted and after the object observation is resumed are associated.

16 . The non-transitory computer-readable storage medium of claim 13 , wherein the program, when executed by the processor, causes the processor to further implement:

aligning the associated objects before the object observation is interrupted and after the object observation is resumed.