Object annotation method and apparatus, movement control method and apparatus, device, and storage medium
View Patent ↗Embodiments of this application disclose an object annotation method and apparatus, a movement control method and apparatus, a device, and a storage medium. The method includes: obtaining a reference image recorded by an image sensor from an environment space, the reference image comprising at least one reference object; obtaining target point cloud data obtained by a three-dimensional space sensor by scanning the environment space, the target point cloud data indicating a three-dimensional space region occupied by a target object in the environment space; determining a target reference object corresponding to the target object from the reference image; determining a projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image; and performing three-dimensional annotation on the target reference object in the reference image according to the determined projection size.
1. An object annotation method, comprising:
obtaining a reference image of an environment space, the reference image comprising at least one reference object;
obtaining target point cloud data by scanning the environment space with a three-dimensional space sensor, the target point cloud data indicating a three-dimensional space region occupied by a target object in the environment space;
determining a target reference object corresponding to the target object from the reference image;
classifying, based on the target point cloud data, the target reference object into a corresponding classification category, to obtain an object information of the target reference object, wherein the object information includes the classification category of the target reference object;
obtaining a standard size value set for the classification category of the target reference object;
determining, according to the standard size value, a projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image; and
performing three-dimensional annotation on the target reference object in the reference image according to the determined projection size.
2. The method according to claim 1 , wherein the object information comprises a two-dimensional annotation size of the target reference object; and the method further comprises:
obtaining a height value of the two-dimensional annotation size; and
determining, according to the height value, a height value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image.
3. The method according to claim 1 , wherein
the standard size value comprises a standard length value and/or a standard width value; and
the method further comprises: determining, according to the standard size value, a length value and a width value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image.
4. The method according to claim 3 , wherein
in response to that the standard size value comprises the standard length value, the determining, according to the standard size value, a length value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image comprises:
adjusting a length value of the three-dimensional space region to the standard length value in response to that a difference between the length value of the three-dimensional space region and the standard length value is greater than a preset length difference, and determining the length value in the projection size according to an adjusted length value of the three-dimensional space region; and
in response to that the standard size value comprises the standard width value, the determining, according to the standard size value, a width value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image comprises:
adjusting a width value of the three-dimensional space region to the standard width value in response to that a difference between the width value of the three-dimensional space region and the standard width value is greater than a preset width difference, and determining the width value in the projection size according to an adjusted width value of the three-dimensional space region.
5. The method according to claim 1 , wherein the determining a target reference object corresponding to the target object from the reference image comprises:
matching the target point cloud data and a reference enclosing box of the at least one reference object in the reference image, to determine a matching degree between the target object corresponding to the target point cloud data and the at least one reference object; and
determining the target reference object from the at least one reference object according to the matching degree, the matching degree between the target reference object and the target object corresponding to the target point cloud data being greater than a preset threshold.
6. The method according to claim 5 , wherein the matching the target point cloud data and a reference enclosing box of the at least one reference object in the reference image, to determine a matching degree between the target object corresponding to the target point cloud data and the at least one reference object comprises:
performing two-dimensional image projection on the three-dimensional space region corresponding to the target point cloud data, to obtain an image projection region of the target point cloud data in the reference image;
determining a minimum enclosing box of the image projection region; and
calculating the matching degree between the target object corresponding to the target point cloud data and the at least one reference object based on matching the minimum enclosing box and the reference enclosing box of the at least one reference object in the reference image.
7. The method according to claim 6 , wherein calculating the matching degree between the target object corresponding to the target point cloud data and the at least one reference object based on matching the minimum enclosing box and the reference enclosing box of the at least one reference object in the reference image comprises:
calculating an intersection over union between the minimum enclosing box and the reference enclosing box of the at least one reference object in the reference image; and
using the intersection over union as the matching degree between the target object corresponding to the target point cloud data and the at least one reference object.
8. The method according to claim 5 , wherein the determining the target reference object from the at least one reference object according to the matching degree comprises:
determining a candidate reference object from the at least one reference object according to the matching degree, the matching degree between the candidate reference object and the target object corresponding to the target point cloud data being greater than the preset threshold; and
determining the target reference object according to a classification category of the candidate reference object and a classification category of the target object corresponding to the target point cloud data.
9. The method according to claim 1 , wherein the obtaining target point cloud data by scanning the environment space with a three-dimensional space sensor comprises:
obtaining a point cloud data frame set by scanning the environment space with the three-dimensional space sensor;
tracking the target object based on the point cloud data frame set, to determine point cloud trajectory data of the target object;
determining reference point cloud data from the point cloud trajectory data according to a recording timestamp of the reference image; and
performing interpolation on the reference point cloud data to obtain the target point cloud data.
10. The method according to claim 9 , wherein the point cloud data frame set comprises at least two point cloud data frames, and the tracking on the target object based on the point cloud data frame set, to determine point cloud trajectory data of the target object comprises:
performing object detection on a target point cloud data frame, to determine the target object, the target point cloud data frame being a point cloud data frame in the point cloud data frame set;
obtaining an associated point cloud data frame associated with the target point cloud data frame, the associated point cloud data frame comprising one point cloud data frame having an acquisition time earlier or later than that of the target point cloud data frame; and
tracking the target object based on the associated point cloud data frame, to determine the point cloud trajectory data of the target object.
11. An object annotation apparatus, comprising: at least one memory storing computer readable instructions, and at least one processor configured, when executing the computer readable instructions, to perform:
obtaining a reference image of an environment space, the reference image comprising at least one reference object;
obtaining target point cloud data by scanning the environment space with a three-dimensional space sensor, the target point cloud data indicating a three-dimensional space region occupied by a target object in the environment space;
determining a target reference object corresponding to the target object from the reference image;
classifying, based on the target point cloud data, the target reference object into a corresponding classification category, to obtain an object information of the target reference object, wherein the object information includes the classification category of the target reference object;
obtaining a standard size value set for the classification category of the target reference object;
determining, according to the standard size value, a projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image; and
performing three-dimensional annotation on the target reference object in the reference image according to the determined projection size.
12. The apparatus according to claim 11 , wherein the object information comprises a two-dimensional annotation size of the target reference object; and the at least one processor is configured to perform:
obtaining a height value of the two-dimensional annotation size; and
determining, according to the height value, a height value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image.
13. The apparatus according to claim 11 , wherein the standard size value comprises:
a standard length value and/or a standard width value; and
the at least one processor is configured to perform: determining, according to the standard size value, a length value and a width value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image.
14. The apparatus according to claim 13 , wherein:
in response to that the standard size value comprises the standard length value, the determining, according to the standard size value, a length value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image comprises:
adjusting a length value of the three-dimensional space region to the standard length value in response to that a difference between the length value of the three-dimensional space region and the standard length value is greater than a preset length difference, and determining the length value in the projection size according to an adjusted length value of the three-dimensional space region; and
in response to that the standard size value comprises the standard width value, the determining, according to the standard size value, a width value in the projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image comprises:
adjusting a width value of the three-dimensional space region to the standard width value in response to that a difference between the width value of the three-dimensional space region and the standard width value is greater than a preset width difference, and determining the width value in the projection size according to an adjusted width value of the three-dimensional space region.
15. The apparatus according to claim 11 , wherein the determining a target reference object corresponding to the target object from the reference image comprises:
matching the target point cloud data and a reference enclosing box of the at least one reference object in the reference image, to determine a matching degree between the target object corresponding to the target point cloud data and the at least one reference object; and
determining the target reference object from the at least one reference object according to the matching degree, the matching degree between the target reference object and the target object corresponding to the target point cloud data being greater than a preset threshold.
16. The apparatus according to claim 15 , wherein the determining the target reference object from the at least one reference object according to the matching degree comprises:
determining a candidate reference object from the at least one reference object according to the matching degree, the matching degree between the candidate reference object and the target object corresponding to the target point cloud data being greater than the preset threshold; and
determining the target reference object according to a classification category of the candidate reference object and a classification category of the target object corresponding to the target point cloud data.
17. The apparatus according to claim 15 , wherein the matching the target point cloud data and a reference enclosing box of the at least one reference object in the reference image, to determine a matching degree between the target object corresponding to the target point cloud data and the at least one reference object comprises:
performing two-dimensional image projection on the three-dimensional space region corresponding to the target point cloud data, to obtain an image projection region of the target point cloud data in the reference image;
determining a minimum enclosing box of the image projection region; and
calculating the matching degree between the target object corresponding to the target point cloud data and the at least one reference object based on matching the minimum enclosing box and the reference enclosing box of the at least one reference object in the reference image.
18. The apparatus according to claim 17 , wherein calculating the matching degree between the target object corresponding to the target point cloud data and the at least one reference object based on matching the minimum enclosing box and the reference enclosing box of the at least one reference object in the reference image comprises:
calculating an intersection over union between the minimum enclosing box and the reference enclosing box of the at least one reference object in the reference image; and
using the intersection over union as the matching degree between the target object corresponding to the target point cloud data and the at least one reference object.
19. The apparatus according to claim 11 , wherein the obtaining target point cloud data by scanning the environment space with a three-dimensional space sensor comprises:
obtaining a point cloud data frame set by scanning the environment space with the three-dimensional space sensor;
tracking the target object based on the point cloud data frame set, to determine point cloud trajectory data of the target object;
determining reference point cloud data from the point cloud trajectory data according to a recording timestamp of the reference image; and
performing interpolation on the reference point cloud data to obtain the target point cloud data.
20. A non-transitory computer readable storage medium, the computer storage medium storing a computer program instruction that, when executed, causing at least one processor to perform:
obtaining a reference image of an environment space, the reference image comprising at least one reference object;
obtaining target point cloud data by scanning the environment space with a three-dimensional space sensor, the target point cloud data indicating a three-dimensional space region occupied by a target object in the environment space;
determining a target reference object corresponding to the target object from the reference image;
classifying, based on the target point cloud data, the target reference object into a corresponding classification category, to obtain an object information of the target reference object, wherein the object information includes the classification category of the target reference object;
obtaining a standard size value set for the classification category of the target reference object;
determining, according to the standard size value, a projection size of the three-dimensional space region corresponding to the target point cloud data and the three-dimensional space region being projected onto the reference image; and
performing three-dimensional annotation on the target reference object in the reference image according to the determined projection size.