Target detection method, device, terminal device, and medium
View Patent ↗The present disclosure provides a target detection method. The method includes: acquiring a scene image of a scene; acquiring a three-dimensional point cloud corresponding to the scene; identifying a supporting plane in the scene based on the three-dimensional point cloud; generating a plurality of region proposals based on the scene image and the supporting plane; and performing a target detection on the plurality of region proposals to determine a target object to be detected in the scene image. In addition, the present disclosure also provides a target detection device, a terminal device, and a medium.
1. A target detection method, comprising:
acquiring a scene image of a scene;
acquiring a three-dimensional point cloud corresponding to the scene;
identifying a supporting plane in the scene based on the three-dimensional point cloud;
generating a plurality of region proposals for a target object to be detected based on the scene image and the supporting plane; and
performing a target detection on the plurality of region proposals to determine the target object to be detected in the scene image,
wherein generating the plurality of region proposals for the target object to be detected based on the scene image and the supporting plane, comprises:
generating a plurality of sampling boxes;
determining a moving range of the plurality of sampling boxes based on a position and an orientation of the supporting plane;
controlling the plurality of sampling boxes to move on the supporting plane by a preset step size in the moving range to generate a plurality of three-dimensional regions; and
projecting the plurality of three-dimensional regions to the scene image to form the plurality of region proposals;
wherein the plurality of sampling boxes and the supporting plane are three-dimensional virtual area.
2. The target detection method according to claim 1 , wherein acquiring the three-dimensional point cloud corresponding to the scene, comprises:
scanning the scene by a simultaneous localization and mapping (SLAM) system to generate the three-dimensional point cloud corresponding to the scene.
3. The target detection method according to claim 1 , wherein identifying the supporting plane in the scene based on the three-dimensional point cloud, comprises:
identifying a horizontal plane or a vertical plane in the scene based on the three-dimensional point cloud, and determining the horizontal plane or the vertical plane as the supporting plane.
4. The target detection method according to claim 1 , wherein generating the plurality of sampling boxes, comprises:
acquiring a target object of interest to the user;
acquiring parameters of the target object of interest; and
generating parameters of the plurality of sampling boxes based on the parameters of the target object of interest.
5. The target detection method according to claim 1 , wherein said identifying the supporting plane in the scene based on the three-dimensional point cloud comprises:
identifying the supporting plane in the scene based on the three-dimensional point cloud by a Random Sample Consensus (RANSAC) algorithm, the three-dimensional point cloud being an input of the RANSAC algorithm.
6. The target detection method according to claim 1 , wherein the supporting plane is a plane used to place actual objects or virtual objects in the scene.
7. A terminal device, comprising: a memory, a processor, and computer programs stored in the memory and executable by the processor, wherein the processor is configured to execute the computer programs to:
acquire a scene image of a scene;
acquire a three-dimensional point cloud corresponding to the scene;
identify a supporting plane in the scene based on the three-dimensional point cloud;
generate a plurality of region proposals for a target object to be detected based on the scene image and the supporting plane; and
perform a target detection on the plurality of region proposals to determine the target object to be detected in the scene image,
wherein the processor is further configured to execute the computer programs to:
generate a plurality of sampling boxes;
determine a moving range of the plurality of sampling boxes based on a position and an orientation of the supporting plane;
control the plurality of sampling boxes to move on the supporting plane by a preset step size in the moving range to generate a plurality of three-dimensional regions; and
project the plurality of three-dimensional regions to the scene image to form the plurality of region proposals;
wherein the plurality of sampling boxes and the supporting plane are three-dimensional virtual area.
8. The terminal device according to claim 7 , wherein the processor is further configured to execute the computer programs to:
scan the scene by a simultaneous localization and mapping (SLAM) system to generate the three-dimensional point cloud corresponding to the scene.
9. The terminal device according to claim 7 , wherein the processor is further configured to execute the computer programs to:
identify a horizontal plane or a vertical plane in the scene based on the three-dimensional point cloud, and determine the horizontal plane or the vertical plane as the supporting plane.
10. The terminal device according to claim 7 , wherein the processor is further configured to execute the computer programs to:
acquire a target object of interest to the user;
acquire parameters of the target object of interest; and
generate parameters of the plurality of sampling boxes based on the parameters of the target object of interest.
11. The terminal device according to claim 7 , wherein the processor is further configured to execute the computer programs to:
identify the supporting plane in the scene based on the three-dimensional point cloud by a Random Sample Consensus (RANSAC) algorithm, the three-dimensional point cloud being an input of the RANSAC algorithm.
12. The terminal device according to claim 7 , wherein the supporting plane is a plane used to place actual objects or virtual objects in the scene.
13. A non-transitory computer readable storage medium, storing computer programs therein, wherein the computer programs, when executed by a processor, cause the processor to:
acquire a scene image of a scene;
acquire a three-dimensional point cloud corresponding to the scene;
identify a supporting plane in the scene based on the three-dimensional point cloud;
generate a plurality of region proposals for a target object to be detected based on the scene image and the supporting plane; and
perform a target detection on the plurality of region proposals to determine the target object to be detected in the scene image,
wherein the computer programs, when executed by a processor, further cause the processor to:
generate a plurality of sampling boxes;
determine a moving range of the plurality of sampling boxes based on a position and an orientation of the supporting plane;
control the plurality of sampling boxes to move on the supporting plane by a preset step size in the moving range to generate a plurality of three-dimensional regions; and
project the plurality of three-dimensional regions to the scene image to form the plurality of region proposals;
wherein the plurality of sampling boxes and the supporting plane are three-dimensional virtual area.
14. The non-transitory computer readable storage medium according to claim 13 , wherein the computer programs, when executed by a processor, further cause the processor to:
scan the scene by a simultaneous localization and mapping (SLAM) system to generate the three-dimensional point cloud corresponding to the scene.
15. The non-transitory computer readable storage medium according to claim 13 , wherein the computer programs, when executed by a processor, further cause the processor to:
identify a horizontal plane or a vertical plane in the scene based on the three-dimensional point cloud, and determine the horizontal plane or the vertical plane as the supporting plane.
16. The non-transitory computer readable storage medium according to claim 13 , wherein the computer programs, when executed by a processor, further cause the processor to:
acquire a target object of interest to the user;
acquire parameters of the target object of interest; and
generate parameters of the plurality of sampling boxes based on the parameters of the target object of interest.
17. The target detection method according to claim 1 , wherein the plurality of region proposals are located on the supporting plane.