Granted Patent B2

US 12,456,210 · App. 17/893,007 · Granted Oct 28, 2025

Inventors: Shenghao Zhang (Shenzhen, CN); Yonggen Ling (Shenzhen, CN); Wanchao Chi (Shenzhen, CN); Yu Zheng (Shenzhen, CN); Xinyang Jiang (Shenzhen, CN)

Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

G06T7/55G06V10/44G06V20/10G06T2207/10024G06T2207/10028

Abstract

Claims (82)

1. A scene contour recognition method, the method comprising:

obtaining a plurality of scene images of an environment;

determining three-dimensional (3D) information of a target plane in the plurality of scene images based on depth information for each of the plurality of scene images, the target plane corresponding to a target object in the plurality of scene images, the 3D information of the target plane including a plane equation of the target plane and 3D coordinates corresponding to points on the target plane;

generating a 3D contour corresponding to the target object by fusing the target plane in each of the plurality of scene images based on the 3D information of the target plane in each of the plurality of scene images; and

generating a contour diagram of the target object by projecting the 3D contour onto a two-dimensional (2D) plane, wherein

the determining the 3D information of the target plane includes:

detecting the target plane in the plurality of scene images with plane fitting based on the depth information of the plurality of scene images; and

determining the 3D coordinates corresponding to the points on the target plane, and the plane equation of the target plane in a world coordinate system for the target plane in the plurality of scene images.

2. The method according to claim 1 , wherein the generating the 3D contour comprises:

determining 3D point information of edge feature points on the target plane based on the 3D information of the target plane for each of the plurality of scene images; and

generating the 3D contour corresponding to the target object by fusing the target plane in each of the plurality of scene images based on the 3D point information of the edge feature points on the target plane in each of the plurality of scene images.

3. The method according to claim 2 , wherein the generating the 3D contour corresponding to the target object comprises:

determining a target coordinate system based on a position of an image capturing apparatus at a current time, the image capturing apparatus being configured to capture the plurality of scene images;

determining target coordinates of the edge feature points in the target coordinate system according to the 3D point information of the edge feature points on the target plane, for the target plane in each of the plurality of scene images based on the target coordinate system; and

generating the 3D contour corresponding to the target object by fusing the target plane in each of the plurality of scene images based on the target coordinates of the edge feature points on the target plane in each of the plurality of scene images.

4. The method according to claim 1 , further comprising:

determining an equation corresponding to the target plane based on the 3D information of the target plane;

determining a center point of the target plane based on the equation corresponding to the target plane; and

obtaining a 3D contour of an adjusted target plane by adjusting the target plane based on a distance between an optical center of an image capturing apparatus configured to capture the plurality of scene images and the center point.

5. The method according to claim 4 , wherein the obtaining the 3D contour of the adjusted target plane comprises:

determining an optimization weight of the 3D contour based on the distance between the optical center of the image capturing apparatus and the center point, the optimization weight being inversely proportional to the distance;

determining an optimization parameter corresponding to the target plane based on the optimization weight;

adjusting the target plane based on the optimization parameter; and

obtaining the 3D contour of the adjusted target plane.

6. The method according to claim 1 , wherein the generating the contour diagram of the target object comprises:

detecting discrete points distributed on each plane in the 3D contour, and 3D coordinates corresponding to the discrete points;

generating 2D coordinates corresponding to the discrete points by performing dimension reduction on the 3D coordinates corresponding to the discrete points; and

generating the contour diagram of the target object by combining the 2D coordinates corresponding to the discrete points.

7. The method according to claim 1 , wherein after the generating the contour diagram of the target object, the method further comprises:

detecting a contour edge corresponding to the contour diagram based on discrete points in the contour diagram;

determining a contour range corresponding to the contour diagram based on the contour edge; and

obtaining an optimized contour diagram by eliminating discrete points outside the contour range based on the contour range.

8. The method according to claim 1 , wherein the obtaining the plurality of scene images comprises:

obtaining photographing parameters including a photographing period; and

capturing the plurality of scene images corresponding to each traveling position based on the photographing period in a traveling process, the plurality of scene images including a depth map and a color map.

9. The method according to claim 1 , wherein after the generating the contour diagram of the target object, the method further comprises:

determining an obtaining manner of a robotic device to obtain the target object based on a position of the target object corresponding to the contour diagram in a current traveling scenario; and

obtaining the target object based on the obtaining manner.

10. The method according to claim 9 , further comprising:

detecting a contour diagram corresponding to a target placement position of the target object;

determining a placement manner of the robotic device to place the target object based on the contour diagram corresponding to the target placement position; and

placing the target object at the target placement position based on the placement manner.

11. The method according to claim 1 , wherein after the generating the contour diagram of the target object, the method further comprises:

determining a traveling manner in a current traveling scenario based on the contour diagram, the traveling manner including at least one of a traveling direction, a traveling height, and a traveling distance; and

performing a traveling task based on the traveling manner.

12. A scene contour recognition apparatus, comprising:

processing circuitry configured to:

obtain a plurality of scene images of an environment;

determine three-dimensional (3D) information of a target plane in the plurality of scene images based on depth information for each of the plurality of scene images, the target plane corresponding to a target object in the plurality of scene images, the 3D information of the target plane including a plane equation of the target plane and 3D coordinates corresponding to points on the target plane;

generate a 3D contour corresponding to the target object by fusing the target plane in each of the plurality of scene images based on the 3D information of the target plane in each of the plurality of scene images; and

generate a contour diagram of the target object by projecting the 3D contour onto a two-dimensional (2D) plane, wherein

the determination of the 3D information of the target plane includes:

detecting the target plane in the plurality of scene images with plane fitting based on the depth information of the plurality of scene images; and

determining the 3D coordinates corresponding to the points on the target plane, and the plane equation of the target plane in a world coordinate system for the target plane in the plurality of scene images.

13. The scene contour recognition apparatus according to claim 12 , wherein the processing circuitry is configured to:

determine 3D point information of edge feature points on the target plane based on the 3D information of the target plane for each of the plurality of scene images; and

generate the 3D contour corresponding to the target object by fusing the target plane in each of the plurality of scene images based on the 3D point information of the edge feature points on the target plane in each of the plurality of scene images.

14. The scene contour recognition apparatus according to claim 13 , wherein the processing circuitry is configured to:

determine a target coordinate system based on a position of an image capturing apparatus at a current time, the image capturing apparatus being configured to capture the plurality of scene images;

determine target coordinates of the edge feature points in the target coordinate system according to the 3D point information of the edge feature points on the target plane, for the target plane in each of the plurality of scene images based on the target coordinate system; and

generate the 3D contour corresponding to the target object by fusing the target plane in each of the plurality of scene images based on the target coordinates of the edge feature points on the target plane in each of the plurality of scene images.

15. The scene contour recognition apparatus according to claim 12 , wherein the processing circuitry is configured to:

determine an equation corresponding to the target plane based on the 3D information of the target plane;

determine a center point of the target plane based on the equation corresponding to the target plane; and

obtain a 3D contour of an adjusted target plane by adjusting the target plane based on a distance between an optical center of an image capturing apparatus configured to capture the plurality of scene images and the center point.

16. The scene contour recognition apparatus according to claim 15 , wherein the processing circuitry is configured to:

determine an optimization weight of the 3D contour based on the distance between the optical center of the image capturing apparatus and the center point, the optimization weight being inversely proportional to the distance;

determine an optimization parameter corresponding to the target plane based on the optimization weight;

adjust the target plane based on the optimization parameter; and

obtain the 3D contour of the adjusted target plane.

17. The scene contour recognition apparatus according to claim 12 , wherein the processing circuitry is configured to:

detect discrete points distributed on each plane in the 3D contour, and 3D coordinates corresponding to the discrete points;

generate 2D coordinates corresponding to the discrete points by performing dimension reduction on the 3D coordinates corresponding to the discrete points; and

generate the contour diagram of the target object by combining the 2D coordinates corresponding to the discrete points.

18. A non-transitory computer-readable storage medium, storing instructions which when executed by a processor cause the processor to perform:

obtaining a plurality of scene images of an environment;

determining three-dimensional (3D) information of a target plane in the plurality of scene images based on depth information for each of the plurality of scene images, the target plane corresponding to a target object in the plurality of scene images, the 3D information of the target plane including a plane equation of the target plane and 3D coordinates corresponding to points on the target plane;

generating a 3D contour corresponding to the target object by fusing the target plane in each of the plurality of scene images based on the 3D information of the target plane in each of the plurality of scene images; and

generating a contour diagram of the target object by projecting the 3D contour onto a two-dimensional (2D) plane, wherein

the determining the 3D information of the target plane includes:

detecting the target plane in the plurality of scene images with plane fitting based on the depth information of the plurality of scene images; and

determining the 3D coordinates corresponding to the points on the target plane, and the plane equation of the target plane in a world coordinate system for the target plane in the plurality of scene images.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (20)