IP Library Granted Patent US 11,379,963
Granted Patent B2
US 11,379,963 · App. 16/609,447 · Granted Jul 5, 2022

Information processing method and device, cloud-based processing device, and computer program product

Inventors: Ye Li (Shenzhen, CN); Shiguo Lian (Shenzhen, CN)
Assignee: CloudMinds Robotics Co., Ltd.
G06T7/0002G06T5/002G06T5/20G06T7/11G06T7/50G06T2207/10028G06T2207/20024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,379,963
App. No.
16/609,447
Granted
Jul 5, 2022
Kind
B2
Abstract

An information processing method, device, cloud-based processing device, and computer program product are related to the field of data processing technologies and can cause an increased efficiency in detecting whether a road area contains a depression region. The information processing method includes: acquiring a depth image; processing the depth image to obtain a means-by-row graph, based on which a road area in the depth image is determined; determining a suspected depression region in the road area; and judging over the suspected depression region based on a depression threshold to determine whether the depth image contains a depression region.

Claims (132)

1. An information processing method, comprising:

acquiring a depth image;

processing the depth image to obtain a means-by-row graph, and determining a road area in the depth image based on the means-by-row graph, wherein the means-by-row graph is obtained by performing average filtering on each row in the depth image;

determining a suspected depression region in the road area; and

judging over the suspected depression region based on a depression threshold to determine whether the depth image contains a depression region;

wherein the determining a suspected depression region in the road area comprises:

calculating row means in the road area;

establishing a band-stop filter; and

filtering the row means by means of the band-stop filter to thereby obtain the suspected depression region.

2. The method of claim 1 , wherein the depth image is an image under a camera coordinate system, wherein the processing the depth image to obtain a means-by-row graph, and determining a road area in the depth image based on the means-by-row graph comprises:

performing a coordinate transformation over the depth image to thereby convert the camera coordinate system into a world coordinate system;

processing the depth image in the world coordinate system, and calculating row means in the depth image to thereby obtain a means-by-row graph;

processing the means-by-row graph to determine a suspected road area; and

judging over the suspected road area based on a preset position threshold of a main plane to determine the road area in the depth image.

3. The method of claim 1 , wherein the judging over the suspected depression region based on a depression threshold to determine whether the depth image contains a depression region comprises:

preprocessing the suspected depression region;

extracting a contour of the suspected depression region and regarding the contour as a candidate depression region;

calculating an area of the candidate depression region; and

determining, if an area of the candidate depression region is larger than an area threshold, that the depth image contains a depression region.

4. The method of claim 3 , further comprising:

deleting, if the area of the candidate depression region is less than or equal to the area threshold, the candidate depression region.

5. The method of claim 1 , further comprising:

outputting a prompt message upon determining that the depth image contains a depression region.

6. The method of claim 2 , wherein the processing the depth image in the world coordinate system, and calculating row means in the depth image to thereby obtain a means-by-row graph comprises:

performing a preprocessing over the depth image in the world coordinate system, wherein the preprocessing comprises at least one of smoothing, filtering, and denoising.

7. The method of claim 2 , wherein in the judging over the suspected road area based on a preset position threshold of a main plane to determine the road area in the depth image, a selection strategy comprises:

selecting an area with a largest area and with a distance from a lowest position of the suspected road area to a lowest position of a depth map Z w not exceeding ε rows , wherein:

ε rows <5%· H Zw ;

where ε rows represents a threshold value for the position of the main plane, and H Zw represents a height of the depth map Z w .

8. The method of claim 1 , wherein the formula of the band-stop filter is as follows:

Z

wGnd

(

i

,

j

)

=

{

0

,

Z

wGnd

(

i

,

j

)

-

I

rowsMeanGnd

(

i

)

δ

Z

w

Gnd

(

i

,

j

)

,

Z

wGnd

(

i

,

j

)

-

I

rowsMeanGnd

(

i

)

>

δ

where Z wGnd (i, j) is a depth value of the depth image corresponding to the road area at coordinates (i, j), and I rowsMeanGnd (i) is a mean value of the depth image corresponding to the road area at a row i; and δ is a preset level of tolerance for depressions on the road surface.

9. The method of claim 8 , wherein δ is between [5, 30].

10. The method of claim 3 , wherein the calculating an area of the candidate depression region comprises:

calculating an area of a rectangular box comprising XwR, XwL, ZwT, and ZwB to substitute the area of the candidate depression region, wherein:

XwR, and XwL correspond respectively to a right-most value and a left-most value of the candidate depression region along an Xw axis; and

ZwT and ZwB correspond respectively to an upper-most value and a lower-most value of the candidate depression region along an Zw axis.

11. The method of claim 3 , wherein the area threshold is between [100 px, 400 px].

12. An information processing device, comprising:

a processor; and

a memory storing instructions, which, when executed by the processor, cause the processor to:

acquire a depth image;

process the depth image to obtain a means-by-row graph, and then to determine a road area in the depth image based on the means-by-row graph, wherein the means-by-row graph is obtained by performing average filtering on each row in the depth image;

determine a suspected depression region in the road area; and

judge over the suspected depression region based on the depression threshold to thereby determine whether the depth image contains a depression region;

wherein the processor is configured to:

calculate row means in the road area;

establish a band-stop filter; and

filter the row means by means of the band-stop filter to thereby obtain the suspected depression region.

13. The device of claim 12 , wherein the depth image is an image under a camera coordinate system, wherein the processor is configured to:

perform a coordinate transformation over the depth image to thereby convert the camera coordinate system into a world coordinate system;

process the depth image in the world coordinate system, and calculate row means therein to thereby obtain a means-by-row graph;

process the means-by-row graph to determine a suspected road area; and

judge over the suspected road area based on a preset position threshold of a main plane to determine the road area in the depth image.

14. The device of claim 12 , wherein the processor is configured to:

preprocess the suspected depression region;

extract a contour of the suspected depression region and regard the contour as a candidate depression region;

calculate an area of the candidate depression region; and

determine, if an area of the candidate depression region is larger than an area threshold, that the depth image comprises a depression region.

15. The device of claim 14 , wherein the processor is further configured to delete, if the area of the candidate depression region is less than or equal to the area threshold, the candidate depression region.

16. The device of claim 12 , wherein the processor is further configured to output a prompt message upon determining that the depth image comprises a depression region.

17. The device of claim 12 , wherein the device is based on a cloud.

18. A non-transitory computer-readable storage medium, having computer program instructions stored thereon, wherein the program instructions, when being executed by a processor, are configured to perform the operations of:

acquiring a depth image;

processing the depth image to obtain a means-by-row graph, and determining a road area in the depth image based on the means-by-row graph, wherein the means-by-row graph is obtained by performing average filtering on each row in the depth image;

determining a suspected depression region in the road area; and

judging over the suspected depression region based on a depression threshold to determine whether the depth image contains a depression region;

wherein the determining a suspected depression region in the road area comprises:

calculating row means in the road area;

establishing a band-stop filter; and

filtering the row means by means of the band-stop filter to thereby obtain the suspected depression region.

Assignments (7)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2026
From: DATAA ROBOTICS (ANJI) CO., LTD.
To: CHONGQING XINGJIE SHUXING TECHNOLOGY PARTNERSHIP ENTERPRISE (LIMITED PARTNERSHIP)
Reel/Frame 074153/0627 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2025
From: CLOUDMINDS ROBOTICS CO., LTD.
To: DATAA ROBOTICS (ANJI) CO., LTD.
Reel/Frame 072089/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE'S NAME AND ADDRESS PREVIOUSLY RECORDED AT REEL: 055973 FRAME: 0581. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded May 24, 2022
From: DATHA ROBOT CO., LTD.
To: CLOUDMINDS ROBOTICS CO., LTD.
Reel/Frame 060173/0560 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE'S NAME AND ADDRESS PREVIOUSLY RECORDED AT REEL: 055973 FRAME: 0581. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded May 19, 2022
From: DATHA ROBOT CO., LTD.
To: CLOUDMINDS ROBOTICS CO., LTD
Reel/Frame 060384/0843 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 9, 2021
From: DATHA ROBOT CO., LTD.
To: CLOUDMINDS (SHANGHAI) ROBOTICS CO., LTD.
Reel/Frame 055973/0581 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2021
From: CLOUDMINDS (SHENZHEN) ROBOTICS SYSTEMS CO., LTD.
To: DATHA ROBOT CO., LTD.
Reel/Frame 055613/0424 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2019
From: LI, YE; LIAN, SHIGUO
To: CLOUDMINDS (SHENZHEN) ROBOTICS SYSTEMS CO., LTD.
Reel/Frame 051430/0264 →
Continuity (1)
Related Publication 20200090323A1 · Mar 19, 2020