Granted Patent B2

US 12,541,978 · App. 17/642,003 · Granted Feb 3, 2026

Inventor: Tong Liu (Beijing, CN)

Assignee: Beijing BOE Technology Development Co., Ltd.

G06V20/52G06T7/73G06V10/273G06V10/28G06V10/457G06V10/751G06V40/10G06T2207/10016G06T2207/30232G06V2201/07

Abstract

Claims (85)

1 . A method for detecting obstacles, comprising:

acquiring a monitored image of a monitored region;

performing object detection on the monitored image;

performing pedestrian detection on the monitored image;

in response to acquiring an object identification group comprising at least one object identification box by the object detection, and acquiring at least one pedestrian identification box by the pedestrian detection, acquiring a to-be-determined object identification group by removing an object identification box in the object identification group associated with the at least one pedestrian identification box, wherein the object identification box comprises an image of an object in the monitored image, and the pedestrian identification box comprises an image of a pedestrian in the monitored image; and

in response to determining that an object identification box is present in the to-be-determined object identification group, determining that an obstacle is present in the monitored region; and

wherein performing the object detection on the monitored image comprises:

acquiring a pixel difference image between the monitored image and a target template image in a template group, wherein a pixel value of a pixel point at a first position in the pixel difference image is a target difference value, the target difference value is an absolute value of a difference value between a pixel value of a pixel point at the first position in the monitored image and a pixel value of a pixel point at the first position in the target template image, the template group comprises at least two template images of the monitored region without obstacles or pedestrians under different lighting conditions, or the template group comprises one template image of the monitored region without obstacles or pedestrians, and the target template image is a template image in the template group, which has the smallest difference from the monitored image;

acquiring a binarized image by binarizing the pixel difference image;

performing connected component analysis on the binarized image;

in response to acquiring at least one connected component by the connected component analysis, determining whether a connected component with an area greater than a first threshold is present in the at least one connected component; and

in response to determining that a connected component with an area greater than the first threshold is present, acquiring a circumscribed graph of the connected component with the area greater than the first threshold, wherein the circumscribed graph is the object identification box.

2 . The method according to claim 1 , wherein prior to acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box, the method comprises:

determining whether an object identification box overlapped with the at least one pedestrian identification box is present in the object identification group; and

in response to determining that a first object identification box overlapped with a first pedestrian identification box among the at least one pedestrian identification box is present, determining the first object identification box as the object identification box associated with the first pedestrian identification box.

3 . The method according to claim 1 , wherein prior to acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box, the method comprises:

determining whether an object identification box overlapped with the at least one pedestrian identification box is present in the object identification group; and

in response to determining that a first object identification box overlapped with a first pedestrian identification box among the at least one pedestrian identification box is present, determining whether the first object identification box is within a range of the first pedestrian identification box in a first direction, wherein the first direction is a height direction of the pedestrian in the first pedestrian identification box; and

in response to determining that the first object identification box is within the range of the first pedestrian identification box in the first direction, determining the first object identification box as the object identification box associated with the first pedestrian identification box.

4 . The method according to claim 1 , wherein in response to determining that the object identification box is present in the to-be-determined object identification group, upon determining that the obstacle is present in the monitored region, the method further comprises:

acquiring a target position of a target region in the monitored region in the monitored image;

determining whether an object identification box overlapped with the target position is present in the to-be-determined object identification group; and

in response to determining that the object identification box overlapped with the target position is present, determining that an obstacle is present in the target region.

5 . The method according to claim 4 , wherein in response to determining that the object identification box overlapped with the target position is present, upon determining that the obstacle is present in the target region, the method further comprises:

issuing an obstacle notice;

suspending the object detection and the pedestrian detection that are performed on the monitored image; and

upon receipt of an instruction that the obstacle is cleared in response to the obstacle notice, resuming the object detection and the pedestrian detection on the monitored image, and performing the step of acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box.

6 . The method according to claim 4 , wherein an edge of a specified color is present at the target position, and acquiring the target position of the target region in the monitored region in the monitored image comprises:

determining the target position of the target region in the monitored image by image recognition.

7 . The method according to claim 1 , wherein acquiring the monitored image of the monitored region comprises:

acquiring a video data stream from a streaming media server;

storing the video data stream in a data buffer; and

acquiring the monitored image of the monitored region from the data buffer.

8 . A device for detecting obstacles, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the processor, when loading and executing the at least one instruction, the at least one program, the code set, or the instruction set, is caused to perform a method for detecting obstacles, the method comprising:

acquiring a monitored image of a monitored region;

performing object detection on the monitored image;

performing pedestrian detection on the monitored image;

in response to acquiring an object identification group comprising at least one object identification box by the object detection, and acquiring at least one pedestrian identification box by the pedestrian detection, acquiring a to-be-determined object identification group by removing an object identification box in the object identification group associated with the at least one pedestrian identification box, wherein the object identification box comprises an image of an object in the monitored image, and the pedestrian identification box comprises an image of a pedestrian in the monitored image; and

in response to determining that an object identification box is present in the to-be-determined object identification group, determining that an obstacle is present in the monitored region; and

wherein performing the object detection on the monitored image comprises:

acquiring a pixel difference image between the monitored image and a target template image in a template group, wherein a pixel value of a pixel point at a first position in the pixel difference image is a target difference value, the target difference value is an absolute value of a difference value between a pixel value of a pixel point at the first position in the monitored image and a pixel value of a pixel point at the first position in the target template image, the template group comprises at least two template images of the monitored region without obstacles or pedestrians under different lighting conditions, or the template group comprises one template image of the monitored region without obstacles or pedestrians, and the target template image is a template image in the template group, which has the smallest difference from the monitored image;

acquiring a binarized image by binarizing the pixel difference image;

performing connected component analysis on the binarized image;

in response to acquiring at least one connected component by the connected component analysis, determining whether a connected component with an area greater than a first threshold is present in the at least one connected component; and

in response to determining that a connected component with an area greater than the first threshold is present, acquiring a circumscribed graph of the connected component with the area greater than the first threshold, wherein the circumscribed graph is the object identification box.

9 . The device according to claim 8 , wherein prior to acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box, the method comprises:

determining whether an object identification box overlapped with the at least one pedestrian identification box is present in the object identification group; and

in response to determining that a first object identification box overlapped with a first pedestrian identification box among the at least one pedestrian identification box is present, determining the first object identification box as the object identification box associated with the first pedestrian identification box.

10 . The device according to claim 8 , wherein prior to acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box, the method comprises:

determining whether an object identification box overlapped with the at least one pedestrian identification box is present in the object identification group; and

in response to determining that a first object identification box overlapped with a first pedestrian identification box among the at least one pedestrian identification box is present, determining whether the first object identification box is within a range of the first pedestrian identification box in a first direction, wherein the first direction is a height direction of the pedestrian in the first pedestrian identification box; and

in response to determining that the first object identification box is within the range of the first pedestrian identification box in the first direction, determining the first object identification box as the object identification box associated with the first pedestrian identification box.

11 . The device according to claim 8 , wherein in response to determining that the object identification box is present in the to-be-determined object identification group, upon determining that the obstacle is present in the monitored region, the method further comprises:

acquiring a target position of a target region in the monitored region in the monitored image;

determining whether an object identification box overlapped with the target position is present in the to-be-determined object identification group; and

in response to determining that the object identification box overlapped with the target position is present, determining that an obstacle is present in the target region.

12 . The device according to claim 11 , wherein in response to determining that the object identification box overlapped with the target position is present, upon determining that the obstacle is present in the target region, the method further comprises:

issuing an obstacle notice;

suspending the object detection and the pedestrian detection that are performed on the monitored image; and

upon receipt of an instruction that the obstacle is cleared in response to the obstacle notice, resuming the object detection and the pedestrian detection on the monitored image, and performing the step of acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box.

13 . The device according to claim 11 , wherein an edge of a specified color is present at the target position, and acquiring the target position of the target region in the monitored region in the monitored image comprises:

determining the target position of the target region in the monitored image by image recognition.

14 . The device according to claim 8 , wherein acquiring the monitored image of the monitored region comprises:

acquiring a video data stream from a streaming media server;

storing the video data stream in a data buffer; and

acquiring the monitored image of the monitored region from the data buffer.

15 . A non-volatile computer storage medium storing at least one instruction, at least one program, code set, or instruction set, and a processor, when loading and executing the at least one instruction, the at least one program, the code set, or the instruction set, is caused to perform a method for detecting obstacles, the method comprising:

acquiring a monitored image of a monitored region;

performing object detection on the monitored image;

performing pedestrian detection on the monitored image;

in response to acquiring an object identification group comprising at least one object identification box by the object detection, and acquiring at least one pedestrian identification box by the pedestrian detection, acquiring a to-be-determined object identification group by removing an object identification box in the object identification group associated with the at least one pedestrian identification box, wherein the object identification box comprises an image of an object in the monitored image, and the pedestrian identification box comprises an image of a pedestrian in the monitored image; and

in response to determining that an object identification box is present in the to-be-determined object identification group, determining that an obstacle is present in the monitored region; and

wherein performing the object detection on the monitored image comprises:

acquiring a pixel difference image between the monitored image and a target template image in a template group, wherein a pixel value of a pixel point at a first position in the pixel difference image is a target difference value, the target difference value is an absolute value of a difference value between a pixel value of a pixel point at the first position in the monitored image and a pixel value of a pixel point at the first position in the target template image, the template group comprises at least two template images of the monitored region without obstacles or pedestrians under different lighting conditions, or the template group comprises one template image of the monitored region without obstacles or pedestrians, and the target template image is a template image in the template group, which has the smallest difference from the monitored image;

acquiring a binarized image by binarizing the pixel difference image;

performing connected component analysis on the binarized image;

in response to acquiring at least one connected component by the connected component analysis, determining whether a connected component with an area greater than a first threshold is present in the at least one connected component; and

in response to determining that a connected component with an area greater than the first threshold is present, acquiring a circumscribed graph of the connected component with the area greater than the first threshold, wherein the circumscribed graph is the object identification box.

16 . The non-volatile computer storage medium according to claim 15 , wherein prior to acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box, the method comprises:

determining whether an object identification box overlapped with the at least one pedestrian identification box is present in the object identification group; and

in response to determining that a first object identification box overlapped with a first pedestrian identification box among the at least one pedestrian identification box is present, determining the first object identification box as the object identification box associated with the first pedestrian identification box.

17 . The non-volatile computer storage medium according to claim 15 , wherein prior to acquiring the to-be-determined object identification group by removing the object identification box in the object identification group associated with the at least one pedestrian identification box, the method comprises:

determining whether an object identification box overlapped with the at least one pedestrian identification box is present in the object identification group; and

in response to determining that a first object identification box overlapped with a first pedestrian identification box among the at least one pedestrian identification box is present, determining whether the first object identification box is within a range of the first pedestrian identification box in a first direction, wherein the first direction is a height direction of the pedestrian in the first pedestrian identification box; and

in response to determining that the first object identification box is within the range of the first pedestrian identification box in the first direction, determining the first object identification box as the object identification box associated with the first pedestrian identification box.

Assignments (2)

Continuity (1)

References Cited (16)