IP Library › Granted Patent US 12,266,153
Granted Patent B2
US 12,266,153 · App. 17/877,634 · Granted Apr 1, 2025

Method and device for detecting object and vehicle

Inventors: Yufan Yang (Beijing, CN); Xiong Zhao (Beijing, CN); Jun Zou (Beijing, CN)
Assignee: XIAOMI EV TECHNOLOGY CO., LTD.
G06V10/764G06T7/11G06V10/25G06V20/56G06V2201/07
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,266,153
App. No.
17/877,634
Granted
Apr 1, 2025
Kind
B2
Abstract

A method for detecting an object includes obtaining an original image of a road through a sensor by performing an image capturing operation; extracting a region of interest (ROI) including a first target pixel point from the original image, in which the first target pixel point is a pixel point in the original image corresponding to a position on the road that is farthest from the sensor; and generating an object detection result by performing object detection on the ROI.

Claims (54)

1. A method for detecting an object, comprising:

obtaining an original image of a road through a sensor by performing an image capturing operation;

extracting a region of interest (ROI) including a target pixel point from the original image, wherein the target pixel point comprises a first target pixel point and a second target pixel point, the first target pixel point is a pixel point in the original image corresponding to a position on the road that is farthest from the sensor, and the second target pixel point refers to a pixel point of the original image corresponding to a position point on the road that exceeds a preset distance away from the sensor; and

generating an object detection result by performing object detection on the ROI.

2. The method of claim 1 , further comprising:

generating a mask map of the original image by performing road segmentation on the original image; and

determining the first target pixel point based on the mask map.

3. The method of claim 2 , wherein generating the mask map comprises:

inputting the original image to a road segmentation model, and obtaining the mask map outputted by the road segmentation model.

4. The method of claim 2 , wherein determining the first target pixel point comprises:

traversing the mask map from a first row of the mask map until a pixel point with black color is found, and in response to finding the pixel point with the black color, determining the pixel point as the first target pixel point.

5. The method of claim 1 , further comprising:

obtaining depth information of pixel points of the original image corresponding to position points of the road; and

determining a pixel point whose depth information is equal to or greater than a preset threshold as the second target pixel point.

6. The method of claim 1 , wherein extracting the ROI comprises:

determining the target pixel point in the original image, and extracting a position of the target pixel point from the original image; and

extracting the ROI from the original image based on the position of the target pixel point.

7. The method of claim 1 , wherein generating the object detection result comprises:

generating a target image by cropping the original image based on the ROI; and

generating the object detection result by performing the object detection on the target image.

8. A vehicle, comprising:

a processor; and

a memory, for storing machine-readable instructions that, when executed by the processor cause the processor to:

obtain an original image of a road through a sensor by performing an image capturing operation;

extract a region of interest (ROI) including a target pixel point from the original image, wherein the target pixel point comprises a first target pixel point and a second target pixel point, the first target pixel point a pixel point in the original image corresponding to a position on the road that is farthest from the sensor, and the second target pixel point refers to a pixel point of the original image corresponding to a position point on the road that exceeds a preset distance away from the sensor; and

generate an object detection result by performing object detection on the ROI.

9. The vehicle of claim 8 , the memory further comprising machine-readable instructions that, when executed by the processor, cause the processor to:

generate a mask map of the original image by performing road segmentation on the original image; and

determine the first target pixel point based on the mask map.

10. The vehicle of claim 9 , the memory further comprising machine-readable instructions that, when executed by the processor, cause the processor to:

input the original image to a road segmentation model, and obtain the mask map outputted by the road segmentation model.

11. The vehicle of claim 9 , the memory further comprising machine-readable instructions that, when executed by the processor, cause the processor to:

traverse the mask map from a first row of the mask map until a pixel point with black color is found, and in response to finding the pixel point with the black color, determining the pixel point as the first target pixel point.

12. The vehicle of claim 8 , the memory further comprising machine-readable instructions that, when executed by the processor, cause the processor to:

obtain depth information of pixel points of the original image corresponding to position points of the road; and

determine a pixel point whose depth information is equal to or greater than a preset threshold as the second target pixel point.

13. The vehicle of claim 8 , the memory further comprising machine-readable instructions that, when executed by the processor, cause the processor to:

determine the target pixel point in the original image, and extract a position of the target pixel point from the original image; and

extract the ROI from the original image based on the position of the target pixel point.

14. The vehicle of claim 8 , the memory further comprising machine-readable instructions that, when executed by the processor, cause the processor to:

generate a target image by cropping the original image based on the ROI; and

generate the object detection result by performing the object detection on the target image.

15. A non-transitory computer-readable storage medium, having machine-readable instructions stored thereon that, when executed by a processor of an electronic device, cause the processor to:

obtain an original image of a road through a sensor by performing an image capturing operation;

extract a region of interest (ROI) including a target pixel point from the original image, wherein the target pixel point comprises a first target pixel point and a second target pixel point, the first target pixel point a pixel point in the original image corresponding to a position on the road that is farthest from the sensor, and the second target pixel point refers to a pixel point of the original image corresponding to a position point on the road that exceeds a preset distance away from the sensor; and

generate an object detection result by performing object detection on the ROI.

16. The non-transitory computer-readable storage medium of claim 15 , further comprising machine-readable instructions that, when executed by the processor, cause the processor to:

generating a mask map of the original image by performing road segmentation on the original image; and

determining the first target pixel point based on the mask map.

17. The non-transitory computer-readable storage medium of claim 16 , further comprising machine-readable instructions that, when executed by the processor, cause the processor to, when generating the mask map:

input the original image to a road segmentation model, and obtaining the mask map outputted by the road segmentation model.

18. The non-transitory computer-readable storage medium of claim 16 , further comprising machine-readable instructions that, when executed by the processor, cause the processor to, when determining the first target pixel point:

traverse the mask map from a first row of the mask map until a pixel point with black color is found; and, in response to finding the pixel point with the black color,

determine the pixel point as the first target pixel point.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2022
From: YANG, YUFAN; ZHAO, XIONG; ZOU, JUN
To: XIAOMI EV TECHNOLOGY CO., LTD.
Reel/Frame 060676/0888 →
Priority Claims (1)
CN 202210260712.1 · Mar 16, 2022 · national
Continuity (1)
Related Publication 20230298317A1 · Sep 21, 2023
References Cited (26)
US 10937181B2 · Amano · 2021 [cited by examiner]
US 10997740B2 · Biswas · 2021 [cited by examiner]
US 20020036692A1 · Okada · 2002 [cited by examiner]
US 20060078165A1 · Watanabe · 2006 [cited by examiner]
US 20090041337A1 · Nakano · 2009 [cited by examiner]
US 20160162741A1 · Shin · 2016 [cited by examiner]
US 20160232413A1 · Ito · 2016 [cited by applicant]
US 20160379064A1 · van Beek · 2016 [cited by examiner]
US 20170091565A1 · Yokoi · 2017 [cited by examiner]
US 20190042860A1 · Lee · 2019 [cited by examiner]
US 20190266418A1 · Xu · 2019 [cited by examiner]
US 20200082178A1 · Ahn · 2020 [cited by examiner]
US 20200117920A1 · Lee · 2020 [cited by examiner]
US 20210019897A1 · Biswas · 2021 [cited by examiner]
US 20210326611A1 · Kim · 2021 [cited by examiner]
US 20210400190A1 · Tanaka · 2021 [cited by examiner]
US 20220277470A1 · Xiao · 2022 [cited by examiner]
US 20220319147A1 · Yasui · 2022 [cited by examiner]
US 20220374659A1 · Hu · 2022 [cited by examiner]
US 20230014874A1 · Shen · 2023 [cited by examiner]
US 20230258471A1 · Iqbal · 2023 [cited by examiner]
US 20230298317A1 · Yang · 2023 [cited by examiner]
US 20230394843A1 · Lee · 2023 [cited by examiner]
European Patent Application No. 22187467.0, Search and Opinion dated May 17, 2023, 9 pages. [cited by applicant]
Suhr J.K. “Noise-resilient Road Surface and Free Space Estimation Using Dense Stereo” 2013 IEEE Intelligent Vehicles Symposium (IV) Jun. 201, pp. 461-466. [cited by applicant]
Kuan, D. et al. “A Realtime Road Following and Road Junction Detection Vision System for Autonomous Vehicles” Proceedings AAAI, Natl Conference on Artificial Intelligence, vol. 2, Aug. 1986, pp. 1127-1132. [cited by applicant]