Granted Patent B2

US 12,405,363 · App. 17/951,115 · Granted Sep 2, 2025

Inventors: Jiun-In Guo (Hsinchu, TW); Jia-Jheng Lin (Hsinchu, TW)

Assignee: Wistron Corporation

G01S13/867G01S7/2955G01S13/723

Abstract

Claims (54)

1. A dual sensing method of an object, comprising:

performing a first clustering on radar information, wherein the radar information comprises a plurality of sensing points, and the first clustering is for determining a first part of the sensing points to be an object;

performing a second clustering on a result of the first clustering, wherein the second clustering is for determining that in response to the first part of the sensing points being the object in the result of the first clustering being located in a region of a first density;

taking a result of the second clustering as a region of interest;

using a fixed region as the region of interest, wherein the fixed region is predefined, and a result of the second clustering is a region that does not have the first density; and

performing at least one of object detection and object tracking on combined information according to the region of interest, wherein the combined information is formed by combining the radar information and an image, and a detection region of the radar information is overlapped with a photographing region of the image.

2. The method of claim 1 , wherein a point number of the second clustering is less than a point number of the first clustering, and a range of the second clustering is smaller than a range of the first clustering; or, a point number of the second clustering is greater than a point number of the first clustering, and a range of the second clustering is larger than a range of the first clustering.

3. The method of claim 1 , wherein taking the result of the second clustering as the region of interest comprises:

taking a center of the sensing points in the region of the first density as a center of the region of interest.

4. The method of claim 1 , further comprising:

converting coordinates of the sensing points in the radar information into a plurality of image coordinates for generating converted radar information, wherein the converted radar information comprises the radar information on the image coordinates; and

combining the converted radar information and the image for generating the combined information.

5. The method of claim 4 , wherein converting the coordinates of the sensing points in the radar information into the image coordinates comprises:

setting a coordinate relationship between a radar world coordinate and a camera world coordinate according to a relative position of a radar and an image capturing device, wherein the radar is for obtaining the radar information, and the image capturing device is for obtaining the image; and

determining the image coordinates according to the coordinate relationship, wherein the sensing points in the radar information are sequentially converted from a radar coordinate, the radar world coordinate, the camera world coordinate, and a camera coordinate into one of the image coordinates.

6. The method of claim 4 , wherein a first sensing type in the radar information comprises at least one of a relative distance, a relative velocity, and an intensity, a second sensing type in the image comprises a plurality of color types defined by a color space, and combining the converted radar information and the image comprises:

combining the converted radar information and the image into a sensing image of a plurality of channels according to the first sensing type and the second sensing type, wherein the channels respectively correspond to the color types and at least one of the relative distance, the relative velocity, and the intensity.

7. The method of claim 6 , wherein performing at least one of the object detection and the object tracking on the combined information according to the region of interest comprises:

inputting the sensing image of the channels into a detection model for the object detection to output a prediction result of the detection model; and

performing the object tracking according to the prediction result.

8. The method of claim 1 , wherein performing the at least one of the object detection and the object tracking on the combined information according to the region of interest comprises:

determining an overlapped region of a first framed region in a prediction result of the object detection and a second framed region of the object tracking, wherein the overlapped region is an intersection of the first framed region and the second framed region; and

determining whether the object in the first framed region and the second framed region is the same according to a ratio of the overlapped region to a total region, wherein the total region is a union of the first framed region and the second framed region.

9. The method of claim 1 , wherein performing the at least one of the object detection and the object tracking on the combined information according to the region of interest comprises:

detecting whether there is the object in the fixed region, wherein the fixed region is predefined, and a second part of the fixed region is not overlapped with the result of the second clustering; and

updating the region of interest according to a detection result of the fixed region.

10. A computing apparatus for object sensing, comprising:

a memory, for storing a program code; and

a processor, coupled to the memory, and for loading and executing the program code to:

perform a first clustering on radar information, wherein the radar information comprises a plurality of sensing points, and the first clustering is for determining a first part of the sensing points to be an object;

perform a second clustering on a result of the first clustering, wherein the second clustering is for determining that in response to the first part of the sensing points being the object in the result of the first clustering being located in a region of a first density;

take a result of the second clustering as a region of interest;

using a fixed region as the region of interest, wherein the fixed region is predefined, and a result of the second clustering is a region that does not have the first density; and

perform at least one of object detection and object tracking on combined information according to the region of interest, wherein the combined information is formed by combining the radar information and an image, and a detection region of the radar information is overlapped with a photographing region of the image.

11. The computing apparatus of claim 10 , wherein a point number of the second clustering is less than a point number of the first clustering, and a range of the second clustering is smaller than a range of the first clustering; or, a point number of the second clustering is greater than a point number of the first clustering, and a range of the second clustering is larger than a range of the first clustering.

12. The computing apparatus of claim 10 , wherein the processor further:

takes a center of sensing points in the region of the first density as a center of the region of interest.

13. The computing apparatus of claim 10 , wherein the processor further:

converts coordinates of the sensing points in the radar information into a plurality of image coordinates for generating converted radar information, wherein the converted radar information comprises the radar information on the image coordinates; and

combines the converted radar information and the image for generating the combined information.

14. The computing apparatus of claim 13 , wherein the processor further:

sets a coordinate relationship between a radar world coordinate and a camera world coordinate according to a relative position of a radar and an image capturing device, wherein the radar obtains the radar information, and the image capturing device obtains the image; and

determines the image coordinates according to the coordinate relationship, wherein the sensing points in the radar information are sequentially converted from a radar coordinate, the radar world coordinate, the camera world coordinate, and a camera coordinate into one of the image coordinates.

15. The computing apparatus of claim 13 , wherein a first sensing type in the radar information comprises at least one of a relative distance, a relative velocity, and an intensity, a second sensing type in the image comprises a plurality of color types defined by a color space, and the processor further:

combines the converted radar information and the image into a sensing image of a plurality of channels according to the first sensing type and the second sensing type, wherein the channels respectively correspond to the color types and at least one of the relative distance, the relative velocity, and the intensity.

16. The computing apparatus of claim 15 , wherein the processor further:

inputs the sensing image of the channels into a detection model for the object detection to output a prediction result of the detection model; and

performs the object tracking according to the prediction result.

17. The computing apparatus of claim 10 , wherein the processor further:

determines an overlapped region of a first framed region in a prediction result of the object detection and a second framed region of the object tracking, wherein the overlapped region is an intersection of the first framed region and the second framed region; and

determines whether the object in the first framed region and the second framed region is the same according to a ratio of the overlapped region to a total region, wherein the total region is a union of the first framed region and the second framed region.

18. The computing apparatus of claim 10 , wherein the processor further:

detects whether there is the object in the fixed region, wherein the fixed region is predefined, and a second part of the fixed region is not overlapped with the result of the second clustering; and

updates the region of interest according to a detection result of the fixed region.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (28)