Granted Patent B2

US 12,675,898 · App. 18/082,738 · Granted Jul 7, 2026

Inventors: James Gregson (Ottawa, CA); Shao Hua Chen (Ottawa, CA)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

G06T7/73G06V10/82G06V20/58G06V20/588G06T2207/20084G06T2207/30256G06T2207/30261G08G1/16

Abstract

Claims (62)

1 . A method of determining a pose of a vehicle, comprising:

receiving a set of unordered points associated with road markings;

inputting the unordered points to a trained artificial neural network, wherein, from the point of view of the network, any permutation of the points that is input to the network is equivalent to any other permutation of the points that is input to the network;

using the artificial neural network to generate an output by applying non-linear regression to the unordered points;

determining, based on the output, the pose of the vehicle;

providing the determined pose to a collision warning system configured to identify, in one or more images, one or more objects and their respective one or more positions relative to the vehicle; and

using the collision warning system to adjust, based on the determined pose, the position of at least one identified object.

2 . The method of claim 1 , wherein receiving the set of unordered points comprises:

receiving the set of unordered points from a lane detection computer module configured to identify, within one or more images of the road markings, the unordered points.

3 . The method of claim 2 , wherein the lane detection computer module is configured to identify the unordered points using one or more convolutional neural networks.

4 . The method of claim 1 , further comprising, before inputting the unordered points to the artificial neural network:

determining that the set of unordered points is valid.

5 . The method of claim 4 , wherein determining that the set of unordered points is valid comprises determining that one or more of:

the unordered points are associated with at least two sets of lane markings, each set of lane markings comprising one or more aligned lane markings; and

at least a preset number of the unordered points are associated with a given set of lane markings, each set of lane markings comprising one or more aligned lane markings.

6 . The method of claim 1 , further comprising, before inputting the unordered points to the artificial neural network:

normalizing the set of unordered points based on a camera intrinsic matrix associated with a camera onboard the vehicle.

7 . The method of claim 1 , further comprising, before inputting the unordered points to the artificial neural network:

augmenting the set of unordered points by increasing a dimensionality of each unordered point by at least one.

8 . The method of claim 7 , wherein augmenting the set of unordered points comprises associating one or more of the following with each unordered point:

a direction associated with the unordered point and at least one other unordered point;

a group comprising the unordered point and at least one other unordered point; and

a line fitted to the unordered point and at least one other unordered point.

9 . The method of claim 1 , further comprising, before inputting the unordered points to the artificial neural network, one or more of:

increasing the number of unordered points by randomly sampling with replacement at least some of the unordered points;

in response to determining that the number of unordered points is above a threshold, reducing the number of unordered points by randomly sampling at least some of the unordered points;

in response to determining that the number of unordered points is above a threshold, reducing the number of unordered points by filtering out one or more unordered points that are neither within a minimum preset distance nor a maximum preset distance from the vehicle; and

for each unordered point, filtering out of the unordered point if it is determined to not relate to a road marking.

10 . The method of claim 1 , wherein generating the output by applying non-linear regression to the set of unordered points comprises:

passing each unordered point through a first multi-layer perceptron neural network to generate an updated unordered point comprising an R-dimensional feature;

passing each updated unordered point through a second multi-layer perceptron neural network to generate an S-dimensional feature, wherein S>R;

generating an S-dimensional global feature by applying an order-independent reduction operation to each S-dimensional feature; and

generating the output by passing the S-dimensional global feature through a third multi-layer perceptron neural network.

11 . The method of claim 1 , wherein determining the pose of the vehicle comprises:

adjusting the output based on an orientation of a camera onboard the vehicle; and

determining the pose based on the adjusted output.

12 . The method of claim 1 , wherein the pose of the vehicle comprises a pitch of the vehicle.

13 . A vehicle comprising:

a vehicle pose module comprising one or more processors communicative with one or more computer-readable media and configured to:

receive a set of unordered points associated with road markings;

input the unordered points to a trained artificial neural network, wherein, from the point of view of the network, any permutation of the points that is input to the network is equivalent to any other permutation of the points that is input to the network;

use the artificial neural network to generate an output by applying non-linear regression to the unordered points; and

determine, based on the output, a pose of the vehicle.

14 . The vehicle of claim 13 , wherein the one or more processors are further configured to:

receive the set of unordered points from a lane detection computer module configured to identify, within one or more images of the road markings, the unordered points.

15 . The vehicle of claim 13 , wherein the one or more processors are further configured to, before inputting the unordered points to the artificial neural network:

normalize the set of unordered points based on a camera intrinsic matrix associated with a camera onboard the vehicle.

16 . The vehicle of claim 13 , wherein the one or more processors are further configured to, before inputting the unordered points to the artificial neural network:

augment the set of unordered points by increasing a dimensionality of each unordered point by at least one.

17 . The vehicle of claim 13 , wherein the one or more processors are further configured to, before inputting the unordered points to the artificial neural network, one or more of:

increase the number of unordered points by randomly sampling with replacement at least some of the unordered points;

in response to determining that the number of unordered points is above a threshold, reduce the number of unordered points by randomly sampling at least some of the unordered points;

in response to determining that the number of unordered points is above a threshold, reduce the number of unordered points by filtering out one or more unordered points that are neither within a minimum preset distance nor a maximum preset distance from the vehicle; and

for each unordered point, filter out of the unordered point if it is determined to not relate to a road marking.

18 . The vehicle of claim 13 , wherein the one or more processors are further configured to:

provide the determined pose to a collision warning system configured to identify, in one or more images, one or more objects and their respective one or more positions relative to the vehicle; and

use the collision warning system to adjust, based on the determined pose, the position of at least one identified object.

19 . A non-transitory computer-readable medium comprising computer program code configured, when executed by one or more processors, to cause the one or more processors to perform a method comprising:

receiving a set of unordered points associated with road markings;

inputting the unordered points to a trained artificial neural network, wherein, from the point of view of the network, any permutation of the points that is input to the network is equivalent to any other permutation of the points that is input to the network;

using the artificial neural network to generate an output by applying non-linear regression to the unordered points; and

determining, based on the output, a pose of the vehicle.

Assignments (1)

Continuity (1)

References Cited (12)