Granted Patent B2

US 12,617,414 · App. 18/448,034 · Granted May 5, 2026

Inventors: Kiran Bangalore Ravi (Paris, FR); Varun Ravi Kumar (San Diego, CA); Senthil Kumar Yogamani (Headford, IE)

Assignee: QUALCOMM Incorporated

B60W50/06G06T7/33G06T7/35B60W2556/35G06T2207/10028G06T2207/20081G06T2207/30252

Abstract

Claims (76)

1 . A method comprising:

receiving first sensor data from a first sensor arrangement and second sensor data from a second sensor arrangement, wherein the first sensor arrangement is associated with a first vehicle type and the second sensor arrangement is associated with a second vehicle type different from the first vehicle type, and wherein at least one of the first sensor data and the second sensor data include both image data and position data;

determining, based on a first set of sensor parameters of the first sensor arrangement and a second set of sensor parameters of the second sensor arrangement, first differences between the first sensor arrangement and the second sensor arrangement;

determining, based on the first sensor data, a first set of top-down features and, based on the second sensor data, a second set of top-down features;

determining, based on the first differences, the first set of top-down features, and the second set of top-down features, a transformation to align the second set of top-down features with the first set of top-down features within a top view representation; and

determining, based on the transformation, vehicle control instructions for a vehicle of the second vehicle type.

2 . The method of claim 1 , further comprising:

determining, based on the first set of top-down features and the second set of top-down features, second differences between the first set of top-down features and the second set of top-down features.

3 . The method of claim 2 , wherein the second differences are determined as one or more stochastic distances between the first sensor arrangement and the second sensor arrangement based on different sensor values, different feature values, and different feature locations.

4 . The method of claim 2 , further comprising:

determining the transformation based on the second differences, a first set of latent features for the first sensor data, and a second set of latent features for the second sensor data.

5 . The method of claim 4 , wherein determining the first set of top-down features and determining the second set of top-down features comprises:

determining the first set of latent features based on the first sensor data;

determining the first set of top-down features based on the first set of latent features;

determining the second set of latent features based on the second sensor data; and

determining the second set of top-down features based on the second set of latent features.

6 . The method of claim 4 , wherein the transformation is determined by training a variational autoencoder to minimize statistical differences between the first set of top-down features and the second set of top-down features, wherein the statistical differences are determined based on the second differences, the first differences, the first set of latent features, and the second set of latent features.

7 . The method of claim 2 , wherein the transformation is determined in response to determining that (i) the second differences exceed a first predetermined threshold, (ii) the first differences exceed a second predetermined threshold, or (iii) a combination thereof.

8 . The method of claim 1 , wherein the first differences are determined as a weighted difference between the first sensor arrangement and the second sensor arrangement based on a different number of sensors, different field of view coverage for sensors, different sensor ranges, or a combination thereof.

9 . A system comprising:

a processing system including one or more processors and one or more memories storing instructions which, when executed by the processing system, cause the processing system to perform operations comprising:

receiving first sensor data from a first sensor arrangement and second sensor data from a second sensor arrangement, wherein the first sensor arrangement is associated with a first vehicle type and the second sensor arrangement is associated with a second vehicle type different from the first vehicle type, and wherein at least one of the first sensor data and the second sensor data include both image data and position data;

determining, based on a first set of sensor parameters of the first sensor arrangement and a second set of sensor parameters of the second sensor arrangement, first differences between the first sensor arrangement and the second sensor arrangement;

determining, based on the first sensor data, a first set of top-down features and, based on the second sensor data, a second set of top-down features;

determining, based on the first differences, the first set of top-down features, and the second set of top-down features, a transformation to align the second set of top-down features with the first set of top-down features within a top view representation; and

determining, based on the transformation, vehicle control instructions for a vehicle of the second vehicle type.

10 . The system of claim 9 , wherein the operations further comprise:

determining, based on the first set of top-down features and the second set of top-down features, second differences between the first set of top-down features and the second set of top-down features.

11 . The system of claim 10 , wherein the second differences are determined as one or more stochastic distances between the first sensor arrangement and the second sensor arrangement based on different sensor values, different feature values, and different feature locations.

12 . The system of claim 10 , wherein the operations further comprise:

determining the transformation based on the second differences, a first set of latent features for the first sensor data, and a second set of latent features for the second sensor data.

13 . The system of claim 12 , wherein determining the first set of top-down features and determining the second set of top-down features comprises:

determining the first set of latent features based on the first sensor data;

determining the first set of top-down features based on the first set of latent features;

determining the second set of latent features based on the second sensor data; and

determining the second set of top-down features based on the second set of latent features.

14 . The system of claim 12 , wherein the transformation is determined by training a variational autoencoder to minimize statistical differences between the first set of top-down features and the second set of top-down features, wherein the statistical differences are determined based on the second differences, the first differences, the first set of latent features, and the second set of latent features.

15 . The system of claim 10 , wherein the transformation is determined in response to determining that (i) the second differences exceed a first predetermined threshold, (ii) the first differences exceed a second predetermined threshold, or (iii) a combination thereof.

16 . The system of claim 9 , wherein the first differences are determined as a weighted difference between the first sensor arrangement and the second sensor arrangement based on a different number of sensors, different field of view coverage for sensors, different sensor ranges, or a combination thereof.

17 . A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause the processor to perform operations comprising:

receiving first sensor data from a first sensor arrangement and second sensor data from a second sensor arrangement, wherein the first sensor arrangement is associated with a first vehicle type and the second sensor arrangement is associated with a second vehicle type different from the first vehicle type, and wherein at least one of the first sensor data and the second sensor data include both image data and position data;

determining, based on a first set of sensor parameters of the first sensor arrangement and a second set of sensor parameters of the second sensor arrangement, first differences between the first sensor arrangement and the second sensor arrangement;

determining, based on the first sensor data, a first set of top-down features and, based on the second sensor data, a second set of top-down features;

determining, based on the first differences, the first set of top-down features, and the second set of top-down features, a transformation to align the second set of top-down features with the first set of top-down features within a top view representation; and

determining, based on the transformation, vehicle control instructions for a vehicle of the second vehicle type.

18 . The non-transitory, computer-readable medium of claim 17 , wherein the operations further comprise:

determining, based on the first set of top-down features and the second set of top-down features, second differences between the first set of top-down features and the second set of top-down features.

19 . The non-transitory, computer-readable medium of claim 18 , wherein the second differences are determined as one or more stochastic distances between the first sensor arrangement and the second sensor arrangement based on different sensor values, different feature values, and different feature locations.

20 . The non-transitory, computer-readable medium of claim 18 , wherein the operations further comprise:

determining the transformation based on the second differences, a first set of latent features for the first sensor data, and a second set of latent features for the second sensor data.

21 . The non-transitory, computer-readable medium of claim 20 , wherein determining the first set of top-down features and determining the second set of top-down features comprises:

determining the first set of latent features based on the first sensor data;

determining the first set of top-down features based on the first set of latent features;

determining the second set of latent features based on the second sensor data; and

determining the second set of top-down features based on the second set of latent features.

22 . The non-transitory, computer-readable medium of claim 20 , wherein the transformation is determined by training a variational autoencoder to minimize statistical differences between the first set of top-down features and the second set of top-down features, wherein the statistical differences are determined based on the second differences, the first differences, the first set of latent features, and the second set of latent features.

23 . The non-transitory, computer-readable medium of claim 17 , wherein the first differences are determined as a weighted difference between the first sensor arrangement and the second sensor arrangement based on a different number of sensors, different field of view coverage for sensors, different sensor ranges, or a combination thereof.

24 . A vehicle comprising:

a processing system including one or more processors and one or more memories storing instructions which, when executed by the processing system, cause the processing system to perform operations comprising:

receiving first sensor data from a first sensor arrangement and second sensor data from a second sensor arrangement, wherein the first sensor arrangement is associated with a first vehicle type and the second sensor arrangement is associated with a second vehicle type different from the first vehicle type, and wherein at least one of the first sensor data and the second sensor data include both image data and position data;

determining, based on a first set of sensor parameters of the first sensor arrangement and a second set of sensor parameters of the second sensor arrangement, first differences between the first sensor arrangement and the second sensor arrangement;

determining, based on the first sensor data, a first set of top-down features and, based on the second sensor data, a second set of top-down features;

determining, based on the first differences, the first set of top-down features, and the second set of top-down features, a transformation to align the second set of top-down features with the first set of top-down features within a top view representation; and

determining, based on the transformation, vehicle control instructions for a vehicle of the second vehicle type.

25 . The vehicle of claim 24 , wherein the operations further comprise:

determining, based on the first set of top-down features and the second set of top-down features, second differences between the first set of top-down features and the second set of top-down features.

26 . The vehicle of claim 25 , wherein the second differences are determined as one or more stochastic distances between the first sensor arrangement and the second sensor arrangement based on different sensor values, different feature values, and different feature locations.

27 . The vehicle of claim 25 , wherein the operations further comprise:

determining the transformation based on the second differences, a first set of latent features for the first sensor data, and a second set of latent features for the second sensor data.

28 . The vehicle of claim 27 , wherein determining the first set of top-down features and determining the second set of top-down features comprises:

determining the first set of latent features based on the first sensor data;

determining the first set of top-down features based on the first set of latent features;

determining the second set of latent features based on the second sensor data; and

determining the second set of top-down features based on the second set of latent features.

29 . The vehicle of claim 27 , wherein the transformation is determined by training a variational autoencoder to minimize statistical differences between the first set of top- down features and the second set of top-down features, wherein the statistical differences are determined based on the second differences, the first differences, the first set of latent features, and the second set of latent features.

30 . The vehicle of claim 24 , wherein the first differences are determined as a weighted difference between the first sensor arrangement and the second sensor arrangement based on a different number of sensors, different field of view coverage for sensors, different sensor ranges, or a combination thereof.

Assignments (1)

Continuity (1)

References Cited (27)