Granted Patent B2

US 12,623,674 · App. 18/619,317 · Granted May 12, 2026

Inventor: Moritz Markofsky (Bruchsal, DE)

Assignee: DR. ING. H.C. F. PORSCHE AKTIENGESELLSCHAFT

B60W50/045B60W60/001B60W2050/046B60W2556/00B60W2756/10

Abstract

Claims (68)

1 . A method that uses a continuous integration approach for improving driver assistance systems installed in vehicles, the method comprising:

using sensors on the vehicles for measuring vehicle operating data while driving the respective vehicles in real traffic;

determining vehicle actuator settings based on the operating data sensed by the sensors;

storing the measured vehicle operating data and the vehicle actuator settings in a test data set that comprises time series of input data and output data of the driver assistance system;

testing a system-under-test with the test data set, the system-under-test being formed by an overall software that is changed continuously; and

performing data-driven validations repeatedly at predetermined time intervals, each iteration of the data-driven validations including the steps of:

loading a current software version of the overall software forming the system-under-test onto a server;

compiling the current software version on the server into an executable computer program;

transferring the computer program to a hardware-in-the-loop test bench;

loading the test data set onto the hardware-in-the-loop test bench;

running the computer program on the hardware-in-the-loop test bench while feeding the input data to the test data set;

logging differences in output between the current software version and the output data of the test data set;

simultaneously matching the output of the current software version with ground-truth data, thereby inferring an improvement or deterioration of a performance of the current software version and assigning a performance score to the respective differences;

forming a performance statistic on all differences and their performance score;

evaluating changes in the overall software based on the performance statistic; outputting a report; and

executing the changed overall software during controlling of a real vehicle,

wherein the report lists any new system-under-test errors that occurred during any one of the data-driven validation iterations.

2 . The method of claim 1 , wherein the test data set is generated from at least one of: customer vehicles that participate in normal road traffic, vehicles that participate in a test of the system-under-test in normal road traffic.

3 . The method of claim 1 , wherein the test data set is formed by at least one time series of data selected from: sensor data, map material, traffic conditions, and output of the driver assistance system.

4 . The method of claim 1 , wherein the system-under-test is an automated driver assistance system (ADAS) or an automated driving system (ADS).

5 . The method of claim 4 , wherein system-under-test is an open loop ADAS or ADS.

6 . The method of claim 4 , wherein the system-under-test is an open-loop ADAS or ADS that comprises at least one of traffic sign recognition, night vision, ego motion locator.

7 . The method of claim 1 , further comprising testing at least one subcomponent of a closed-loop system using functional decomposition, wherein the closed-loop system comprises at least one closed-loop subcomponent and at least one open-loop subcomponent, and wherein the at least one open-loop subcomponent is selected as the at least one subcomponent to be tested.

8 . The method of claim 1 , further comprising using a plurality of the test benches in parallel by partitioning the test data sets ( 11 ) into a plurality of time periods and supplying test data sets ( 11 ) to the respective test benches at the respective time periods.

9 . The method of claim 1 , wherein based on the evaluating of the changes in the overall software based on the performance statistic, respective changes in the software version are discarded and a new iteration run is started.

10 . A test system that uses a continuous integration approach for improving driver assistance systems installed in vehicles, the test system comprising a server and a hardware-in-the-loop test bench having a computing unit, the computing unit being configured to execute an algorithm according to the method of claim 1 .

11 . A method that uses a continuous integration approach for improving driver assistance systems installed in vehicles, the method comprising:

using sensors on the vehicles for measuring vehicle operating data while driving the respective vehicles in real traffic;

determining vehicle actuator settings based on the operating data sensed by the sensors;

storing the measured vehicle operating data and the vehicle actuator settings in a test data set that comprises time series of input data and output data of the driver assistance system;

testing a system-under-test with the test data set, the system-under-test being formed by an overall software that is changed continuously;

testing at least one subcomponent of a closed-loop system using functional decomposition, wherein the closed-loop system comprises at least one closed-loop subcomponent and at least one open-loop subcomponent, and wherein the at least one open-loop subcomponent is selected as the at least one subcomponent to be tested; and

performing data-driven validations repeatedly at predetermined time intervals, each iteration of the data-driven validations including the steps of:

loading a current software version of the overall software forming the system-under-test onto a server;

compiling the current software version on the server into an executable computer program;

transferring the computer program to a hardware-in-the-loop test bench;

loading the test data set onto the hardware-in-the-loop test bench;

running the computer program on the hardware-in-the-loop test bench while feeding the input data to the test data set;

logging differences in output between the current software version and the output data of the test data set;

simultaneously matching the output of the current software version with ground-truth data, thereby inferring an improvement or deterioration of a performance of the current software version and assigning a performance score to the respective differences;

forming a performance statistic on all differences and their performance score;

evaluating changes in the overall software based on the performance statistic; outputting a report; and

executing the changed overall software during controlling of a real vehicle.

12 . The method of claim 11 , wherein the test data set is generated from at least one of: customer vehicles that participate in normal road traffic, vehicles that participate in a test of the system-under-test in normal road traffic.

13 . The method of claim 11 , wherein the test data set is formed by at least one time series of data selected from: sensor data, map material, traffic conditions, and output of the driver assistance system.

14 . The method of claim 11 , further comprising using a plurality of the test benches in parallel by partitioning the test data sets into a plurality of time periods and supplying test data sets to the respective test benches at the respective time periods.

15 . The method of claim 11 , wherein based on the evaluating of the changes in the overall software based on the performance statistic, respective changes in the software version are discarded and a new iteration run is started.

16 . A method that uses a continuous integration approach for improving driver assistance systems installed in vehicles, the method comprising:

using sensors on the vehicles for measuring vehicle operating data while driving the respective vehicles in real traffic;

determining vehicle actuator settings based on the operating data sensed by the sensors;

storing the measured vehicle operating data and the vehicle actuator settings in a test data set that comprises time series of input data and output data of the driver assistance system;

testing a system-under-test with the test data set, the system-under-test being formed by an overall software that is changed continuously; and

performing data-driven validations repeatedly at predetermined time intervals, each iteration of the data-driven validations including the steps of:

loading a current software version of the overall software forming the system-under-test onto a server;

compiling the current software version on the server into an executable computer program;

transferring the computer program to a hardware-in-the-loop test bench;

loading the test data set onto the hardware-in-the-loop test bench;

running the computer program on the hardware-in-the-loop test bench while feeding the input data to the test data set;

using a plurality of the test benches in parallel by partitioning the test data sets into a plurality of time periods and supplying test data sets to the respective test benches at the respective time periods;

logging differences in output between the current software version and the output data of the test data set;

simultaneously matching the output of the current software version with ground-truth data, thereby inferring an improvement or deterioration of a performance of the current software version and assigning a performance score to the respective differences;

forming a performance statistic on all differences and their performance score;

evaluating changes in the overall software based on the performance statistic; outputting a report; and

executing the changed overall software during controlling of a real vehicle.

17 . The method of claim 16 , wherein the test data set is generated from at least one of: customer vehicles that participate in normal road traffic, vehicles that participate in a test of the system-under-test in normal road traffic.

18 . The method of claim 16 , wherein the test data set is formed by at least one time series of data selected from: sensor data, map material, traffic conditions, and output of the driver assistance system.

19 . The method of claim 16 , wherein based on the evaluating of the changes in the overall software based on the performance statistic, respective changes in the software version are discarded and a new iteration run is started.

20 . The method of claim 16 , wherein the system-under-test is an automated driver assistance system (ADAS) or an automated driving system (ADS).

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (21)