System and method for federated learning of self-supervised networks in automated driving systems
A computer implemented method and related aspects for updating a perception function of a plurality of vehicles having an Automated Driving System (ADS) are disclosed. The method includes obtaining one or more locally updated model parameters of a self-supervised machine-learning algorithm from a plurality of remote vehicles, and updating one or more model parameters of a global self-supervised machine-learning algorithm based on the obtained one or more locally updated model parameters. Further, the method includes fine-tuning the global self-supervised machine-learning algorithm based on an annotated dataset in order to generate a fine-tuned global machine-learning algorithm comprising one or more fine-tuned model parameters. The method further includes forming a machine-learning algorithm for an in-vehicle perception module based on the fine-tuned global machine-learning algorithm, and transmitting one or more model parameters of the formed machine-learning algorithm for the in-vehicle perception module to the plurality of remote vehicles.
1. A computer-implemented method for updating a perception function of a plurality of vehicles having an Automated Driving System (ADS), the method (S 100 ) comprising:
obtaining one or more locally updated model parameters of a self-supervised machine-learning algorithm from a plurality of remote vehicles;
updating one or more model parameters of a global self-supervised machine-learning algorithm based on the obtained one or more locally updated model parameters;
fine-tuning the global self-supervised machine-learning algorithm based on an annotated dataset in order to generate a fine-tuned global machine-learning algorithm comprising one or more fine-tuned model parameters;
forming a machine-learning algorithm for an in-vehicle perception module based on the fine-tuned global machine-learning algorithm; and
transmitting one or more model parameters of the formed machine-learning algorithm for the in-vehicle perception module to the plurality of remote vehicles.
2. The method according to claim 1 , wherein the forming of the machine-learning algorithm for the in-vehicle perception module comprises:
distilling a machine-learning algorithm for the in-vehicle perception module from the fine-tuned global machine-learning algorithm acting as a teacher model, wherein the distillation is performed using input data in the form of images of traffic scenarios.
3. The method according to claim 1 , wherein the forming of the machine-learning algorithm for the in-vehicle perception module comprises:
transmitting the fined-tuned model parameters of the fined-tuned global self-supervised machine-learning algorithm to the plurality of remote vehicles;
obtaining one or more locally distilled model parameters of the machine-learning algorithm for the in-vehicle perception module from each of the plurality of remote vehicles; and
forming the machine-learning algorithm for the in-vehicle perception module based on a consolidation of the one or more locally distilled model parameters.
4. The method according to claim 1 , wherein the forming of the machine-learning algorithm for the in-vehicle perception module comprises:
validating the formed machine-learning algorithm for the in-vehicle perception module based on an annotated dataset prior to the transmission of the one or more model parameters of the formed machine-learning algorithm for the in-vehicle perception module.
5. The method according to claim 4 , further comprising:
transmitting the one or more updated model parameters of the global self-supervised machine-learning algorithm to the plurality of remote vehicles.
6. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a processing system, the one or more programs comprising instructions for performing the method according to claim 1 .
7. A processing system for updating a perception function of a vehicle having an Automated Driving System (ADS), the processing system comprising control circuitry configured to:
obtain one or more locally updated model parameters of a self-supervised machine-learning algorithm from a plurality of remote vehicles;
update one or more model parameters of a global self-supervised machine-learning algorithm based on the obtained one or more locally updated model parameters;
fine-tune the global self-supervised machine-learning algorithm based on an annotated dataset in order to generate a fine-tuned global machine-learning algorithm comprising one or more fine-tuned model parameters;
form a machine-learning algorithm for an in-vehicle perception module based on the fine-tuned global machine-learning algorithm; and
transmit one or more model parameters of the formed machine-learning algorithm for the in-vehicle perception module to the plurality of remote vehicles.
8. The processing system according to claim 7 , wherein the control circuitry is configured to form the machine-learning algorithm for the in-vehicle perception module by being configured to:
distil a machine-learning algorithm for the in-vehicle perception module from the fine-tuned global machine-learning algorithm acting as a teacher model, wherein the distillation is performed using input data in the form of images of traffic scenarios.
9. The processing system according to claim 7 , wherein the control circuitry is configured form the machine-learning algorithm for the in-vehicle perception module by being configured to:
transmit the fined-tuned model parameters of the fined-tuned global self-supervised machine-learning algorithm to the plurality of remote vehicles;
obtain one or more locally distilled model parameters of a local machine-learning algorithm for the in-vehicle perception module from each of the plurality of remote vehicles; and
form the machine-learning algorithm for the in-vehicle perception module based on a consolidation of the one or more locally distilled model parameters.
10. A remote server comprising the processing system according to claim 7 .
11. A cloud environment comprising one or more remote servers according to claim 10 .
12. A computer-implemented method, performed by an in-vehicle processing system, for updating a perception function of a vehicle having an Automated Driving System (ADS), the method comprising:
obtaining perception data from one or more vehicle-mounted sensors configured to monitor a surrounding environment of the vehicle;
processing the obtained perception data using a self-supervised machine-learning algorithm and locally updating one or more model parameters of the self-supervised machine-learning algorithm;
transmitting the locally updated model parameters of the self-supervised machine-learning algorithm to a remote entity;
obtaining a centrally fine-tuned machine-learning algorithm formed from a consolidated version of the self-supervised machine-learning algorithm from the remote entity;
distilling a machine-learning algorithm for an in-vehicle perception module from the centrally fine-tuned machine-learning algorithm acting as a teacher model, wherein the distillation is performed using perception data obtained from one or more vehicle-mounted sensors;
transmitting one or more model parameters of the distilled machine-learning algorithm to the remote entity;
receiving one or more consolidated model parameters of the machine-learning algorithm for the in-vehicle perception module from the remote entity; and
updating the machine-learning algorithm for the in-vehicle perception module ( 43 ) based on the received one or more consolidated model parameters.
13. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an in-vehicle processing system, the one or more programs comprising instructions for performing the method according to claim 12 .
14. An in-vehicle processing system for updating a perception function of a vehicle having an Automated Driving System (ADS), the processing system comprising control circuitry configured to:
obtain perception data from one or more vehicle-mounted sensors configured to monitor a surrounding environment of the vehicle;
process the obtained perception data using a self-supervised machine-learning algorithm and locally update one or more model parameters of the self-supervised machine-learning algorithm;
transmit the locally updated model parameters of the self-supervised machine-learning algorithm to a remote entity;
obtain a centrally fine-tuned machine-learning algorithm formed from a consolidated version of the self-supervised machine-learning algorithm from the remote entity;
distil a machine-learning algorithm for an in-vehicle perception module from the centrally fine-tuned machine-learning algorithm acting as a teacher model, wherein the distillation is performed using perception data obtained from one or more vehicle-mounted sensors;
transmit one or more model parameters of the distilled machine-learning algorithm to the remote entity;
receive one or more consolidated model parameters of the machine-learning algorithm for the in-vehicle perception module from the remote entity; and
update the machine-learning algorithm for the in-vehicle perception module based on the received one or more consolidated model parameters.
15. A vehicle comprising:
an Automated Driving System (ADS); and
an in-vehicle processing system according to claim 14 .