Method and system for closed loop perception in autonomous driving vehicles

1. A method implemented on a computer having at least one processor, a storage, and a model update center, comprising:

establishing at least one model to detect objects surrounding a driving vehicle, wherein the at least one model includes a global model established for distribution to all vehicles in a fleet of driving vehicles and a class model established for distribution to a particular type of vehicle in the fleet of driving vehicles or to a vehicle in the fleet of driving vehicles operating in a particular type of operating environment, wherein the global model and the class model detect objects and features of objects including depth;

receiving, from the fleet of driving vehicles, a plurality of labeled data items each of which is detected, based on the at least one model, from sensor data characterizing surroundings of the driving vehicles, wherein the labeled data items are generated automatically on-the-fly by the driving vehicles;

causing updating of the at least one model based on the plurality of labeled data items, the model update center residing in a cloud comprising a plurality of distributed model update centers that perform a coordinated update of the at least one model, each distributed model update center performing a specific portion of the coordinated update;

generating model update information based on the updated at least one model; and

distributing the model update information to the fleet of driving vehicles.

2. The method of claim 1 , wherein

each of the plurality of distributed model update centers is designated for an update of the global model or for an update of the class model.

3. The method of claim 1 , further comprising:

selecting, based on a global model update data selection configuration, a data item for updating the at least one model; and

labelling the data item as one of:

an inconsistent detection event for correction purpose, and

a consistent detection event for reinforcement purpose.

4. The method of claim 1 , the method further comprising:

selecting a subset of the plurality of labeled data items for updating the at least one model; and

classifying the selected subset to determine whether at least one of the global model or the class model is to be updated.

5. The method of claim 1 , wherein

the distributing of the model update information to the fleet of driving vehicles is in response to generation of the model update information.

6. The method of claim 1 , wherein a first distributed model update center updates the global model and a second distributed model update center updates the class model.

7. The method of claim 6 , wherein the plurality of distributed model update centers includes the first distributed model update center and the second distributed model update center.

8. The method of claim 1 , wherein the global model is updated based on a first set of events of interest and the class model is updated based on a second set of events of interest the first set of events of interest.

9. A machine readable and non-transitory medium having data recorded thereon for a model update center, wherein the data, once read by a machine, cause the machine to perform operations comprising:

establishing at least one model to detect objects surrounding a driving vehicle, wherein the at least one model includes a global model established for distribution to all vehicles in a fleet of driving vehicles and a class model established for distribution to a particular type of vehicle in the fleet of driving vehicles or to a vehicle in the fleet of driving vehicles operating in a particular type of operating environment, wherein the global model and the class model detect objects and features of objects including depth;

receiving, from the fleet of driving vehicles, a plurality of labeled data items each of which is detected, based on the at least one model, from sensor data characterizing surroundings of the driving vehicles, wherein the labeled data items are generated automatically on-the-fly by the driving vehicles;

causing updating of the at least one model based on the plurality of labeled data items, the model update center residing in a cloud comprising a plurality of distributed model update centers that perform a coordinated update of the at least one model, each distributed model update center performing a specific portion of the coordinated update;

generating model update information based on the updated at least one model; and

distributing the model update information to the fleet of driving vehicles.

10. The machine readable and non-transitory medium of claim 9 , wherein

each of the plurality of distributed model update centers is designated for an update of the global model or for an update of the class model.

11. The machine readable and non-transitory medium of claim 9 , wherein the data, once read by the machine, cause the machine to further perform:

selecting, based on a global model update data selection configuration, a data item for updating the at least one model; and

labelling the data item as one of:

an inconsistent detection event for correction purpose, and

a consistent detection event for reinforcement purpose.

12. The machine readable and non-transitory medium of claim 9 ,

wherein the data, once read by a machine, cause the machine to further perform:

selecting a subset of the plurality of labeled data items for updating the at least one model; and

classifying the selected subset to determine whether at least one of the global model or the class model is to be updated.

13. The machine readable and non-transitory medium of claim 9 , wherein

the distributing of the model update information to the fleet of driving vehicles is in response to generation of the model update information.

14. The machine readable and non-transitory medium of claim 9 , wherein a first distributed model update center updates the global model and a second distributed model update center updates the class model.

15. The machine readable and non-transitory medium of claim 14 , wherein the plurality of distributed model update centers includes the first distributed model update center and the second distributed model update center.

16. A system for centralized model update, comprising:

at least one model established to detect objects surrounding a driving vehicle, wherein the at least one model includes a global model established for distribution to all vehicles in a fleet of driving vehicles and a class model established for distribution to a particular type of vehicle in the fleet of driving vehicles or to a vehicle in the fleet of driving vehicles operating in a particular type of operating environment, wherein the global model and the class model detect objects and features of objects including depth; and

a model update center configured for:

receiving, from the fleet of driving vehicles, a plurality of labeled data items each of which is detected, based on the at least one model, from sensor data characterizing surroundings of the driving vehicles, wherein the labeled data items are generated automatically on-the-fly by the driving vehicles;

causing updating of the at least one model based on the plurality of labeled data items, the model update center residing in a cloud comprising a plurality of distributed model update centers that perform a coordinated update of the at least one model, each distributed model update center performing a specific portion of the coordinated update;

generating model update information based on the updated at least one model; and

distributing the model update information to the fleet of driving vehicles.

17. The system of claim 16 , wherein

each of the plurality of distributed model update centers is designated for an update of the global model or for an update of the class model.

18. The system of claim 16 , the model update center is configured further for:

selecting, based on a global model update data selection configuration, a data item for updating the at least one model; and

labelling the data item as one of:

an inconsistent detection event for correction purpose, and

a consistent detection event for reinforcement purpose.

19. The system of claim 18 , wherein the model update center is configured further for:

selecting a subset of the plurality of labeled data items for updating the at least one model; and

classifying the selected subset to determine whether at least one of the global model or the class model is to be updated.

20. The system of claim 18 , wherein

the distributing of the model update information to the fleet of driving vehicles is in response to generation of the model update information.