Data-to-camera (D2C) based filters for improved object detection in images based on vehicle-to-everything communication
The disclosure describes a method for an ego vehicle. The method includes receiving a vehicle-to-everything (V2X) message that describes an object that is within proximity of an ego vehicle. The method further includes generating a set of data-to-camera (D2C) filters that are specific to the object described by the V2X message. The method further includes applying the set of D2C filters to image data that describes an initial image of the object. The method further includes generating a modified image, based on applying the set of D2C filters to the image data, wherein the modified image includes an indication of (1) a location and a size of the object in the initial image and (2) a type of object in the initial image.
1. A method comprising:
receiving a vehicle-to-everything (V2X) message that describes an object that is within proximity of an ego vehicle;
generating a set of data-to-camera (D2C) filters that are specific to the object described by the V2X message by:
determining where the object is expected to appear in an initial image,
generating a first D2C filter for an object type, and
generating a second D2C filter for an object location based on where the object is expected to appear in the initial image;
applying the set of D2C filters to image data that describes the initial image of the object; and
generating a modified image, based on applying the set of D2C filters to the image data, wherein the modified image includes an indication of (1) a location and a size of the object in the initial image and (2) a type of object in the initial image.
2. The method of claim 1 , further comprising:
providing the modified image to an Advanced Driver Assistance System of the ego vehicle to reduce or eliminate a driving hazard caused by the object.
3. The method of claim 1 , wherein the V2X message includes a Basic Safety Message (BSM) that describes the location of the object, the size of the object, an identity of the object, and a heading of the object.
4. The method of claim 1 , wherein the D2C filters are generated using a trained machine-learning model.
5. The method of claim 1 , further comprising:
receiving range data describing a range between the object and the ego vehicle, wherein the range is measured by a range finding sensor of the ego vehicle.
6. The method of claim 1 , wherein the object is a remote vehicle and the V2X message is a Basic Safety Message (BSM) received from the remote vehicle.
7. The method of claim 1 , wherein the set of D2C filters are generated in real time and the set of D2C filters describe digital data that is operable to modify the initial image.
8. The method of claim 1 , wherein the modified image further includes a bounding box that surrounds the object.
9. A system for an ego vehicle, comprising:
a processor; and
a non-transitory memory storing computer code which, when executed by the processor, causes the processor to:
receive a vehicle-to-everything (V2X) message that describes an object that is within proximity of an ego vehicle;
generate a set of data-to-camera (D2C) filters that are specific to the object described by the V2X message by:
determining where the object is expected to appear in an initial image,
generating a first D2C filter for an object type, and
generating a second D2C filter for an object location based on where the object is expected to appear in the initial image;
apply the set of D2C filters to image data that describes an initial image of the object; and
generate a modified image, based on applying the set of D2C filters to the image data, wherein the modified image includes an indication of (1) a location and a size of the object in the initial image and (2) a type of object in the initial image.
10. The system of claim 9 , wherein the computer code, when executed by the processor, further causes the processor to:
provide the modified image to an Advanced Driver Assistance System of the ego vehicle to reduce or eliminate a driving hazard caused by the object.
11. The system of claim 9 , wherein the V2X message includes a Basic Safety Message (BSM) that describes the location of the object, the size of the object, an identity of the object, and a heading of the object.
12. The system of claim 9 , wherein the D2C filters are generated using a trained machine-learning model.
13. The system of claim 9 , wherein the computer code, when executed by the processor, further causes the processor to:
receive range data describing a range between the object and the ego vehicle, wherein the range is measured by a range finding sensor of the ego vehicle.
14. The system of claim 9 , wherein the object is a remote vehicle and the V2X message is a Basic Safety Message (BSM) received from the remote vehicle.
15. A computer program product comprising a non-transitory memory storing computer-executable code that, when executed by a processor, causes the processor to:
receive a vehicle-to-everything (V2X) message that describes an object that is within proximity of an ego vehicle;
generate a set of data-to-camera (D2C) filters that are specific to the object described by the V2X message by:
determining where the object is expected to appear in an initial image,
generating a first D2C filter for an object type, and
generating a second D2C filter for an object location based on where the object is expected to appear in the initial image;
apply the set of D2C filters to image data that describes an initial image of the object; and
generate a modified image, based on applying the set of D2C filters to the image data, wherein the modified image includes an indication of (1) a location and a size of the object in the initial image and (2) a type of object in the initial image.
16. The computer program product of claim 15 , wherein the computer-executable code further causes the processor to:
provide the modified image to an Advanced Driver Assistance System of the ego vehicle to reduce or eliminate a driving hazard caused by the object.
17. The computer program product of claim 15 , wherein the V2X message includes a Basic Safety Message (BSM) that describes the location of the object, the size of the object, an identity of the object, and a heading of the object.
18. The computer program product of claim 15 , wherein the D2C filters are generated using a trained machine-learning model.
19. The computer program product of claim 15 , wherein the computer-executable code further causes the processor to:
receive range data describing a range between the object and the ego vehicle, wherein the range is measured by a range finding sensor of the ego vehicle.
20. The computer program product of claim 15 , wherein the object is a remote vehicle and the V2X message is a Basic Safety Message (BSM) received from the remote vehicle.