Granted Patent B2

US 12,320,653 · App. 18/330,303 · Granted Jun 3, 2025

Inventors: Michel Laverne (Pittsburgh, PA); Ilan Biala (Pittsburgh, PA)

Assignee: Volkswagen Group of America Investments, LLC

G01C21/3423G01C21/3602G01C21/3893H04W4/44H04W4/46

Abstract

Claims (81)

1. A method comprising:

receiving, by a smart infrastructure device of a smart infrastructure system and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;

by a camera of the smart infrastructure system, capturing an image, video, or real-time feed of an environment of the smart infrastructure device;

by a processor of the smart infrastructure system:

identifying a second vehicle in the image, video, or real-time feed;

determining that the smart infrastructure device has not received communication from the second vehicle;

responsive to determining that the smart infrastructure device has not received communication from the second vehicle, transmitting a communication request to the second vehicle;

determining that the second vehicle is incapable of performing communication with the smart infrastructure device in response to not receiving a response to the communication request from the second vehicle within a threshold period of time;

responsive to determining that the second vehicle is incapable of performing communication with the smart infrastructure device, generating second information associated with the second vehicle, wherein the second information comprises a representation of the second vehicle in a 3D coordinate system;

converting the second information from the representation in the 3D coordinate system into the first format; and

sending the second information in the first format associated with the first communication protocol to the first vehicle.

2. The method of claim 1 , further comprising, before sending the second information to the first vehicle, converting the first information into a format of the 3D coordinate system and adding the first information to the 3D coordinate system.

3. The method of claim 1 , wherein converting the second information from the representation in the 3D coordinate system into the first format comprises using the 3D coordinate system including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

4. The method of claim 1 , wherein converting the second information from the representation in the 3D coordinate system into the first format comprises converting the second information from a second format into an agnostic format, and then converting the second information from the agnostic format into the first format.

5. The method of claim 1 , wherein determining that the smart infrastructure device has not received communication from the second vehicle comprises:

identifying a location of the first vehicle;

comparing the location of the first vehicle to a location of a vehicle in the image, video, or real-time feed;

determining that the location of the first vehicle is a same location as the location of the vehicle in the image, video, or real-time feed;

analyzing the image, video, or real-time feed to identify a location of the second vehicle; and

determining that the location of the first vehicle is not the same as the location of the second vehicle in the image, video, or real-time feed.

6. The method of claim 1 , further comprising:

converting at least one of the first information or the second information into third information in a third format associated with a third communication protocol; and

sending the third information in the third format to a third vehicle.

7. The method of claim 1 , further comprising:

receiving, by the smart infrastructure device and from a non-vehicle device, information associated with the non-vehicle device in a different format associated with a communication protocol other than the first communication protocol;

converting the information associated with the non-vehicle device into the first format; and

sending the information associated with the non-vehicle device in the first format associated with the first communication protocol to the first vehicle.

8. A smart infrastructure system comprising:

a processor;

a camera; and

a memory storing computer-executable instructions, that when executed by the processor, will cause the processor to:

receive, by a smart infrastructure device of the smart infrastructure system and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;

receive, from the camera, a captured image, video, or real-time feed of an environment of the smart infrastructure device;

identify a second vehicle in the image, video, or real-time feed;

determine that the smart infrastructure device has not received communication from the second vehicle;

responsive to determining that the smart infrastructure device has not received communication from the second vehicle, transmit a communication request to the second vehicle;

determine that the second vehicle is incapable of performing communication with the smart infrastructure device in response to not receiving a response to the communication request from the second vehicle within a threshold period of time;

responsive to determining that the second vehicle is incapable of performing communication with the smart infrastructure device, analyze the image, video, or real-time feed to generate second information associated with the second vehicle, wherein the second information comprises a representation of the second vehicle in a 3D coordinate system;

converting the second information from the representation in the 3D coordinate system into the first format; and

sending the second information in the first format associated with the first communication protocol to the first vehicle.

9. The system of claim 8 , wherein the computer-executable instructions will, when executed by the processor, further cause the processor to, before sending the second information to the first vehicle, convert the first information into a format of the 3D coordinate system and adding the first information to the 3D coordinate system.

10. The system of claim 8 , wherein the instructions to convert the second information from the representation in the 3D coordinate system into the first format include instructions to use the 3D coordinate system including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

11. The system of claim 8 , wherein the instructions to convert the second information from the representation in the 3D coordinate system into the first format comprise instructions to convert the second information from a second format into an agnostic format, and then converting the second information from the agnostic format into the first format.

12. The system of claim 8 , wherein the instructions to determine that the smart infrastructure device has not received communication from the second vehicle comprise instructions to do so in response to:

identifying a location of the first vehicle;

comparing the location of the first vehicle to a location of a vehicle in the image, video, or real-time feed;

determining that the location of the first vehicle is a same location as the location of the vehicle in the image, video, or real-time feed;

analyzing the image, video, or real-time feed to identify a location of the second vehicle; and

determining that the location of the first vehicle is not the same as the location of the second vehicle in the image, video, or real-time feed.

13. The system of claim 8 , wherein the computer-executable instructions further cause the processor to:

convert at least one of the first information or the second information into third information in a third format associated with a third communication protocol; and

send the third information in the third format to a third vehicle.

14. The system of claim 8 , wherein the computer-executable instructions further cause the processor to:

receive, by the smart infrastructure device and from a non-vehicle device, fourth information associated with the non-vehicle device in a different format associated with a communication protocol other than the first communication protocol;

convert the information associated with the non-vehicle device into the first format; and

send the information associated with the non-vehicle device in the first format associated with the first communication protocol to the first vehicle.

15. A non-transitory computer readable medium including computer-executable instructions stored thereon, which when executed by one or more processors, will cause the one or more processors to perform operations comprising:

receiving, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;

capturing an image, video, or real-time feed of an environment of the smart infrastructure device;

identifying a second vehicle in the image, video, or real-time feed;

determining that communication has not been received from the second vehicle;

responsive to determining that communication has not been received from the second vehicle, transmitting a communication request to the second vehicle;

determining that the second vehicle is incapable of performing communication with the smart infrastructure device in response to not receiving a response to the communication request from the second vehicle within a threshold period of time;

responsive to determining that the second vehicle is incapable of performing communication with the smart infrastructure device, generating second information associated with the second vehicle, wherein the second information comprises a representation of the second vehicle in a 3D coordinate system;

converting the second information from the representation in the 3D coordinate system into the first format; and

sending the second information in the first format associated with the first communication protocol to the first vehicle.

16. The non-transitory computer readable medium of claim 15 , wherein the computer-executable instructions will, when executed by the one or more processors, further cause the one or more processors to, before sending the second information to the first vehicle, convert the first information into a format of the 3D coordinate system and adding the first information to the 3D coordinate system.

17. The non-transitory computer readable medium of claim 15 , wherein the instructions to convert the second information from the representation in the 3D coordinate system into the first format include instructions to use the 3D coordinate system including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

18. The non-transitory computer readable medium of claim 15 , wherein the instructions to determine that communication has not been received from the second vehicle comprise instructions to do so in response to:

identifying a location of the first vehicle;

comparing the location of the first vehicle to a location of a vehicle in the image, video, or real-time feed;

determining that the location of the first vehicle is a same location as the location of the vehicle in the image, video, or real-time feed;

analyzing the image, video, or real-time feed to identify a location of the second vehicle; and

determining that the location of the first vehicle is not the same as the location of the second vehicle in the image, video, or real-time feed.

19. The non-transitory computer readable medium of claim 15 , wherein the computer-executable instructions further cause the one or more processors to perform operations comprising:

converting at least one of the first information or the second information into third information in a third format associated with a third communication protocol; and

sending the third information in the third format to a third vehicle.

20. The non-transitory computer readable medium of claim 15 , wherein the computer-executable instructions further cause the one or more processors to perform operations comprising:

receiving, by the smart infrastructure device and from a non-vehicle device, fourth information associated with the non-vehicle device in a different format associated with a communication protocol other than the first communication protocol;

converting the information associated with the non-vehicle device into the first format; and

sending the information associated with the non-vehicle device in the first format associated with the first communication protocol to the first vehicle.

Assignments (2)

Continuity (2)

References Cited (25)