IP Library Granted Patent US 11,709,061
Granted Patent B2
US 11,709,061 · App. 17/073,119 · Granted Jul 25, 2023

Systems and methods for multi-modal transfer capabilities for smart infrastructure

Inventors: Michel H. J. Laverne (Pittsburgh, PA); Ilan Biala (Pittsburgh, PA)
Assignee: Argo AI, LLC
G01C21/3423G01C21/3602G01C21/3893H04W4/44H04W4/46
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,709,061
App. No.
17/073,119
Granted
Jul 25, 2023
Kind
B2
Abstract

Systems, methods, and computer-readable media are disclosed for improved smart infrastructure data transfer. An example method may involve receiving, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol. The first information is converted from the first format into an agnostic format. An image, video, or real-time feed of an environment of the smart infrastructure device is captured. The first vehicle and a second vehicle in the image, video, or real-time feed is identified. It is determined that the second vehicle is temporarily or permanently incapable of performing a communication with the smart infrastructure device based on the image, video, or real-time feed. The image, video, or real-time feed is analyzed to generate second information associated with the second vehicle. The second information is converted into the agnostic format.

Claims (80)

1. A method comprising:

receiving, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;

converting the first information from the first format into an agnostic format;

capturing an image, video, or real-time feed of an environment of the smart infrastructure device;

identifying the first vehicle and a second vehicle in the image, video, or real-time feed;

determining that communication has not been received from the second vehicle;

responsive to determining that communication has not been received from the second vehicle, transmitting a communication request to the second vehicle;

determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device in response to not receiving a response from the second vehicle within a threshold period of time;

responsive to determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device, analyzing the image, video, or real-time feed to generate second information associated with the second vehicle;

converting the second information into the agnostic format;

converting the first information and the second information from the agnostic format into third information in the first format; and

sending the third information to the first vehicle.

2. The method of claim 1 , wherein the agnostic format includes a 3D coordinate map including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

3. The method of claim 1 , wherein determining that communication has not been received from the second vehicle comprises:

extracting information relating to a location of the first vehicle from the first information;

comparing the information relating to the location of the first vehicle that is extracted from the first information to a location of a vehicle in the image, video, or real-time feed;

determining that the location of the first vehicle that is extracted from the first information is a same location as the location of the vehicle in the image, video, or real-time feed;

analyzing the image, video, or real-time feed to identify a location of the second vehicle; and

determining that the location of the first vehicle from the first information is not the same as the location of the second vehicle in the image, video, or real-time feed.

4. The method of claim 1 , further comprising:

converting at least one of the first information or the second information into the third information in a second format associated with a second communication protocol; and

sending the third information in the second format to a third vehicle using the second communication protocol.

5. The method of claim 1 , further comprising:

receiving, by the smart infrastructure device and from a non-vehicle device, fourth information associated with the non-vehicle device in a fourth format associated with a fourth communication protocol; and

converting the fourth information into the agnostic format.

6. The method of claim 1 , wherein converting the first information from the first format into the agnostic format is performed using a look-up table including information about the first format.

7. A system comprising:

a processor; and

a memory storing computer-executable instructions, that when executed by the processor,

cause the processor to:

receive, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;

convert the first information from the first format into an agnostic format;

capture an image, video, or real-time feed of an environment of the smart infrastructure device;

identify the first vehicle and a second vehicle in the image, video, or real-time feed;

determine that communication has not been received from the second vehicle;

responsive to determining that communication has not been received from the second vehicle, transmit a communication request to the second vehicle;

determine that the second vehicle is permanently incapable of performing communication with the smart infrastructure device in response to not receiving a response from the second vehicle within a threshold period of time;

responsive to determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device, analyze the image, video, or real-time feed to generate second information associated with the second vehicle;

convert the second information into the agnostic format;

convert the first information and the second information from the agnostic format into third information in the first format; and

send the third information to the first vehicle.

8. The system of claim 7 , wherein the agnostic format includes a 3D coordinate map including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

9. The system of claim 7 , wherein determining that communication has not been received from the second vehicle comprises:

extract information relating to a location of the first vehicle from the first information;

compare the information relating to the location of the first vehicle that is extracted from the first information to a location of a vehicle in the image, video, or real-time feed;

determine that the location of the first vehicle that is extracted from the first information is a same location as the location of the vehicle in the image, video, or real-time feed;

analyze the image, video, or real-time feed to identify a location of the second vehicle; and

determine that the location of the first vehicle from the first information is not the same as the location of the second vehicle in the image, video, or real-time feed.

10. The system of claim 7 , wherein the computer-executable instructions further cause the processor to:

convert at least one of the first information or the second information into the third information in a second format associated with a second communication protocol; and

send the third information in the second format to a third vehicle using the second communication protocol.

11. The system of claim 7 , wherein the computer-executable instructions further cause the processor to:

receive, by the smart infrastructure device and from a non-vehicle device, fourth information associated with the non-vehicle device in a fourth format associated with a fourth communication protocol; and

convert the fourth information into the agnostic format.

12. The system of claim 7 , wherein converting the first information from the first format into the agnostic format is performed using a look-up table including information about the first format.

13. A non-transitory computer readable medium including computer-executable instructions stored thereon, which when executed by one or more processors, cause the one or more processors to perform operations of:

receiving, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;

converting the first information from the first format into an agnostic format;

capturing an image, video, or real-time feed of an environment of the smart infrastructure device;

identifying the first vehicle and a second vehicle in the image, video, or real-time feed;

determining that communication has not been received from the second vehicle;

responsive to determining that communication has not been received from the second vehicle, transmitting a communication request to the second vehicle;

determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device in response to not receiving a response from the second vehicle within a threshold period of time;

responsive to determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device, analyzing the image, video, or real-time feed to generate second information associated with the second vehicle;

converting the second information into the agnostic format;

converting the first information and the second information from the agnostic format into third information in the first format; and

sending the third information to the first vehicle.

14. The non-transitory computer readable medium of claim 13 , wherein the agnostic format includes a 3D coordinate map including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

15. The non-transitory computer readable medium of claim 13 , wherein determining that communication has not been received from the second vehicle comprises:

extracting information relating to a location of the first vehicle from the first information;

comparing the information relating to the location of the first vehicle that is extracted from the first information to a location of a vehicle in the image, video, or real-time feed;

determining that the location of the first vehicle that is extracted from the first information is a same location as the location of the vehicle in the image, video, or real-time feed;

analyzing the image, video, or real-time feed to identify a location of the second vehicle; and

determining that the location of the first vehicle from the first information is not the same as the location of the second vehicle in the image, video, or real-time feed.

16. The non-transitory computer readable medium of claim 13 , wherein the computer-executable instructions further cause the one or more processors to perform operations of:

converting at least one of the first information or the second information into the third information in a second format associated with a second communication protocol; and

sending the third information in the second format to a third vehicle using the second communication protocol.

17. The non-transitory computer readable medium of claim 13 , wherein the computer-executable instructions further cause the one or more processors to perform operations of:

receiving, by the smart infrastructure device and from a non-vehicle device, fourth information associated with the non-vehicle device in a fourth format associated with a fourth communication protocol; and

converting the fourth information into the agnostic format.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2024
From: ARGO AI, LLC
To: VOLKSWAGEN GROUP OF AMERICA INVESTMENTS, LLC
Reel/Frame 069177/0099 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: LAVERNE, MICHEL H.J.; BIALA, ILAN
To: ARGO AI, LLC
Reel/Frame 054169/0493 →
Continuity (1)
Related Publication 20220120571A1 · Apr 21, 2022