IP Library Granted Patent US 11,796,999
Granted Patent B2
US 11,796,999 · App. 17/400,433 · Granted Oct 24, 2023

Systems and methods for monitoring a self-driving vehicle

Inventors: Anthony William Tod (St. Agatha, CA); Ryan Christopher Gariepy (Kitchener, CA)
Assignee: CLEARPATH ROBOTICS INC.
G05D1/0038G05D1/0016G05D1/0291H04N7/183
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,796,999
App. No.
17/400,433
Granted
Oct 24, 2023
Kind
B2
Abstract

Systems and methods for monitoring a self-driving vehicle are presented. The system comprises a camera, a processor, a communications transceiver, a computer-readable medium, and a display device. The processor can be configured to receive an image of a self-driving vehicle from the camera, and vehicle information from the self-driving vehicle. A graphic comprising the image of the self-driving vehicle and a visual representation of the vehicle information is then displayed on the display device. The vehicle information may comprise any or all of vehicle-status information, vehicle-mission information, vehicle-metric information, and vehicle-environment information.

Claims (68)

1. A device for monitoring a plurality of self-driving vehicles, comprising:

a processor;

a camera in communication with the processor;

a communications transceiver in communication with the processor for communicating with the plurality of self-driving vehicles;

a computer-readable medium in communication with the processor;

a display device in communication with the processor, the display device being adjacent to the camera;

the computer-readable medium storing instructions that, when executed, configure the processor to:

receive an image of at least one unidentified self-driving vehicle of the plurality of self-driving vehicles, the image captured by the camera;

identify the at least one unidentified self-driving vehicle based on the image captured by the camera;

determine an orientation of the at least one identified self-driving vehicle relative to the camera based on the image captured by the camera;

receive vehicle information associated with the at least one identified self-driving vehicle;

generate a graphic comprising the image captured by the camera and a visual representation of the vehicle information, wherein the visual representation of the vehicle information is oriented within the graphic based on the orientation of the at least one identified self-driving vehicle relative to the camera; and

display the graphic on the display device.

2. The device of claim 1 , wherein the vehicle information comprises vehicle-status information.

3. The device of claim 2 , wherein the vehicle-status information comprises a vehicle operating state.

4. The device of claim 1 , wherein the vehicle information comprises vehicle-mission information.

5. The device of claim 4 , wherein the vehicle-mission information comprises a destination location.

6. The device of claim 1 , wherein the vehicle information comprises vehicle-environment information.

7. The device of claim 6 , wherein the vehicle-environment information comprises sensor scan data.

8. The device of claim 6 , wherein the vehicle-environment information comprises camera data.

9. The device of claim 1 , wherein the graphic further comprises at least one input button for receiving commands for controlling the movement of the at least one identified self-driving vehicle, the at least one input button being oriented within the graphic relative to the orientation of the at least one identified self-driving vehicle.

10. A method for monitoring a plurality of self-driving vehicles, comprising:

capturing an image of at least one unidentified self-driving vehicle of the plurality of self-driving vehicles using a monitoring device in communication with the plurality of self-driving vehicles;

identifying the at least one unidentified self-driving vehicle based on the image captured by the monitoring device;

determining an orientation of the at least one identified self-driving vehicle relative to the monitoring device based on the image captured by the monitoring device;

receiving vehicle information associated with the at least one identified self-driving vehicle using the monitoring device;

generating a graphic comprising the image captured by the monitoring device and a visual representation of the vehicle information, wherein the visual representation of the vehicle information is oriented within the graphic based on the orientation of the at least one identified self-driving vehicle relative to the monitoring device; and

displaying the graphic at the monitoring device.

11. The method of claim 10 , wherein identifying the at least one unidentified self-driving vehicle in the image captured by the monitoring device is based on at least one visual identifier tag on the at least one unidentified self-driving vehicle in the image captured by the monitoring device.

12. The method of claim 10 , wherein the at least one identified self-driving vehicle is a first identified self-driving vehicle, the image is a first image, the vehicle information is first-vehicle information, the method further comprising:

capturing a second image of a second unidentified self-driving vehicle using the monitoring device;

identifying the second unidentified self-driving vehicle based on the second image captured by the monitoring device;

determining an orientation of the second identified self-driving vehicle relative to the monitoring device based on the second image captured by the monitoring device; and

receiving second-vehicle information associated with the second identified self-driving vehicle using the monitoring device;

wherein the displayed graphic further comprises the first image and the visual representation of the first vehicle information in association with the first image, and the second image and a visual representation of the second-vehicle information in association with the second image, wherein the visual representation of the second-vehicle information is oriented within the graphic based on the orientation of the second identified self-driving vehicle relative to the monitoring device.

13. The method of claim 10 , wherein the vehicle information comprises vehicle-mission information.

14. The method of claim 10 , wherein the vehicle information comprises vehicle-environment information.

15. The method of claim 10 , wherein the graphic further comprises at least one input button for receiving commands for controlling the movement of the at least one identified self-driving vehicle, the at least one input button being oriented within the graphic relative to the orientation of the at least one identified self-driving vehicle.

16. A system for monitoring a plurality of self-driving vehicles, comprising:

a fleet-management system for communicating with the plurality of self-driving vehicles;

a display terminal in communication with the fleet-management system;

one or more cameras in communication with the display terminal, at least one camera of the one or more cameras being adjacent to the display terminal; the display terminal having a display device, a processor, and a computer-readable medium storing instructions that, when executed, configure the processor to:

receive an image of at least one unidentified self-driving vehicle of the plurality of self-driving vehicles, the image captured by the at least one camera;

identify the at least one unidentified self-driving vehicle based on the image captured by the at least one camera;

determine an orientation of the at least one identified self-driving vehicle relative to the at least one camera based on the image captured by the at least one camera;

receive vehicle information associated with the at least one identified self-driving vehicle;

generate a graphic comprising the image captured by the at least one camera and a visual representation of the vehicle information, wherein the visual representation of the vehicle information is oriented within the graphic based on the orientation of the at least one identified self-driving vehicle relative to the at least one camera; and

display the graphic on the display device.

17. The system of claim 16 , wherein the processor is configured, when the instructions are executed, to receive vehicle information associated with the at least one identified self-driving vehicle from the fleet-management system based on the identity of the at least one identified self-driving vehicle.

18. The method of claim 10 , wherein the vehicle information comprises vehicle-state information.

19. The device of claim 1 , wherein the camera is a first camera, the at least one identified self-driving vehicle is a first identified self-driving vehicle, the image is a first image, the vehicle information is first-vehicle information, and when the instructions are executed, the processor is configured to:

receive a second image of a second unidentified self-driving vehicle of the plurality of self-driving vehicles, the second image captured by a second camera in communication with the processor;

identify the second unidentified self-driving vehicle based on the second image captured by the second camera;

determine an orientation of the second identified self-driving vehicle relative to the second camera based on the second image captured by the second camera; and

receive second-vehicle information associated with the second identified self-driving vehicle;

wherein the displayed graphic comprises the first image and the visual representation of the first vehicle information in association with the first image, and the second image and a visual representation of the second-vehicle information in association with the second image, wherein the visual representation of the second-vehicle information is oriented within the graphic based on the orientation of the second identified self-driving vehicle relative to the second camera.

20. The method of claim 10 , wherein the at least one identified self-driving vehicle is a first identified self-driving vehicle, the image is a first image, the vehicle information is first-vehicle information, the method further comprising:

receiving a second image of a second unidentified self-driving vehicle of the plurality of self-driving vehicles, the second image captured by a second camera in communication with the processor;

identifying the second unidentified self-driving vehicle based on the second image captured by the second camera;

determining an orientation of the second identified self-driving vehicle relative to the second camera based on the second image captured by the second camera; and

receiving second-vehicle information associated with the second identified self-driving vehicle;

wherein the displayed graphic comprises the first image and the visual representation of the first vehicle information in association with the first image, and the second image and a visual representation of the second-vehicle information in association with the second image, wherein the visual representation of the second-vehicle information is oriented within the graphic based on the orientation of the second identified self-driving vehicle relative to the second camera.

21. The system of claim 16 , wherein the at least one camera is a first camera, the at least one identified self-driving vehicle is a first identified self-driving vehicle, the image is a first image, the vehicle information is first-vehicle information, and when the instructions are executed, the processor is configured to:

receive a second image of a second unidentified self-driving vehicle of the plurality of self-driving vehicles, the second image captured by a second camera of the one or more cameras;

identify the second unidentified self-driving vehicle based on the second image captured by the second camera;

determine an orientation of the second identified self-driving vehicle relative to the second camera based on the second image captured by the second camera; and

receive second-vehicle information associated with the second identified self-driving vehicle;

wherein the displayed graphic comprises the first image and the visual representation of the first vehicle information in association with the first image, and the second image and a visual representation of the second-vehicle information in association with the second image, wherein the visual representation of the second-vehicle information is oriented within the graphic based on the orientation of the second identified self-driving vehicle relative to the second camera.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY'S NAME FROM CLEARPATH ROBOTICS, INC. TO CLEARPATH ROBOTICS INC. (WITHOUT THE COMMA) PREVIOUSLY RECORDED ON REEL 67944 FRAME 916. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jul 25, 2024
From: CLEARPATH ROBOTICS INC.
To: ROCKWELL AUTOMATION, INC.
Reel/Frame 068233/0542 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2024
From: CLEARPATH ROBOTICS, INC.
To: ROCKWELL AUTOMATION, INC.
Reel/Frame 067944/0916 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2024
From: ROCKWELL AUTOMATION, INC.
To: ROCKWELL AUTOMATION TECHNOLOGIES, INC.
Reel/Frame 067944/0982 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2021
From: TOD, ANTHONY; GARIEPY, RYAN CHRISTOPHER
To: CLEARPATH ROBOTICS INC.
Reel/Frame 057161/0532 →
Continuity (3)
Continuation 16044598 · Jul 25, 2018
Provisional Application 62537195 · Jul 26, 2017
Related Publication 20210373555A1 · Dec 2, 2021