IP Library Granted Patent US 11,119,476
Granted Patent B2
US 11,119,476 · App. 16/044,598 · Granted Sep 14, 2021

Systems and methods for monitoring a self driving vehicle

Inventors: Anthony William Tod (St. Agatha, CA); Ryan Christopher Gariepy (Kitchener, CA)
Assignee: Clearpath Robotics Inc.
G05D1/0038G05D1/0016G05D1/0291H04N7/183
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,119,476
App. No.
16/044,598
Granted
Sep 14, 2021
Kind
B2
Abstract

Systems and methods for monitoring a self-driving vehicle are presented. The system comprises a camera, a processor, a communications transceiver, a computer-readable medium, and a display device. The processor can be configured to receive an image of a self-driving vehicle from the camera, and vehicle information from the self-driving vehicle. A graphic comprising the image of the self-driving vehicle and a visual representation of the vehicle information is then displayed on the display device. The vehicle information may comprise any or all of vehicle-status information, vehicle-mission information, vehicle-metric information, and vehicle-environment information.

Claims (50)

1. A device for monitoring a plurality of self-driving vehicles, comprising:

a processor;

an external camera in communication with the processor and located remote from the plurality of self-driving vehicles;

a communications transceiver in communication with the processor for communicating with the plurality of self-driving vehicles;

a computer-readable medium in communication with the processor;

a display device in communication with the processor;

the computer-readable medium storing instructions that, when executed, configure the processor to:

receive an image of at least one unidentified self-driving vehicle of the plurality of self-driving vehicles, the image captured by the external camera;

identify the at least one unidentified self-driving vehicle in the image captured by the external camera;

determine a spatial location and orientation of the at least one identified self-driving vehicle relative to the external camera based on the image captured by the external camera;

request vehicle information associated with the at least one identified self-driving vehicle, and in response to the request, receive the vehicle information;

generate a graphic comprising the image captured by the external camera and a visual representation of the vehicle information, wherein the visual representation of the vehicle information is positioned and oriented within the graphic based on the spatial location and orientation of the at least one identified self-driving vehicle relative to the external camera; and

display the graphic on the display device.

2. The device of claim 1 , wherein the vehicle information comprises vehicle-status information.

3. The device of claim 2 , wherein the vehicle-status information comprises a vehicle operating state.

4. The device of claim 1 , wherein the vehicle information comprises vehicle-mission information.

5. The device of claim 4 , wherein the vehicle-mission information comprises a destination location.

6. The device of claim 1 , wherein the vehicle information comprises vehicle-environment information.

7. The device of claim 6 , wherein the vehicle-environment information comprises sensor scan data.

8. The device of claim 6 , wherein the vehicle-environment information comprises camera data.

9. The device of claim 1 , wherein the graphic further comprises at least one input button for receiving commands for controlling the movement of the at least one identified self-driving vehicle.

10. A method for monitoring a plurality of self-driving vehicles, comprising:

capturing an image of at least one unidentified self-driving vehicle of the plurality of self-driving vehicles using an external monitoring device in communication with the plurality of self-driving vehicles, the external monitoring device being located remote from the plurality of self-driving vehicles;

identifying the at least one unidentified self-driving vehicle in the image captured by the external monitoring device;

determining a spatial location and orientation of the at least one identified self-driving vehicle relative to the external monitoring device based on the image captured by the external monitoring device;

requesting vehicle information from associated with the at least one identified self-driving vehicle using the external monitoring device, and in response to the request, receiving the vehicle information;

generating a graphic comprising the image captured by the external monitoring device and a visual representation of the vehicle information, wherein the visual representation of the vehicle information is positioned and oriented within the graphic based on the spatial location and orientation of the at least one identified self-driving vehicle relative to the external monitoring device; and

displaying the graphic.

11. The method of claim 10 , wherein identifying the at least one unidentified self-driving vehicle in the image captured by the external monitoring device is based on at least one visual identifier tag on the at least one unidentified self-driving vehicle in the image captured by the external monitoring device.

12. The method of claim 11 , wherein the at least one identified self-driving vehicle is a first identified self-driving vehicle, the image is a first image, the vehicle information is first-vehicle information, further comprising:

capturing a second image of a second unidentified self-driving vehicle using the external monitoring device;

identifying the second unidentified self-driving vehicle in the second image captured by the external monitoring device; and

requesting second-vehicle information associated with the second identified self-driving vehicle using the external monitoring device, and in response to the request, receiving the second-vehicle information;

wherein the displayed graphic comprises the first image and the visual representation of the first vehicle information in association with the first image, and the second image and a visual representation of the second-vehicle information in association with the second image.

13. The method of claim 10 , wherein the vehicle information comprises vehicle-state information.

14. The method of claim 10 , wherein the vehicle information comprises vehicle-mission information.

15. The method of claim 10 , wherein the vehicle information comprises vehicle-environment information.

16. The method of claim 10 , wherein the graphic further comprises at least one input button for receiving commands for controlling the movement of the at least one identified self-driving vehicle.

17. A system for monitoring a plurality of self-driving vehicles, comprising:

a fleet-management system for communicating with the plurality of self-driving vehicles;

a display terminal in communication with the fleet-management system;

at least one external camera in communication with the display terminal, the at least one external camera being located remote from the plurality of self-driving vehicles;

the display terminal having a display device, a processor, and a computer-readable medium storing instructions that, when executed, configure the processor to:

receive an image of at least one unidentified self-driving vehicle of the plurality of self-driving vehicles, the image captured by the at least one external camera;

identify the at least one unidentified self-driving vehicle in the image captured by the at least one external camera;

determine a spatial location and orientation of the at least one identified self-driving vehicle relative to the at least one external camera based on the image captured by the at least one external camera;

request vehicle information associated with the at least one identified self-driving vehicle, and in response to the request, receive the vehicle information;

generate a graphic comprising the image captured by the at least one external camera and a visual representation of the vehicle information, wherein the visual representation of the vehicle information is positioned and oriented within the graphic based on the spatial location and orientation of the at least one identified self-driving vehicle relative to the at least one external camera; and

display the graphic on the display device.

18. The system of claim 17 , wherein the processor is configured, when the instructions are executed, to receive vehicle information associated with the at least one identified self-driving vehicle from the fleet-management system based on the identity of the at least one identified self-driving vehicle.

Assignments (4)
CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY'S NAME FROM CLEARPATH ROBOTICS, INC. TO CLEARPATH ROBOTICS INC. (WITHOUT THE COMMA) PREVIOUSLY RECORDED ON REEL 67944 FRAME 916. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jul 25, 2024
From: CLEARPATH ROBOTICS INC.
To: ROCKWELL AUTOMATION, INC.
Reel/Frame 068233/0542 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2024
From: CLEARPATH ROBOTICS, INC.
To: ROCKWELL AUTOMATION, INC.
Reel/Frame 067944/0916 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 27, 2024
From: ROCKWELL AUTOMATION, INC.
To: ROCKWELL AUTOMATION TECHNOLOGIES, INC.
Reel/Frame 067944/0982 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 23, 2021
From: TOD, ANTHONY; GARIEPY, RYAN CHRISTOPHER
To: CLEARPATH ROBOTICS INC.
Reel/Frame 055373/0465 →
Continuity (2)
Provisional Application 62537195 · Jul 26, 2017
Related Publication 20190250603A1 · Aug 15, 2019