IP Library Granted Patent US 11,537,124
Granted Patent B2
US 11,537,124 · App. 17/243,550 · Granted Dec 27, 2022

Built-in safety of control station and user interface for teleoperation

Inventors: Romain Clément (Mountain View, CA); Vinay Shet (Fremont, CA)
Assignee: Teleo, Inc.
G05D1/0038G06F3/0484G06F21/32H04N5/247H04N5/268H04N5/2624H04N5/272H04N7/0117H04N7/181
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,537,124
App. No.
17/243,550
Granted
Dec 27, 2022
Kind
B2
Abstract

A method and system may receive tiled video feed data sourced from one or more remotely situated vehicles. A teleoperator user interface is generated to include a concurrent display of a plurality of distinct video tiles from the tiled video feed data. A respective video tile displayed in the teleoperator user interface may include visual safety cues. A user interface segment that is displaying a respective video tile may be modified in response to teleoperator input.

Claims (57)

1. A system comprising:

one or more processors; and

a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:

receive tiled video feed data sourced from one or more remotely situated vehicles;

generate a teleoperator user interface, the teleoperator user interface including a concurrent display, within the teleoperator user interface, of a plurality of distinct video tiles from the tiled video feed data; and

modify a resolution of one or more segments of the teleoperator user interface (“user interface segments”) displaying respective video tiles in response to teleoperator input while the teleoperator user interface is displayed.

2. The system of claim 1 , wherein receive tiled video feed data sourced from one or more remotely situated vehicles comprises:

receive, in the tiled video feed data, camera view tiles, wherein each respective camera view tile comprises video data sourced from one of a plurality of cameras mounted on a particular remotely situated vehicle.

3. The system of claim 2 , wherein each respective camera view tile includes a pre-assigned graphical indicium overlay on the corresponding video data.

4. The system of claim 2 , wherein each pre-assigned graphical indicium overlay indicates one or more safety distances to be maintained by the particular remotely situated vehicle.

5. The system of claim 2 , wherein generate a teleoperator user interface to include a concurrent display of a plurality of distinct video tiles from the tiled video feed data comprises:

select a subset of the camera view tiles from the plurality of cameras mounted on the particular remotely situated vehicle; and

assign each select camera view tile in the subset for display in a distinct teleoperator user interface segment.

6. The system of claim 5 , further comprising:

while each select camera view tile is concurrently displayed in the teleoperator user interface:

replace a particular select camera view tile with a substitute camera view tile, the substitute camera view tile received in the tiled video feed data, the substitute camera view tile further including a pre-assigned graphical indicium overlay on corresponding video data.

7. The system of claim 6 , wherein the particular select camera view tile is sourced from a first camera and the substitute camera view tile is sourced from a second camera, wherein both the first and the second cameras are mounted on the same particular remotely situated vehicle and have a same directional field of vision.

8. The system of claim 5 , wherein modify a resolution of one or more user interface segments comprises:

receive teleoperator display resolution input;

based on the teleoperator display resolution input:

increase a display resolution of a first camera view tile displayed in a first user interface segment; and

decrease a display resolution of a second camera view tile displayed in a second user interface segment, wherein the first camera view tile is sourced from a first camera and the second camera view tile is sourced from a second camera, wherein both the first and the second cameras are mounted on the same particular remotely situated vehicle and have different directional field of visions.

9. The system of claim 1 , wherein the system further comprises:

determine presence of an authorized teleoperator based on validating a visual image of a portion of the operator's physical appearance; and

determine whether to lock or unlock one or more functionalities selectable by the authorized operator based on validating an extent of teleoperator attention represented in the visual image of a portion of the operator's physical appearance.

10. A computer-implemented method, comprising:

receiving tiled video feed data sourced from one or more remotely situated vehicles;

generating a teleoperator user interface, the teleoperator user interface including a concurrent display, within the teleoperator user interface, of a plurality of distinct video tiles from the tiled video feed data; and

modifying a resolution of one or more segments of the teleoperator user interface (“user interface segments”) displaying respective video tiles in response to teleoperator input while the teleoperator user interface is displayed.

11. The computer-implemented method of claim 10 , wherein receiving tiled video feed data sourced from one or more remotely situated vehicles comprises:

receiving, in the tiled video feed data, camera view tiles, wherein each respective camera view tile comprises video data sourced from one of a plurality of cameras mounted on a particular remotely situated vehicle.

12. The computer-implemented method of claim 11 , wherein each respective camera view tile includes a pre-assigned graphical indicium overlay on the corresponding video data.

13. The computer-implemented method of claim 11 , wherein each pre-assigned graphical indicium overlay indicates one or more safety distances to be maintained by the particular remotely situated vehicle.

14. The computer-implemented method of claim 11 , wherein generating a teleoperator user interface to include a concurrent display of a plurality of distinct video tiles from the tiled video feed data comprises:

selecting a subset of the camera view tiles from the plurality of cameras mounted on the particular remotely situated vehicle; and

assigning each select camera view tile in the subset for display in a distinct teleoperator user interface segment.

15. The computer-implemented method of claim 14 , further comprising:

while each select camera view tile is concurrently displayed in the teleoperator user interface:

replacing a particular select camera view tile with a substitute camera view tile, the substitute camera view tile received in the tiled video feed data, the substitute camera view tile further including a pre-assigned graphical indicium overlay on corresponding video data.

16. The computer-implemented method of claim 15 , wherein the particular select camera view tile is sourced from a first camera and the substitute camera view tile is sourced from a second camera, wherein both the first and the second cameras are mounted on the same particular remotely situated vehicle and have a same directional field of vision.

17. The computer-implemented method of claim 14 , wherein modifying a resolution of one or more user interface segments comprises:

receiving teleoperator display resolution input;

based on the teleoperator display resolution input:

increasing a display resolution of a first camera view tile displayed in a first user interface segment; and

decreasing a display resolution of a second camera view tile displayed in a second user interface segment, wherein the first camera view tile is sourced from a first camera and the second camera view tile is sourced from a second camera, wherein both the first and the second cameras are mounted on the same particular remotely situated vehicle and have different directional field of visions.

18. The computer-implemented method of claim 10 , further comprising:

determining presence of an authorized teleoperator based on validating a visual image of a portion of the operator's physical appearance; and

determining whether to lock or unlock one or more functionalities selectable by the authorized operator based on validating an extent of teleoperator attention represented in the visual image of a portion of the operator's physical appearance.

19. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:

receive tiled video feed data sourced from one or more remotely situated vehicles;

generate a teleoperator user interface, the teleoperator user interface including a concurrent display, within the teleoperator user interface, of a plurality of distinct video tiles from the tiled video feed data; and

modify a resolution of one or more segments of the teleoperator user interface (“user interface segments”) displaying respective video tiles in response to teleoperator input while the teleoperator user interface is displayed,

wherein receive tiled video feed data sourced from one or more remotely situated vehicles comprises:

receive, in the tiled video feed data, camera view tiles;

wherein each respective camera view tile comprises video data sourced from one of a plurality of cameras mounted on a particular remotely situated vehicle;

wherein each respective camera view tile includes a pre-assigned graphical indicium overlay on the corresponding video data; and

wherein each pre-assigned graphical indicium overlay indicates one or more safety distances to be maintained by the particular remotely situated vehicle.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Jul 6, 2026
From: HAVOCAI INC.
To: TELEO LLC
Reel/Frame 075182/0319 →
MERGER AND CHANGE OF NAME Recorded Jun 30, 2026
From: TELEO INC.; TELEO SURVIVOR, LLC
To: TELEO LLC
Reel/Frame 075138/0855 →
SECURITY INTEREST Recorded Dec 23, 2025
From: TELEO INC.
To: HAVOCAI INC.
Reel/Frame 073305/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2021
From: CLÉMENT, ROMAIN; SHET, VINAY
To: TELEO, INC.
Reel/Frame 056076/0561 →
Continuity (2)
Provisional Application 63017605 · Apr 29, 2020
Related Publication 20210344876A1 · Nov 4, 2021