IP Library Granted Patent US 11,675,349
Granted Patent B2
US 11,675,349 · App. 17/107,570 · Granted Jun 13, 2023

User interface for remote vehicle monitoring

Inventors: Auver Cedric Austria (San Francisco, CA); Timothy David Kentley-Klay (Stanford, CA); Jannie Wing Sze Lai (Berkeley, CA); Frank Reinaldo Ramirez (San Francisco, CA); Dennis Schaefer (San Mateo, CA)
Assignee: Zoox, Inc.
G05D1/0016G05D1/005G05D1/0022G05D1/0027G05D1/0038
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,675,349
App. No.
17/107,570
Granted
Jun 13, 2023
Kind
B2
Abstract

Techniques for providing a user interface for remote vehicle monitoring and/or control include presenting a digital representation of an environment and a vehicle as it traverses the environment on a first portion of a display and presenting on a second portion of the display a communication interface that is configured to provide communication with multiple people. The communication interface may enable communications between a remote operator and any number of occupants of the vehicle, other operators (e.g., other remote operators or in-vehicle operators), and/or people in an environment around the vehicle. The user interface may additionally include controls to adjust components of the vehicle, and the controls may be presented on a third portion of the display. Furthermore, the user interface may include a vehicle status interface that provides information associated with a current state of the vehicle.

Claims (77)

1. A system comprising:

one or more processors; and

one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:

presenting, on a display, a user interface that is configured to provide a remote operator with information associated with a fleet of vehicles, the user interface comprising:

a representation of an environment and a vehicle of the fleet of vehicles as it traverses the environment; and

a communication interface configured to provide the remote operator communication with occupants in different seating locations of the vehicle and another remote operator;

receiving, via a first input element of the communication interface, first data indicating that a first chat interface of the communication interface is to be opened, the first chat interface being associated with a first seating location of the different seating locations of the vehicle, the first seating location occupied by a first occupant;

causing presentation of the first chat interface on the display based at least in part on receiving the first data;

receiving, via a second input element of the communication interface, second data representing a first message that is to be sent to the first occupant; and

based at least in part on receiving an indication that the first message is to be sent, causing the first message to be sent and presenting the first message on the display within the first chat interface.

2. The system of claim 1 , the operations further comprising:

receiving, via a third input element of the communication interface, third data indicating that a second chat interface is to be opened, the second chat interface being associated with the other remote operator;

based at least in part on receiving the third data, causing presentation of the second chat interface on the display while suppressing presentation of the first chat interface;

receiving, via the second input element of the communication interface, fourth data representing a second message that is to be sent to the other remote operator; and

presenting the second message on the display within the second chat interface based at least in part on receiving an indication that the second message is to be sent.

3. The system of claim 1 , wherein the user interface further comprises a vehicle control interface that provides control of a component or system of the vehicle, the operations further comprising:

receiving, via a first input element of the vehicle control interface, third data indicating that the component or system of the vehicle is to be controlled; and

transmitting a signal to the vehicle, the signal configured to cause the vehicle to control the component in accordance with the first input element of the vehicle control interface.

4. The system of claim 1 , the operations further comprising:

receiving, via a network, a request to cause at least one of the vehicle or a component of the vehicle to perform an action; and

receiving, via an input element of the user interface, third data to cause the vehicle or the component to perform the action.

5. A method comprising:

presenting, on a first portion of a display, a digital representation of an environment and a vehicle as it traverses the environment;

presenting, on a second portion of the display, a communication interface that is configured to provide communication with different occupants of the vehicle seated in different seating locations of the vehicle;

receiving, via a first input element of the communication interface, first data indicating that a first chat interface is to be opened, wherein the first chat interface is associated with a first occupant and a first seating location;

presenting the first chat interface on the display based at least in part on receiving the first data;

receiving, via a second input element of the communication interface, second data representing a first message that is to be sent; and

based at least in part on receiving an indication that the first message is to be sent, causing the first message to be sent and presenting the first message on the display within the first chat interface.

6. The method of claim 5 , further comprising:

presenting, within the communication interface, a third input element that is configured to receive user inputs indicating whether the first message is to be communicated audibly;

receiving, via the third input element, third data indicating that the first message is to be communicated audibly; and

wherein causing the first message to be sent comprises causing the first message to be communicated audibly within an interior of the vehicle proximate the first seating location.

7. The method of claim 5 , wherein the communication interface is further configured to provide communication with a pedestrian that is proximate an exterior of the vehicle, the method further comprising:

presenting, within the communication interface, a third input element that is configured to receive user inputs indicating whether the first message is to be communicated audibly;

receiving, via the third input element, third data indicating that the first message is to be communicated audibly; and

wherein causing the first message to be sent comprises causing the first message to be communicated audibly via speakers disposed on an exterior surface of the vehicle.

8. The method of claim 5 , wherein the second data comprises a string of text representing the first message, the method further comprising receiving, via a third input element, third data indicating that the first message is to be communicated audibly.

9. The method of claim 5 , wherein the second data comprises audio data representing the first message, the method further comprising receiving, via a third input element, third data indicating that the first message is to be communicated audibly.

10. The method of claim 5 , further comprising:

receiving, via a third input element of the communication interface, third data indicating that a second chat interface is to be opened, wherein the second chat interface is associated with a second occupant and a second seating location;

presenting the second chat interface on the display based at least in part on receiving the third data;

receiving, via the second input element of the communication interface, fourth data representing a second message that is to be sent; and

presenting the second message on the display within the second chat interface based at least in part on receiving an indication that the second message is to be sent.

11. The method of claim 5 , wherein the communication interface is further configured to provide communication with a remote operator of the vehicle.

12. The method of claim 5 , wherein the first chat interface is further associated with a second occupant and a second seating location, the method further comprising:

receiving, via a network, third data representing a first response to the first message from the first occupant;

presenting, on the display and within the first chat interface, the first response including a first identifier that is associated with the first occupant;

receiving, via the network, fourth data representing a second response to at least one of the first message or the first response from the second occupant; and

presenting, on the display and within the first chat interface, the second response including a second identifier that is associated with the second occupant.

13. The method of claim 5 , further comprising:

receiving, from a device associated with at least one of the first occupant or the first seating location, a request to cause at least one of the vehicle or a component of the vehicle to perform an action; and

receiving an input to cause the vehicle or the component to perform the action.

14. The method of claim 5 , further comprising presenting, on a third portion of the display, a vehicle status interface to provide information associated with a current state of the vehicle, the information including one or more of a speed of the vehicle, a direction of travel of the vehicle, or a traffic rule associated with a current location of the vehicle.

15. The method of claim 5 , further comprising presenting, on a third portion of the display, a vehicle control interface that provides control of a component of the vehicle, the vehicle control interface including at least a first input element that is associated with a first component of the vehicle.

16. The method of claim 15 , wherein the first component of the vehicle comprises at least one of a door of the vehicle, a climate control system of the vehicle, or a stereo system of the vehicle, the method further comprising:

receiving, via a first input element of the vehicle control interface, third data indicating that the first component is to be controlled; and

transmitting a signal to the vehicle, the signal configured to cause the vehicle to control the component in accordance with the first input element of the vehicle control interface.

17. A non-transitory computer-readable storage medium storing instructions thereon that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:

presenting, on a first portion of a display, a representation of a vehicle as it traverses an environment;

presenting, on a second portion of the display, a communication interface that is configured to provide communication with a different occupants of the vehicle seated in different seating locations of the vehicle;

receiving, via a first input element of the communication interface, first data indicating that a first chat interface is to be opened, wherein the first chat interface is associated with a first occupant and a first seating location;

presenting the first chat interface on the display based at least in part on receiving the first data;

receiving, via a second input element of the communication interface, second data representing a message that is to be sent; and

based at least in part on receiving an indication that the message is to be sent, causing the message to be sent and presenting the message on the display within the first chat interface.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the communication interface is further configured to provide communication with a pedestrian that is proximate an exterior of the vehicle, the operations further comprising:

presenting, within the communication interface, a third input element that is configured to receive user inputs indicating whether the message is to be communicated audibly; and

receiving, via the third input element, third data indicating that the message is to be communicated audibly to the pedestrian.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the first chat interface is a first chat interface that is associated with at least one of the first occupant or the first seating location, the operations further comprising:

receiving, via a third input element of the communication interface, third data indicating that a second chat interface is to be opened, wherein the second chat interface is associated with a remote operator associated with the vehicle;

presenting the second chat interface on the display while suppressing the first chat interface based at least in part on receiving the third data;

receiving, via the second input element of the communication interface, fourth data representing a second message that is to be sent; and

presenting the second message on the display within the second chat interface based at least in part on receiving an indication that the second message is to be sent.

20. The non-transitory computer-readable storage medium of claim 17 , the operations further comprising:

receiving, from a first device associated with the first occupant, third data representing a first response to the message;

presenting, on the display and within the first chat interface, the first response including a first identifier that is associated with the first occupant;

receiving, from a second device associated with a second occupant, fourth data representing a second response to at least one of the message or the first response; and

presenting, on the display and within the first chat interface, the second response including a second identifier that is associated with the second occupant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2021
From: AUSTRIA, AUVER CEDRIC; KENTLEY-KLAY, TIMOTHY DAVID; LAI, JANNIE WING SZE; RAMIREZ, FRANK REINALDO; SCHAEFER, DENNIS
To: ZOOX, INC.
Reel/Frame 057232/0235 →
Continuity (1)
Related Publication 20220171382A1 · Jun 2, 2022
Cited By (2)
US 1,068,796 US 1,112,281