IP Library › Granted Patent US 12,731,321
Granted Patent B2
US 12,731,321 · App. 18/550,816 · Granted Sep 8, 2026

Information processing device and information processing method

Inventor: Takehito Teraguchi (Kanagawa, JP)
Assignees: Nissan Motor Co., Ltd.; RENAULT S.A.S.
G06T15/00B60W50/14G06T5/50G06V20/56H04N13/117G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,731,321
App. No.
18/550,816
Granted
Sep 8, 2026
Kind
B2
Abstract

The present invention acquires surrounding information of surrounding environment of a vehicle; sets a virtual viewpoint position ahead of the vehicle in a traveling direction; uses the acquired surrounding information and generates virtual surrounding information indicating the surrounding environment at the set virtual viewpoint position; transmits a first control signal for controlling an external device located at a location away from the vehicle to display the generated virtual surrounding information; receives driving operation information on a driving operation of the vehicle output from the external device; generates driving assistance information for assisting driving of the vehicle based on the received driving operation information; and generates a second control signal for controlling an output device installed in the vehicle to output the driving assistance information.

Claims (46)

1 . An information processing device configured to:

acquire surrounding information about the surrounding environment of a vehicle;

set a virtual viewpoint position ahead of the vehicle in a traveling direction, the virtual viewpoint position being a position of a virtual vehicle traveling ahead of the vehicle in the surrounding environment;

use the acquired surrounding information and generate virtual surrounding information indicating the surrounding environment at the set virtual viewpoint position;

transmit a first control signal for controlling an external device located at a location away from the vehicle to output the generated virtual surrounding information;

receive, from the external device, movement instruction information for moving the virtual viewpoint position as driving operation information on a driving operation of the vehicle input by a second user;

move the virtual viewpoint position according to the movement instruction information;

generate driving assistance information for assisting a first user who manually drives the vehicle based on the received driving operation information, and generate a second control signal for controlling an output device in the vehicle to output the driving assistance information, the output device being a device for outputting image information to the first user; and

transmit the second control signal to the output device,

wherein the second control signal is a control signal for controlling the output device to display a virtual vehicle image representing the virtual vehicle so as to be superimposed at the virtual viewpoint position on a real scenery of the surrounding environment of the vehicle.

2 . The information processing device according to claim 1 , further configured to:

acquire a surrounding image of the surrounding environment of the vehicle,

use the acquired surrounding image and generate a virtual surrounding image representing the surrounding environment at the set virtual viewpoint position, and

transmit, as the first control signal, a control signal for controlling the external device to display the generated virtual surrounding image.

3 . The information processing device according to claim 2 , further configured to:

acquire position information indicating a position of the vehicle;

calculate the virtual viewpoint position in the surrounding environment of the vehicle with the position of the vehicle as a reference; and

use the surrounding image and generate the virtual surrounding image based on the virtual viewpoint position.

4 . The information processing device according to claim 2 , further configured to:

acquire the surrounding image from at least one of sensor information acquired by the vehicle, sensor information acquired by another vehicle, and sensor information acquired by an infrastructure sensor installed along a traveling road.

5 . The information processing device according to claim 1 , further configured to:

acquire position information indicating a position of the vehicle; and

transmit a third control signal for controlling the external device to display the position information of the vehicle and position information of the virtual viewpoint position.

6 . The information processing device according to claim 1 , further configured to:

receive, as the driving operation information, device control information for controlling an in-vehicle device of the virtual vehicle; and

generate, based on the device control information, the virtual vehicle image representing the virtual vehicle in which the in-vehicle device is controlled.

7 . The information processing device according to claim 1 , further configured to:

determine whether or not a distance from the vehicle to the virtual viewpoint position is equal to or greater than a predetermined distance; and

when determining that the distance is equal to or greater than the predetermined distance, reset the virtual viewpoint position to a position at which the distance is shorter than the predetermined distance.

8 . The information processing device according to claim 1 , further configured to:

receive, as the driving operation information, sound information regarding driving guidance for the vehicle,

generate a sound control signal for controlling the output device to output the sound information; and

transmit the sound control signal to the output device.

9 . The information processing device according to claim 1 , further configured to:

receive the driving operation information regarding a driving operation at a time at which the vehicle reaches the virtual viewpoint position.

10 . An information processing method comprising:

acquiring surrounding information of surrounding environment of a vehicle;

setting a virtual viewpoint position ahead of the vehicle in a traveling direction, the virtual viewpoint position being a position of a virtual vehicle traveling ahead of the vehicle in the surrounding environment;

using the acquired surrounding information and generating virtual surrounding information indicating the surrounding environment at the set virtual viewpoint position;

transmitting a first control signal for controlling an external device located at a location away from the vehicle to display the generated virtual surrounding information;

receiving, from the external device, movement instruction information for moving the virtual viewpoint position as driving operation information on a driving operation of the vehicle input by a second user;

moving the virtual viewpoint position according to the movement instruction information;

generating driving assistance information for assisting a first user who manually drives the vehicle based on the received driving operation information; and

generating a second control signal for controlling an output device in the vehicle to output the driving assistance information, the output device being a device for outputting image information to the first user; and

transmitting the second control signal to the output device,

wherein the second control signal is a control signal for controlling the output device to display a virtual vehicle image representing the virtual vehicle so as to be superimposed at the virtual viewpoint position on a real scenery of the surrounding environment of the vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2023
From: TERAGUCHI, TAKEHITO
To: NISSAN MOTOR CO., LTD.; RENAULT S.A.S.
Reel/Frame 065135/0155 →
Continuity (1)
Related Publication 20240185499A1 · Jun 6, 2024
References Cited (7)
US 20190066382A1 · Kubo et al. · 2019 [cited by applicant]
US 20190361436A1 · Ueda · 2019 [cited by examiner]
US 20200031283A1 · Nakasho · 2020 [cited by examiner]
US 20200133259A1 · Van Wiemeersch · 2020 [cited by examiner]
US 20220044501A1 · Lavoie · 2022 [cited by examiner]
JP 2019184356A · 2019 [cited by applicant]
JP 2020003890A · 2020 [cited by applicant]