IP Library Granted Patent US 10,759,446
Granted Patent B2
US 10,759,446 · App. 15/567,268 · Granted Sep 1, 2020

Information processing system, information processing method, and program

Inventors: Hideto Motomura (Kyoto, JP); Mohamed Sahim Kourkouss (Osaka, JP); Yoshihide Sawada (Tokyo, JP); Toshiya Mori (Osaka, JP); Masanaga Tsuji (Osaka, JP)
Assignee: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
B60W50/14B60K35/00B60R16/02B60W40/09B60W50/00B60W50/0097G05D1/0088G06K9/00791G06K9/00845G06K9/6274G06N3/0454G06N3/08G08G1/0112G08G1/0129G08G1/0141G08G1/0962G08G1/09626G08G1/096716G08G1/096725G08G1/096741G08G1/096775G08G1/167B60W2050/0002B60W2050/0075B60W2050/146B60W2420/42B60W2420/52B60W2520/10B60W2530/14B60W2540/10B60W2540/12B60W2540/20B60W2554/00B60W2554/80B60W2556/50G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,759,446
App. No.
15/567,268
Granted
Sep 1, 2020
Kind
B2
Abstract

An information processing system that appropriately estimates a driving conduct includes: a detector that detects a vehicle environment state, which is at least one of surroundings of a vehicle and a driving state of the vehicle; a behavior learning unit configured to cause a neural network to learn a relationship between the vehicle environment state detected by the detector and a behavior of the vehicle implemented after the vehicle environment state; and a behavior estimation unit configured to estimate a behavior of the vehicle by inputting, into the neural network that learned, the vehicle environment state detected at a current point in time by the detector.

Claims (59)

1. An information processing system, comprising:

a detector that detects a vehicle environment state, the vehicle environment state being at least one of surroundings of a vehicle and a driving state of the vehicle;

a behavior learning circuit configured to cause a neural network to learn a relationship between the vehicle environment state detected by the detector and a behavior of the vehicle implemented in response to the vehicle environment state; and

a behavior estimation circuit configured to estimate a behavior of the vehicle by inputting, into the neural network that learned, the vehicle environment state detected at a current point in time by the detector; and

an evaluation circuit configured to determine whether the behavior estimated by the behavior estimation circuit is valid or not, and output the behavior estimated when the behavior estimated is determined to be valid,

wherein the behavior learning circuit includes:

a general purpose behavior learning circuit configured to build a general purpose neural network by causing the neural network to learn, for each of a plurality of drivers, the relationship between the vehicle environment state detected by the detector and the behavior of the vehicle implemented in response to the vehicle environment state; and

a dedicated behavior learning circuit configured to build a dedicated neural network for a specific driver by transfer learning involving causing the general purpose neural network to relearn by using a vehicle environment state detected by the detector for the specific driver and a behavior of the vehicle implemented in response to the vehicle environment state detected by the detector for the specific driver when the vehicle is driven by the specific driver,

wherein the behavior estimation circuit includes:

a general purpose behavior estimation circuit configured to estimate a tentative behavior of the vehicle being driven by the specific driver by inputting the vehicle environment state detected by the detector for the specific driver into the general purpose neural network;

a dedicated behavior estimation circuit configured to estimate a behavior of the vehicle being driven by the specific driver by using the dedicated neural network,

wherein the information processing system further comprises:

a histogram generator that generates a histogram of an estimation result of the tentative behavior by the general purpose behavior estimation circuit, and the dedicated behavior learning circuit is configured to build the dedicated neural network by the transfer learning involving referring to the histogram generated;

wherein the vehicle is at least partially autonomous; and

wherein an autonomous operation of the vehicle is modified in response to the building of the dedicated neural network.

2. The information processing system according to claim 1 , further comprising:

an input circuit configured to receive a behavior of the vehicle input by the specific driver,

wherein the evaluation circuit

is configured to evaluate the behavior of the vehicle estimated by the behavior estimation circuit, based on the behavior of the vehicle received by the input circuit, and

when the evaluation circuit evaluates that there is an error in the behavior of the vehicle estimated by the behavior estimation circuit, the evaluation circuit is configured to cause the behavior learning circuit to cause the neural network to relearn by using the behavior of the vehicle received by the input circuit and the surroundings of the vehicle detected by the detector at a time of the estimation of the behavior of the vehicle.

3. The information processing system according to claim 1 , wherein

the dedicated behavior learning circuit

is configured to build, for each scene in which the vehicle is driven by the specific driver, the dedicated neural network for the specific driver in accordance with the scene, and select, from among a plurality of the dedicated neural networks, the dedicated neural network that is in accordance with a current scene in which the vehicle is driven by the specific driver, and

the dedicated behavior estimation circuit

is configured to estimate the behavior of the vehicle being driven by the specific driver by using the dedicated neural network selected.

4. The information processing system according to claim 1 , further comprising:

a notifier circuit that notifies a driver of the behavior estimated by the behavior estimation circuit before the behavior is implemented.

5. An information processing method, comprising:

detecting a vehicle environment state, the vehicle environment state being at least one of surroundings of a vehicle and a driving state of the vehicle;

causing a neural network to learn a relationship between the vehicle environment state detected and a behavior of the vehicle implemented in response to the vehicle environment state;

estimating a behavior of the vehicle by inputting, into the neural network that learned, the vehicle environment state detected at a current point in time; and

determining whether the behavior estimated is valid or not, and outputting the behavior estimated when the behavior estimated is determined to be valid,

wherein the causing of a neural network to learn includes:

building a general purpose neural network by causing the neural network to learn, for each of a plurality of drivers, the relationship between the vehicle environment state detected and the behavior of the vehicle implemented in response to the vehicle environment state; and

building a dedicated neural network for a specific driver by transfer learning involving causing the general purpose neural network to relearn by using the vehicle environment state detected for the specific driver and the behavior of the vehicle implemented in response to the vehicle environment state detected by the detector for the specific driver when the vehicle is driven by the specific driver,

wherein the estimating includes:

estimating a tentative behavior of the vehicle being driven by the specific driver by inputting the vehicle environment state detected for the specific driver into the general purpose neural network;

estimating a behavior of the vehicle being driven by the specific driver by using the dedicated neural network,

wherein the method further comprises:

generating a histogram of an estimation result of the tentative behavior, and in the building of the dedicated neural network, and

the dedicated neural network is built by the transfer learning involving referring to the histogram generated;

wherein the vehicle is at least partially autonomous; and

wherein an autonomous operation of the vehicle is modified in response to the building of the dedicated neural network.

6. A non-transitory computer-readable recording medium for use in a computer, the recording medium having a computer program recorded thereon for causing the computer to execute:

detecting a vehicle environment state, the vehicle environment state being at least one of surroundings of a vehicle and a driving state of the vehicle;

causing a neural network to learn a relationship between the vehicle environment state detected and a behavior of the vehicle implemented after the vehicle environment state;

estimating a behavior of the vehicle by inputting, into the neural network that learned, the vehicle environment state detected at a current point in time; and

determining whether the behavior estimated is valid or not, and outputting the behavior estimated when the behavior estimated is determined to be valid,

wherein the causing of a neural network to learn includes:

building a general purpose neural network by causing the neural network to learn, for each of a plurality of drivers, the relationship between the vehicle environment state detected and the behavior of the vehicle implemented after the vehicle environment state; and

building a dedicated neural network for a specific driver by transfer learning involving causing the general purpose neural network to relearn by using the vehicle environment state detected for the specific driver and the behavior of the vehicle implemented after the vehicle environment state detected by the detector for the specific driver when the vehicle is driven by the specific driver,

the estimating includes:

estimating a tentative behavior of the vehicle being driven by the specific driver by inputting the vehicle environment state detected for the specific driver into the general purpose neural network; and

estimating a behavior of the vehicle being driven by the specific driver by using the dedicated neural network,

the program further causes the computer to execute:

generating a histogram of an estimation result of the tentative behavior, and in the building of the dedicated neural network, and

the dedicated neural network is built by the transfer learning involving referring to the histogram generated,

wherein the vehicle is at least partially autonomous; and

wherein an autonomous operation of the vehicle is modified in response to the building of the dedicated neural network.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2024
From: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
To: PANASONIC AUTOMOTIVE SYSTEMS CO., LTD.
Reel/Frame 066703/0209 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2020
From: MOTOMURA, HIDETO; KOURKOUSS, MOHAMED SAHIM; SAWADA, YOSHIHIDE; MORI, TOSHIYA; TSUJI, MASANAGA
To: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Reel/Frame 052956/0518 →