IP Library Granted Patent US 12,307,787
Granted Patent B2
US 12,307,787 · App. 18/036,354 · Granted May 20, 2025

State determination device, state determination method and recording medium

Inventors: Nana Jumonji (Tokyo, JP); Gaku Nakano (Tokyo, JP); Chisato Sugawara (Tokyo, JP); Yosuke Kimura (Tokyo, JP)
Assignee: NEC CORPORATION
G06V20/588B60W40/06
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,307,787
App. No.
18/036,354
Granted
May 20, 2025
Kind
B2
Abstract

A state determination device according to the present invention includes: a memory; and at least one processor coupled to the memory. The processor performs operations. The operations including: combining output from a plurality of determination models for an input image of a road, the plurality of determination models each learned using training data in which at least one of a state of a moving object mounting an imaging device that acquires an image of the road, a state of the road, and an external environment is different.

Claims (36)

1. A state determination device comprising:

a memory storing instructions; and

at least one processor configured to execute the instructions to perform operations comprising:

determining weights for combining outputs of a plurality of determination models respectively determined by the plurality of determination models using a same input image of a target road, each of the plurality of determination models having been trained using training data in which at least one of a state of a moving object on which an imaging device that is to acquire an image of the road is mounted, a state of the road in the acquired image, and an external environment in which the image of the road has been acquired by the imaging device is different than that in the training data used to train one or more other of the plurality of determination models; and

combining the outputs of the plurality of determination models for the same input image of the target road using the determined weights.

2. The state determination device according to claim 1 , wherein the operations further comprise:

determining and outputting a state of the target road based on a result of combining the outputs of the plurality of determination models for the same input image.

3. The state determination device according to claim 1 , wherein combining the outputs of the plurality of determination models for the image of the target road using the determined weights comprises respectively multiplying outputs of the plurality of determination models by the determined weights.

4. The state determination device according to claim 3 , wherein the operations further comprise:

learning a parameter for determining the weights using training data for learning the weights.

5. The state determination device according to claim 4 , wherein learning the parameter for determining the eights using the training data for learning the weights comprises:

learning the parameter for determining the weights in accordance with the external environment.

6. The state determination device according to claim 1 , wherein the plurality of determination models respectively determine the outputs as a state of deterioration of the road and structures related to the road.

7. A state determination method performed by a computer and comprising:

determining weights for combining outputs of a plurality of determination models respectively determined by the plurality of determination models using a same input image of a target road, each of the plurality of determination models having been trained using training data in which at least one of a state of a moving object on which an imaging device that is to acquire an image of the road is mounted, a state of the road in the acquired image, and an external environment in which the image of the road has been acquired by the imaging device is different than that in the training data used to train one or more other of the plurality of determination models; and

combining the outputs of the plurality of determination models for the same input image of the target road using the determined weights.

8. The state determination method according to claim 7 , further comprising:

determining and outputting a state of the target road based on a result of combining the outputs of the plurality of determination models for the same input image.

9. The state determination method according to claim 7 , wherein combining the outputs of the plurality of determination models for the image of the target road using the determined weights comprises respectively multiplying outputs of the plurality of determination models by the determined weights.

10. The state determination method according to claim 9 , further comprising:

learning a parameter for determining the weights using training data for learning the weights.

11. The state determination method according to claim 10 , wherein learning the parameter for determining the eights using the training data for learning the weights comprises:

learning the parameter for determining the weights in accordance with the external environment.

12. The state determination method according to claim 7 , wherein the plurality of determination models respectively determine the outputs as a state of deterioration of the road and structures related to the road.

13. A non-transitory computer-readable recording medium storing a program executable by a computer to perform a method comprising:

determining weights for combining outputs of a plurality of determination models respectively determined by the plurality of determination models using a same input image of a target road, each of the plurality of determination models having been trained using training data in which at least one of a state of a moving object on which an imaging device that is to acquire an image of the road is mounted, a state of the road in the acquired image, and an external environment in which the image of the road has been acquired by the imaging device is different than that in the training data used to train one or more other of the plurality of determination models; and

combining the outputs of the plurality of determination models for the same input image of the target road using the determined weights.

14. The non-transitory computer-readable recording medium according to claim 13 , wherein the method further comprises:

determining and outputting a state of the target road based on a result of combining the outputs of the plurality of determination models for the same input image.

15. The non-transitory computer-readable recording medium according to claim 13 , wherein combining the outputs of the plurality of determination models for the image of the target road using the determined weights comprises respectively multiplying outputs of the plurality of determination models by the determined weights.

16. The non-transitory computer-readable recording medium according to claim 15 , wherein the method further comprises:

learning a parameter for determining the weights using training data for learning the weights.

17. The non-transitory computer-readable recording medium according to claim 16 , wherein learning the parameter for determining the eights using the training data for learning the weights comprises:

learning the parameter for determining the weights in accordance with the external environment.

18. The non-transitory computer-readable recording medium according to claim 13 ,

wherein the plurality of determination models respectively determine the outputs as a state of deterioration of the road and structures related to the road.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2023
From: JUMONJI, NANA; NAKANO, GAKU; SUGAWARA, CHISATO; KIMURA, YOSUKE
To: NEC CORPORATION
Reel/Frame 063601/0001 →
Continuity (1)
Related Publication 20230419687A1 · Dec 28, 2023
References Cited (18)
US 8666562B2 · Tuononen · 2014 [cited by examiner]
US 10373000B2 · Tong · 2019 [cited by examiner]
US 11467596B2 · Kamata · 2022 [cited by examiner]
US 20190047527A1 · Falconer · 2019 [cited by examiner]
US 20200257925A1 · Kuwajima · 2020 [cited by examiner]
US 20200311444A1 · Oishi · 2020 [cited by examiner]
US 20210300401A1 · Hashimoto · 2021 [cited by examiner]
US 20220253065A1 · Toyoura · 2022 [cited by examiner]
US 20240183117A1 · Kim · 2024 [cited by examiner]
JP H04328669A · 1992 [cited by applicant]
JP 2007136430A · 2007 [cited by applicant]
JP 2009282783A · 2009 [cited by applicant]
JP 2018147261A · 2018 [cited by applicant]
JP 2019079166A · 2019 [cited by applicant]
JP 2020013537A · 2020 [cited by applicant]
JP 2020147961A · 2020 [cited by applicant]
International Search Report for PCT Application No. PCT/JP2020/043471, mailed on Jan. 12, 2021. [cited by applicant]
English translation of Written opinion for PCT Application No. PCT/JP2020/043471, mailed on Jan. 12, 2021. [cited by applicant]
Cited By (1)
US 12,662,159