IP Library › Granted Patent US 12,243,320
Granted Patent B2
US 12,243,320 · App. 17/442,075 · Granted Mar 4, 2025

Information processing method, program, and information processing apparatus

Inventor: Haruyoshi Yonekawa (Kanagawa, JP)
Assignee: Sony Semiconductor Solutions Corporation
G06V20/56G01S13/89G06N3/08G06V10/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,243,320
App. No.
17/442,075
Granted
Mar 4, 2025
Kind
B2
Abstract

The present technology relates to an information processing method, a program, and an information processing apparatus capable of analyzing a learning situation of a model using a neural network. In step S 1 , feature data that numerically represents a feature of a feature map generated from input data in a model using a neural network is generated. In step S 2 , analysis data based on the feature data of a plurality of the feature maps is generated. The present technology can be applied to, for example, a system that recognizes a vehicle in front of a vehicle.

Claims (37)

1. An information processing method executed by processing circuitry, the method comprising:

generating feature data that numerically represents a feature of a feature map generated from input data in a model using a neural network;

generating analysis data based on the feature data of a plurality of feature maps; and

displaying the generated analysis data as feature data, based on feature maps generated from a same input data, arranged in numerical order of the feature maps in a first axis direction and feature data, based on feature maps corresponding to different input data, arranged in numerical order of the different input data in a second axis direction orthogonal to the first axis direction, wherein the generated analysis data is displayed as an image in which pixels represent values of the feature data of a given input data and a given feature map, and the image is indicative of a state of learning of the model using the neural network.

2. The information processing method according to claim 1 ,

wherein in the analysis data, the feature data of a plurality of the feature maps generated from a plurality of the input data is arranged in a predetermined hierarchy of the model that generates a plurality of the feature maps from one of the input data.

3. The information processing method according to claim 1 ,

wherein the feature data indicates a degree of dispersion of pixel values of the feature map.

4. The information processing method according to claim 3 , further comprising:

analyzing a learning situation of the model on a basis of a line in the first direction and a line in the second direction of the analysis data.

5. The information processing method according to claim 4 , further comprising:

setting a parameter for learning of the model on a basis of an analysis result of the learning situation of the model.

6. The information processing method according to claim 5 ,

wherein in the parameter setting, a regularization parameter for learning of the model is set on a basis of the number of lines in the second direction of the analysis data.

7. The information processing method according to claim 2 ,

wherein the feature data indicates a frequency distribution of pixel values of the feature map, and

in the analysis data, the feature data of a plurality of the feature maps generated from a plurality of the input data is three-dimensionally arranged in the hierarchy of the model.

8. The information processing method according to claim 1 ,

wherein the input data is image data, and

the model performs recognition processing of an object.

9. The information processing method according to claim 8 ,

wherein the model performs recognition processing of a vehicle.

10. The information processing method according to claim 9 ,

wherein the input data is image data representing a distribution of intensity of a received signal of a millimeter wave radar in a bird's-eye view.

11. The information processing method according to claim 10 ,

wherein the model converts the image data into an image in a camera coordinate system.

12. The information processing method according to claim 1 ,

wherein the analysis data includes the feature data satisfying a predetermined condition among the feature data of a plurality of the feature maps.

13. A non-transitory computer readable medium storing instructions that, when executed by processing circuitry, perform an information processing method comprising:

generating feature data that numerically represents a feature of a feature map generated from input data in a model using a neural network;

generating analysis data based on the feature data of a plurality of feature maps; and

displaying the generated analysis data as feature data, based on feature maps generated from a same input data, arranged in numerical order of the feature maps in a first axis direction and feature data, based on feature maps corresponding to different input data, arranged in numerical order of the different input data in a second axis direction orthogonal to the first axis direction, wherein the generated analysis data is displayed as an image in which pixels represent values of the feature data of a given input data and a given feature map, and the image is indicative of a state of learning of the model using the neural network.

14. An information processing apparatus comprising:

processing circuitry configured to:

generate feature data that numerically represents a feature of a feature map generated from input data in a model using a neural network;

generate analysis data based on the feature data of a plurality of feature maps; and

display the generated analysis data as feature data, based on feature maps generated from a same input data, arranged in numerical order of the feature maps in a first axis direction and feature data, based on feature maps corresponding to different input data, arranged in numerical order of the different input data in a second axis direction orthogonal to the first axis direction, wherein the generated analysis data is displayed as an image in which pixels represent values of the feature data of a given input data and a given feature map, and the image is indicative of a state of learning of the model using the neural network.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2022
From: YONEKAWA, HARUYOSHI
To: SONY SEMICONDUCTOR SOLUTIONS CORPORATION
Reel/Frame 060515/0905 →
Priority Claims (1)
JP 2019-065378 · Mar 29, 2019 · national
Continuity (1)
Related Publication 20220172484A1 · Jun 2, 2022
References Cited (20)
US 20160321540A1 · Towal · 2016 [cited by applicant]
US 20170169313A1 · Choi · 2017 [cited by examiner]
US 20180365794A1 · Lee · 2018 [cited by examiner]
US 20190026588A1 · Ryan · 2019 [cited by applicant]
US 20200211219A1 · Yamazaki · 2020 [cited by examiner]
US 20210232848A1 · Wang · 2021 [cited by examiner]
EP 3289528A · 2018 [cited by applicant]
EP 3627931A1 · 2020 [cited by applicant]
JP H0561976 · 1993 [cited by applicant]
JP 2018520404 · 2018 [cited by applicant]
WO WO2017192889A1 · 2017 [cited by applicant]
WO WO2018165753A1 · 2018 [cited by applicant]
WO WO2018212606A1 · 2018 [cited by applicant]
WO WO2018230832A1 · 2018 [cited by applicant]
Written Opinion and English translation thereof mailed Jun. 9, 2020 in connection with International Application No. PCT/JP2020/011601. [cited by applicant]
International Preliminary Report on Patentability and English translation thereof mailed Oct. 14, 2021 in connection with International Application No. PCT/JP2020/011601. [cited by applicant]
Extended European Search Report issued Apr. 4, 2022 in connection with European Application No. 20784531.4. [cited by applicant]
XP055566692, Matthew D. Zeiler et al: “Visualizing and Understanding Convolutional Networks”, in: “Advances in Databases and Information Systems”, Jan. 1, 2014, Springer International Publishing, Cham 032682, ISBN: 978-… [cited by applicant]
International Search Report and English translation thereof mailed Jun. 9, 2020 in connection with International Application No. PCT/JP2020/011601. [cited by applicant]
Zeiler et al., Visualizing and Understanding Convolutional Networks. European Conference on Computer Vision (ECCV). 2014. p. 818-833. [cited by applicant]