Behavior estimation device, behavior estimation method, and recording medium
In the behavior estimation device, a person feature extraction means extracts a feature of a person detected from a plurality of images in a time series. An object feature extraction means extracts a feature of an object detected from the plurality of images. A peripheral feature extraction means extracts a feature of a periphery of the person in the plurality of images. A feature aggregation means executes aggregation processing for aggregating the feature of the person, the feature of the object, and the feature of the periphery of the person. A behavior estimation processing means executes estimation processing for estimating the person's behavior included in the plurality of images based on information including a processing result of the aggregation processing.
1 . A behavior estimation device comprising:
a memory configured to store instructions; and
one or more processors configured to execute the instructions to:
extract a first feature quantity corresponding to at least one person area detected from a plurality of images in a time series using a neural network;
extract a second feature quantity corresponding of an object area detected from the plurality of images using a neural network, wherein the object area is different from the person area;
extract a third feature quantity corresponding to a background area in the plurality of images using a neural network, wherein the background area is different from the person area and the object area;
acquire a fourth feature quantity by weighted addition of the first feature quantity and the third feature quantity;
acquire a fifth feature quantity by weighted addition of the second feature quantity and the fourth feature quantity;
acquire a sixth feature quantity by weighted addition of the third feature quantity and the fifth feature quantity;
acquire a first estimation result of the behavior of a person included in the person area by applying the fourth feature quantity to a first estimation model;
acquire a second estimation result of the behavior of the person by applying the fifth feature quantity to a second estimation model;
acquire a third estimation result of the behavior of the person by applying the sixth feature quantity to a third estimation model; and
acquire a final estimation result of the behavior of the person based on the first estimation result, the second estimation result, and the third estimation result.
2 . The behavior estimation device according to claim 1 , wherein the one or more processors acquire the values according to weights used in the weighted addition in acquiring the fifth feature as relevance degree information that is information indicating a relevance degree of an object included in the object area to the behavior of a person included in the person area, and set weights to be used in the weighted addition when acquiring the sixth feature using the acquired relevance degree information.
3 . A behavior estimation method comprising:
extracting a first feature quantity corresponding to at least one person area detected from a plurality of images in a time series using a neural network;
extracting a second feature quantity corresponding to an object area detected from the plurality of images using a neural network, wherein the object area is different from the person area;
extracting a third feature quantity corresponding to a background area in the plurality of images using a neural network, wherein the background area is different from the person area and the object area;
acquiring a fourth feature quantity by weighted addition of the first feature quantity and the third feature quantity;
acquiring a fifth feature quantity by weighted addition of the second feature quantity and the fourth feature quantity;
acquiring a sixth feature quantity by weighted addition of the third feature quantity and the fifth feature quantity;
acquiring a first estimation result of the behavior of a person included in the person area by applying the fourth feature quantity to a first estimation model;
acquiring a second estimation result of the behavior of the person by applying the fifth feature quantity to a second estimation model;
acquiring a third estimation result of the behavior of the person by applying the sixth feature quantity to a third estimation model; and
acquiring a final estimation result of the behavior of the person based on the first estimation result, the second estimation result, and the third estimation result.
4 . A non-transitory recording medium recording a program, the program causing a computer to execute the behavior estimation method according to claim 3 .