Granted Patent B2

US 12,367,664 · App. 17/959,156 · Granted Jul 22, 2025

Inventors: Yoshie Kimura (Kawasaki, JP); Genta Suzuki (Kawasaki, JP)

Assignee: FUJITSU LIMITED

G06V10/7747G06V10/25G06V10/763G06V20/41G06V40/20

Abstract

Claims (58)

1. A non-transitory computer-readable recording medium storing a label change program for causing a computer to execute a process comprising:

acquiring image data that includes a plurality of areas;

setting a label for each of the plurality of areas by inputting the image data to a first machine learning model;

specifying a behavior performed by a person located in a first area among the plurality of areas for an object located in a second area; and

changing a label set for the second area based on a specified behavior of the person,

wherein a process is executed including

setting each reference line that indicates a movement route of a person in an aisle region of the image data by using tracking information obtained by tracking the same person based on video data that includes the image data obtained by imaging an inside of a room,

specifying a position of each person that appears in the video data based on skeleton information of the each person,

specifying a movement trajectory of the each person in the video data by using a position of the each person,

generating a plurality of clusters by clustering based on a distance between the each reference line and a movement trajectory of the each person in the image data,

extracting a region of interest that includes a cluster for which an evaluation value based on an angle formed by each movement trajectory that belongs to the cluster and the reference line is equal to or larger than a threshold, for each of the plurality of clusters, and

changing the label set for each of the plurality of areas set by the first machine learning model based on a region of interest that includes the cluster.

2. The non-transitory computer-readable recording medium according to claim 1 ,

wherein the specifying of a behavior executes a process of

generating skeleton information of the person located in the first area by inputting the acquired image data to a second machine learning model, and

wherein the changing executes a process including specifying a behavior of the person for an object in the second area based on the generated skeleton information, and

changing a label set for the second area by using the specified behavior.

3. The non-transitory computer-readable recording medium according to claim 1 , wherein the changing executes a process including

setting the each reference line in an aisle region identified by the first machine learning model,

generating a plurality of clusters by clustering based on a distance between each pixel that belongs to the aisle region and the each reference line,

specifying a cluster of interest that corresponds to the region of interest among the plurality of clusters, correcting a region of the cluster of interest to a region that includes the corresponding region of interest, and

changing a label already set for the corrected region by the first machine learning model to a label that corresponds to the region of interest.

4. The non-transitory computer-readable recording medium according to claim 1 , wherein a process is executed including

specifying, from each piece of image data in video data that includes the image data, a position of each person that appears in the video data,

specifying a region of interest that is a target of behavior analysis of the person in the first area based on an angle formed by a face direction of the person and a body direction of the person at a position of the each person, and

changing a label of each of the plurality of areas set by the first machine learning model based on the region of interest.

5. The non-transitory computer-readable recording medium according to claim 4 , wherein the changing executes a process including

setting each reference line that indicates a movement route of a person in an aisle region identified by the first machine learning model,

generating a plurality of clusters by clustering based on a distance between each pixel that belongs to the aisle region and the each reference line,

specifying a cluster of interest that corresponds to the region of interest among the plurality of clusters,

correcting a region of the cluster of interest to a region that includes the corresponding region of interest, and

changing a label set for the corrected region by the first machine learning model to a label that corresponds to the region of interest.

6. A label change method comprising:

acquiring image data that includes a plurality of areas;

setting a label for each of the plurality of areas by inputting the image data to a first machine learning model;

specifying a behavior performed by a person located in a first area among the plurality of areas for an object located in a second area; and

changing a label set for the second area based on a specified behavior of the person,

wherein the method includes

setting each reference line that indicates a movement route of a person in an aisle region of the image data by using tracking information obtained by tracking the same person based on video data that includes the image data obtained by imaging an inside of a room,

specifying a position of each person that appears in the video data based on skeleton information of the each person,

specifying a movement trajectory of the each person in the video data by using a position of the each person,

generating a plurality of clusters by clustering based on a distance between the each reference line and a movement trajectory of the each person in the image data,

extracting a region of interest that includes a cluster for which an evaluation value based on an angle formed by each movement trajectory that belongs to the cluster and the reference line is equal to or larger than a threshold, for each of the plurality of clusters, and

changing the label set for each of the plurality of areas set by the first machine learning model based on a region of interest that includes the cluster.

7. An information processing apparatus comprising:

a memory; and

a processor coupled to the memory and configured to:

acquire image data that includes a plurality of areas;

set a label for each of the plurality of areas by inputting the image data to a first machine learning model;

specify a behavior performed by a person located in a first area among the plurality of areas for an object located in a second area; and

change a label set for the second area based on a specified behavior of the person,

wherein the processor

sets each reference line that indicates a movement route of a person in an aisle region of the image data by using tracking information obtained by tracking the same person based on video data that includes the image data obtained by imaging an inside of a room,

specifies a position of each person that appears in the video data based on skeleton information of the each person,

specifies a movement trajectory of the each person in the video data by using a position of the each person,

generates a plurality of clusters by clustering based on a distance between the each reference line and a movement trajectory of the each person in the image data,

extracts a region of interest that includes a cluster for which an evaluation value based on an angle formed by each movement trajectory that belongs to the cluster and the reference line is equal to or larger than a threshold, for each of the plurality of clusters, and

changes the label set for each of the plurality of areas set by the first machine learning model based on a region of interest that includes the cluster.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (17)