System, information processing apparatus, method, and computer-readable medium
A skeleton information acquisition means analyzes an in-vehicle video acquired from a camera that captures an image of a vehicle interior of a moving object, and acquires skeleton information of each of one or more passengers boarding on the moving object. A holding determination means determines whether the passenger is holding a pole, based on skeleton information of the passenger and a position of the pole in the moving object. An alert output means outputs an alert when it is determined that the passenger is not holding the pole.
1 . An information processing apparatus comprising:
a memory storing instructions; and
a processor configured to execute the instructions to:
by analyzing an in-vehicle video acquired from a camera configured to capture an image of a vehicle interior of a moving object, acquire skeleton information of each of one or more passengers boarding on the moving object, the skeleton information including information indicating a position of a joint point of a wrist of the passenger;
determine whether the passenger is holding a pole, based on skeleton information of the passenger and a position of the pole in the moving object;
output an alert when it is determined that the passenger is not holding the pole; and
control a parameter for autonomous driving of the moving object based on whether the passenger is holding the pole,
wherein the processor is configured to execute the instructions to determine that the passenger is holding the pole in a case where a shortest distance from the joint point of the wrist to the pole in the in-vehicle video is equal to or less than a threshold value.
2 . The information processing apparatus according to claim 1 , wherein the processor is configured to execute the instructions to estimate skeleton structure of the passenger, based on the in-vehicle video, and detect the position of the joint point of the wrist, based on the estimated skeleton structure.
3 . The information processing apparatus according to claim 1 , wherein
the skeleton information further includes information indicating a position of a joint point of an elbow of the passenger, and
the processor is configured to execute the instructions to determine that the passenger is holding the pole, when the pole exists, in the in-vehicle video, between a point, on a straight line passing through the joint point of the elbow and the joint point of the wrist, being separated from the joint point of the wrist, on an opposite side of the joint point of the elbow, by a distance acquired by multiplying a distance between the joint point of the elbow and the joint point of the wrist by a predetermined coefficient, and the joint point of the wrist.
4 . The information processing apparatus according to claim 3 , wherein the processor is configured to execute the instructions to estimate skeleton structure of the passenger, based on the in-vehicle video, and detect a position of the joint point of the elbow and the position of the joint point of the wrist, based on the estimated skeleton structure.
5 . The information processing apparatus according to claim 1 , wherein
the skeleton information further includes information indicating a position of a joint point of an elbow of the passenger, and
the processor is configured to execute the instructions to determine that the passenger is holding a pole, when a part of the pole exists, in the in-vehicle video, in a fan-shaped region passing through a point separated, on an opposite side of the joint point of the elbow, by a distance acquired by multiplying a distance between the joint point of the elbow and the joint point of the wrist by a predetermined coefficient, with the joint point of the wrist as a base point.
6 . The information processing apparatus according to claim 1 , the processor is further configured to execute the instructions to acquire a position of the pole, based on the in-vehicle video.
7 . The information processing apparatus according to claim 6 , wherein the processor is configured to execute the instructions to identify a region of the pole in the in-vehicle video by using color information, and detects a position of the pole.
8 . The information processing apparatus according to claim 1 , the processor is configured to execute the instructions to specify a position of the passenger and a position of the pole on a floor of the vehicle interior of the moving object, extract a pole that can be held by the passenger, based on a positional relationship between a position of the passenger on the floor and a position of the pole on the floor, and determines whether the passenger is holding the extracted pole.
9 . The information processing apparatus according to claim 8 , wherein the processor is configured to execute the instructions to extract, by a position of the passenger on the floor, a pole existing in a range in which an arm of the passenger reaches, as a pole that the passenger can hold.
10 . The information processing apparatus according to claim 9 , wherein the processor is configured to execute the instructions to estimate a height of the passenger, based on the skeleton information, and set a range in which an arm of the passenger reaches according to the estimated height of the passenger.
11 . The information processing apparatus according to claim 8 , wherein, when the pole is not attached to a floor of the vehicle interior in the in-vehicle video, the processor is configured to execute the instructions to expand the pole in a direction of the floor, and estimate a position of the pole on the floor.
12 . The information processing apparatus according to claim 1 , wherein the processor is configured to execute the instructions to determine whether a predetermined alert condition is satisfied, based on vehicle information received from the moving object, and output the alert when it is determined that the passenger is not holding the pole and the alert condition is satisfied.
13 . The information processing apparatus according to claim 12 , wherein
the vehicle information includes information indicating a speed of the moving object, and
the processor is configured to execute the instructions to determine whether the moving object is traveling, based on the vehicle information, and determine that the alert condition is satisfied when determining that the moving object is traveling.
14 . The information processing apparatus according to claim 12 , wherein
the vehicle information includes information indicating an opening/closing state of a door of the moving object, and
the processor is configured to execute the instructions to determine whether a door is closed in the moving object, based on the vehicle information, and determine that the alert condition is satisfied when determining that the door is closed.
15 . The information processing apparatus according to claim 1 , wherein the alert output means outputs the alert to the moving object.
16 . A system comprising:
a moving object including a camera configured to capture an image of a vehicle interior; and
the information processing apparatus according to claim 1 .
17 . An information processing method comprising:
by analyzing an in-vehicle video acquired from a camera configured to capture an image of a vehicle interior of a moving object, acquiring skeleton information of each of one or more passengers boarding on the moving object, the skeleton information including information indicating a position of a joint point of a wrist of the passenger;
determining whether the passenger is holding a pole, based on skeleton information of the passenger and a position of the pole in the moving object;
outputting an alert when determining that the passenger is not holding the pole;
controlling a parameter for autonomous driving of the moving object based on whether the passenger is holding the pole; and
determining that the passenger is holding the pole in a case where a shortest distance from the joint point of the wrist to the pole in the in-vehicle video is equal to or less than a threshold value.
18 . A non-transitory computer-readable medium storing a program for causing a processor to execute processing of:
by analyzing an in-vehicle video acquired from a camera configured to capture an image of a vehicle interior of a moving object, acquiring skeleton information of each of one or more passengers boarding on the moving object, the skeleton information including information indicating a position of a joint point of a wrist of the passenger;
determining whether the passenger is holding a pole, based on skeleton information of the passenger and a position of the pole in the moving object;
outputting an alert when determining that the passenger is not holding the pole;
controlling a parameter for autonomous driving of the moving object based on whether the passenger is holding the pole; and
determining that the passenger is holding the pole in a case where a shortest distance from the joint point of the wrist to the pole in the in-vehicle video is equal to or less than a threshold value.