Image processing device, image processing method, and image processing program

1. A video processing device that generates a movement vector for a video content, the video processing device comprising processing circuitry configured to:

estimate scene class information from the video content, the scene class information being information identifying a scene class for the video content; and

perform one of a plurality of movement information estimations to estimate a movement vector from the video content,

wherein one movement information estimation that is selected from the plurality of the movement information estimations in response to the scene class identified by the scene class information is performed to estimate the movement vector; and

wherein the scene classes include at least a class representing a first-person viewpoint video, a class representing a bird's-eye view video that contains a person, and a class representing a bird's- eye view video that contains no person.

2. The video processing device according to claim 1 , wherein the scene class is defined in accordance with at least one of information identifying a viewpoint of filming the video content, information identifying presence or absence of a specific target object or a number of the specific target objects in the video content, and information identifying a genre of the video content.

3. The video processing device according to claim 1 , wherein the processing circuitry estimates the scene class information using a pre-trained scene identification model that has learned a relationship between the video content and the scene class information in advance.

4. The video processing device according to claim 1 , wherein when the scene class identified by the scene class information represents a bird's-eye view video that contains a person, the selected one movement information estimation is performed to estimate coordinates of a skeletal structure of a person contained in the video content in response to input on the video content, calculate movement vectors for the coordinates of the skeletal structure using time-series data about the estimated coordinates of the skeletal structure of the person, and output a representative value of the movement vectors for the coordinates of the skeletal structure as a movement vector for the video content.

5. The video processing device according to claim 1 , wherein when the scene class identified by the scene class information represents a bird's-eye view video that contains no person, the selected one movement information estimation is performed to calculate optical flow for the input video content and output a representative value of the optical flow as a movement vector for the video content.

6. The video processing device according to claim 1 , wherein the one movement information estimation that is selected when the scene class identified by the scene class information represents a first-person viewpoint video is performed to:

calculate optical flow in a first region of the input video content, the first region reflecting a movement experienced by a photographer in a traveling direction;

calculate optical flow in a second region of the input video content, the second region reflecting a movement experienced by the photographer in a direction other than the traveling direction; and

determine a weighted sum of a representative value of the optical flow in the first region and a representative value of the optical flow in the second region and output the determined weighted sum as a movement vector for the video content.

7. The video processing device according to claim 1 , wherein the processing circuitry is configured to generate force sense control information from the movement vector to drive a haptic device.

8. A video processing method comprising:

estimating scene class information that is information identifying a scene class for a video content;

selecting one movement information estimation based on the estimated scene class information from a plurality of movement information estimations; and

performing the one movement information estimation to estimate a movement vector from the video content; and

wherein the scene classes include at least a class representing a first-person viewpoint video, a class representing a bird's-eye view video that contains a person, and a class representing a bird's- eye view video that contains no person.

9. A non-transitory computer readable recording medium storing a video processing program allowing a computer to function as the video processing device according to claim 1 .