Video query method, apparatus, and device, and storage medium
A video query method includes obtaining a media feature of a query media and a static image feature corresponding to a candidate video. The query media includes the target object, and the candidate video includes the moving object. A video feature of the candidate video is determined according to the static image feature and motion time sequence information of the moving object in the candidate video. Whether the moving object in the candidate video is related to the target object in the query media can be determined according to the media feature and the video feature.
1. A video query method, implemented by an electronic device, comprising:
obtaining a media feature of a query media and a static image feature corresponding to a candidate video, the query media comprising a target object, the candidate video comprising a moving object;
determining a video feature of the candidate video according to the static image feature and motion time sequence information of the moving object in the candidate video, wherein:
the video feature comprises sub-features corresponding to n video segments in the candidate video, n being an integer greater than 1; the motion time sequence information reflects motion trends of the moving object between adjacent video segments; and
a t th video segment corresponds to a t th sub-feature, t being a positive integer greater than 1 and not greater than n, and the t th sub-feature is determined by:
determining, in the t th video segment, a target motion trend of the moving object from a (t−1) th sub-feature;
adjusting a target region corresponding to the moving object in a feature plane of the (t−1) t sub-feature according to the target motion trend; and
determining the t th sub-feature according to the adjusted (t−1) th sub-feature and a static image feature corresponding to the t th video segment; and
determining, according to the media feature and the video feature, whether the moving object in the candidate video is related to the target object.
2. The method according to claim 1 , wherein the adjusting a target region corresponding to the moving object in a feature plane of the (t−1) th sub-feature according to the target motion trend comprises:
determining a deviation value corresponding to a control point in the target region according to the target motion trend; and
moving the control point according to the deviation value, and adjusting the target region corresponding to the moving object in the feature plane of the (t−1) th sub-feature based on the moved control point.
3. The method according to claim 1 , wherein the determining, according to the media feature and the video feature, whether the moving object in the candidate video is related to the target object comprises:
determining a target region having an association relationship with the target object in a feature plane of the t th sub-feature;
determining a region feature corresponding to the target region according to the t th sub-feature; and
determining, according to the region feature and the media feature, whether the moving object in the t th video segment is related to the target object.
4. The method according to claim 3 , wherein the query media is a query video, and the determining, according to the region feature and the media feature, whether the moving object in the t th video segment is related to the target object comprises:
determining a weight between video frame content in the query video and the target object by using an attention model;
determining a query video feature of the query video according to the weight; and
determining, according to the region feature and the query video feature, whether the moving object in the t th video segment is related to the target object.
5. The method according to claim 3 , wherein when the moving object in the t th video segment is related to the target object, the method further comprises:
identifying the target region in the t th video segment of the candidate video.
6. The method according to claim 4 , wherein when the moving object in the t th video segment is related to the target object, the method further comprises:
identifying the target region in the t th video segment of the candidate video.
7. A video query device, comprising a processor and a memory:
the memory being configured to store program code, and transmit the program code to the processor; and
the processor being configured to execute the program code to perform:
obtaining a media feature of a query media and a static image feature corresponding to a candidate video, the query media comprising a target object, the candidate video comprising a moving object;
determining a video feature of the candidate video according to the static image feature and motion time sequence information of the moving object in the candidate video, wherein:
the video feature comprises sub-features corresponding to n video segments in the candidate video, n being an integer greater than 1; the motion time sequence information reflects motion trends of the moving object between adjacent video segments; and
a t th video segment corresponds to a t th sub-feature, t being a positive integer greater than 1 and not greater than n, and the t th sub-feature is determined by:
determining, in the t th video segment, a target motion trend of the moving object from a (t−1) th sub-feature;
adjusting a target region corresponding to the moving object in a feature plane of the (t−1) t sub-feature according to the target motion trend; and
determining the t th sub-feature according to the adjusted (t−1) th sub-feature and a static image feature corresponding to the t th video segment; and
determining, according to the media feature and the video feature, whether the moving object in the candidate video is related to the target object.
8. The device according to claim 7 , wherein the adjusting a target region corresponding to the moving object in a feature plane of the (t−1) th sub-feature according to the target motion trend comprises:
determining a deviation value corresponding to a control point in the target region according to the target motion trend; and
moving the control point according to the deviation value, and adjusting the target region corresponding to the moving object in the feature plane of the (t−1) th sub-feature based on the moved control point.
9. The device according to claim 7 , wherein the determining,
according to the media feature and the video feature, whether the moving object in the candidate video is related to the target object comprises:
determining a target region having an association relationship with the target object in a feature plane of the t th sub-feature;
determining a region feature corresponding to the target region according to the t th sub-feature; and
determining, according to the region feature and the media feature, whether the moving object in the t th video segment is related to the target object.
10. The device according to claim 9 , wherein the query media is a query video, and the determining, according to the region feature and the media feature, whether the moving object in the t th video segment is related to the target object comprises:
determining a weight between video frame content in the query video and the target object by using an attention model;
determining a query video feature of the query video according to the weight; and
determining, according to the region feature and the query video feature, whether the moving object in the t th video segment is related to the target object.
11. The device according to claim 10 , wherein when the moving object in the t th video segment is related to the target object, the method further comprises:
identifying the target region in the t th video segment of the candidate video.
12. The device according to claim 10 , wherein when the moving object in the t th video segment is related to the target object, the method further comprises:
identifying the target region in the t th video segment of the candidate video.
13. A non-transitory computer-readable storage medium, configured to store program code, the program code, when being executed by a processor, causing the processor to perform:
obtaining a media feature of a query media and a static image feature corresponding to a candidate video, the query media comprising a target object, the candidate video comprising a moving object;
determining a video feature of the candidate video according to the static image feature and motion time sequence information of the moving object in the candidate video, wherein:
the video feature comprises sub-features corresponding to n video segments in the candidate video, n being an integer greater than 1; the motion time sequence information reflects motion trends of the moving object between adjacent video segments; and
a t th video segment corresponds to a t th sub-feature, t being a positive integer greater than 1 and not greater than n, and the t th sub-feature is determined by:
determining, in the t th video segment, a target motion trend of the moving object from a (t−1) th sub-feature;
adjusting a target region corresponding to the moving object in a feature plane of the (t−1) t sub-feature according to the target motion trend; and
determining the t th sub-feature according to the adjusted (t−1) th sub-feature and a static image feature corresponding to the t th video segment; and
determining, according to the media feature and the video feature, whether the moving object in the candidate video is related to the target object.
14. The storage medium according to claim 13 , wherein the adjusting a target region corresponding to the moving object in a feature plane of the (t−1) th sub-feature according to the target motion trend comprises:
determining a deviation value corresponding to a control point in the target region according to the target motion trend; and
moving the control point according to the deviation value, and adjusting the target region corresponding to the moving object in the feature plane of the (t−1) th sub-feature based on the moved control point.
15. The storage medium according to claim 13 , wherein the determining, according to the media feature and the video feature, whether the moving object in the candidate video is related to the target object comprises:
determining a target region having an association relationship with the target object in a feature plane of the t th sub-feature;
determining a region feature corresponding to the target region according to the t th sub-feature; and
determining, according to the region feature and the media feature, whether the moving object in the t th video segment is related to the target object.
16. The storage medium according to claim 15 , wherein the query media is a query video, and the determining, according to the region feature and the media feature, whether the moving object in the t th video segment is related to the target object comprises:
determining a weight between video frame content in the query video and the target object by using an attention model;
determining a query video feature of the query video according to the weight; and
determining, according to the region feature and the query video feature, whether the moving object in the t th video segment is related to the target object.
17. The storage medium according to claim 16 , wherein when the moving object in the t th video segment is related to the target object, the method further comprises:
identifying the target region in the t th video segment of the candidate video.