Data processing method and apparatus, computer device, and storage medium

1 . A method performed by a computer device, the method comprising:

determining position information for a plurality of texts and a plurality of pictures of an article;

constructing a relative position encoding matrix based on the position information, wherein the relative position encoding matrix includes a plurality of elements, and each element of the plurality of elements represents a respective relative position information between a text, of the plurality of texts of the article, corresponding to a column to which the respective element belongs and a picture, of the plurality of pictures of the article, corresponding to a row to which the respective element belongs;

determining relative position features between the plurality of texts and the plurality of pictures based on the relative position encoding matrix, the relative position features characterizing sequences and distances between textual data corresponding to the plurality of texts of the article and picture data corresponding to the plurality of pictures of the article, wherein the relative position features include a textual feature and a picture feature of the article;

extracting a textual semantic feature of the textual data, and fusing the textual semantic feature and a textual position feature of the textual data to obtain the textual feature characterizing the textual data in the article;

extracting a picture depth feature of the picture data, and fusing the picture depth feature and a picture position feature to obtain the picture feature characterizing picture data in the article;

determining a first interaction feature for the textual feature based on a partial feature associated with the textual feature in the picture feature, wherein the first interaction feature characterizes the textual feature fused with the picture feature;

determining a second interaction feature for the picture feature based on a partial feature associated with the picture feature in the textual feature, wherein the second interaction feature characterizes the picture feature fused with the textual feature;

obtaining a cross-modal interaction feature by fusing the first interaction feature and the second interaction feature; and

determining an article classification to which the article belongs based on the cross-modal interaction feature.

2 . The method according to claim 1 , wherein the first interaction feature comprises a title interaction feature and a text interaction feature, and determining the first interaction feature for the textual feature based on the partial feature associated with the textual feature in the picture feature comprises:

determining the title interaction feature for a title feature based on partial features associated with the title feature in the textual feature and the picture feature, the title interaction feature characterizing the title feature fused with the textual feature and the picture feature; and

determining the text interaction feature for the textual feature based on partial features associated with the textual feature in the title feature and the picture feature, the text interaction feature characterizing the textual feature fused with the title feature and the picture feature.

3 . The method according to claim 2 , wherein determining the title interaction feature for the title feature based on the partial features associated with the title feature in the textual feature and the picture feature further comprises:

determining a first title interaction feature based on a partial feature associated with the title feature in the picture feature;

determining a second title interaction feature based on a partial feature associated with the title feature in the picture feature;

determining a third title interaction feature by concatenating the first title interaction feature and the second title interaction feature; and

encoding and decoding the third title interaction feature to determine the title interaction feature.

4 . The method according to claim 1 , wherein the determining the second interaction feature for the picture feature based on the partial feature associated with the picture feature in the textual feature comprises:

determining the second interaction feature for the picture feature based on partial features associated with the picture feature in a title feature and the textual feature.

5 . The method according to claim 1 , wherein determining the article classification to which the article belongs based on the cross-modal interaction feature comprises:

obtaining a full-connection feature of the article from the cross-modal interaction feature;

obtaining a probability prediction result of the article from the full-connection feature, the probability prediction result comprising a plurality of prediction probabilities corresponding to a plurality of classifications; and

determining a classification corresponding to one of the plurality of prediction probabilities meeting a target condition as an article classification to which the article belongs.

6 . The method according to claim 1 , wherein:

the textual data includes title data and text data; and

extracting the textual semantic feature of the textual data further comprises:

extracting the title data and the text data from the textual data;

extracting a title semantic feature of the title data; and

fusing the title semantic feature and a title position feature to obtain a title feature of the title data.

7 . The method according to claim 1 , wherein:

a dimension of the picture depth feature is different from a dimension of the picture position feature; and

fusing the picture depth feature and the picture position feature to obtain the picture feature further comprises:

performing a dimension transformation on the picture depth feature to obtain a dimension-transformed picture depth feature whose dimension is the same as the picture position feature; and

fusing the dimension-transformed picture depth feature and the picture position feature to obtain the picture feature.

8 . The method according to claim 1 , wherein:

the plurality of pictures constitutes a picture sequence; and

extracting the picture depth feature of the picture data further comprises:

performing a convolution operation is performed on the picture sequence to obtain first feature map;

inputting the first feature map into a plurality of cascaded depthwise separable convolution layers of a machine learning model and obtaining a feature map, wherein each depthwise separable convolution layer of the plurality of cascaded depthwise separable convolution layers is configured to perform a respective depthwise separable convolution operation on a feature map that is output by a previous layer; and

inputting the feature map into a full-connection layer of the machine learning model; and

obtaining the picture depth feature of the picture data via the full-connection layer of the machine learning model.

9 . A computer device, comprising:

one or more processors; and

one or more memories, the one or more memories storing therein at least one computer program, and the at least one computer program being loaded and executed by the one or more processors and causing the computer device to perform operations including:

determining position information for a plurality of texts and a plurality of pictures of an article:

constructing a relative position encoding matrix based on the position information, wherein the relative position encoding matrix includes a plurality of elements, and each element of the plurality of elements represents a respective relative position information between a text, of the plurality of texts of the article, corresponding to a column to which the respective element belongs and a picture, of the plurality of pictures of the article, corresponding to a row to which the respective element belongs;

determining relative position features between the plurality of texts and the plurality of pictures based on the relative position encoding matrix, the relative position features characterizing sequences and distances between textual data corresponding to the plurality of texts of the article and picture data corresponding to the plurality of pictures of the article, wherein the relative position features include a textual feature and a picture feature of the article;

extracting a textual semantic feature of the textual data, and fusing the textual semantic feature and a textual position feature of the textual data to obtain the textual feature characterizing the textual data in the article;

extracting a picture depth feature of the picture data, and fusing the picture depth feature and a picture position feature to obtain the picture feature characterizing picture data in the article;

determining a first interaction feature for the textual feature based on a partial feature associated with the textual feature in the picture feature, wherein the first interaction feature characterizes the textual feature fused with the picture feature;

determining a second interaction feature for the picture feature based on a partial feature associated with the picture feature in the textual feature, wherein the second interaction feature characterizes the picture feature fused with the textual feature;

obtaining a cross-modal interaction feature by fusing the first interaction feature and the second interaction feature; and

determining an article classification to which the article belongs based on the cross-modal interaction feature.

10 . The computer device according to claim 9 , wherein the first interaction feature comprises a title interaction feature and a text interaction feature, and determining the first interaction feature for the textual feature based on the partial feature associated with the textual feature in the picture feature comprises:

determining the title interaction feature for a title feature based on partial features associated with the title feature in the textual feature and the picture feature, the title interaction feature characterizing the title feature fused with the textual feature and the picture feature; and

determining the text interaction feature for the textual feature based on partial features associated with the textual feature in the title feature and the picture feature, the text interaction feature characterizing the textual feature fused with the title feature and the picture feature.

11 . The computer device according to claim 9 , wherein determining the article classification to which the article belongs based on the cross-modal interaction feature comprises:

obtaining a full-connection feature of the article from the cross-modal interaction feature;

obtaining a probability prediction result of the article from the full-connection feature, the probability prediction result comprising a plurality of prediction probabilities corresponding to a plurality of classifications; and

determining a classification corresponding to one of the plurality of prediction probabilities meeting a target condition as an article classification to which the article belongs.

12 . The computer device according to claim 9 , wherein:

the textual data includes title data and text data; and

extracting the textual semantic feature of the textual data further comprises:

extracting the title data and the text data from the textual data;

extracting a title semantic feature of the title data; and

fusing the title semantic feature and a title position feature to obtain a title feature of the title data.

13 . The computer device according to claim 9 , wherein:

a dimension of the picture depth feature is different from a dimension of the picture position feature; and

fusing the picture depth feature and the picture position feature to obtain the picture feature further comprises:

performing a dimension transformation on the picture depth feature to obtain a dimension-transformed picture depth feature whose dimension is the same as the picture position feature; and

fusing the dimension-transformed picture depth feature and the picture position feature to obtain the picture feature.

14 . The computer device according to claim 9 , wherein:

the plurality of pictures constitutes a picture sequence; and

extracting the picture depth feature of the picture data further comprises:

performing a convolution operation is performed on the picture sequence to obtain first feature map;

inputting the first feature map into a plurality of cascaded depthwise separable convolution layers of a machine learning model and obtaining a feature map, wherein each depthwise separable convolution layer of the plurality of cascaded depthwise separable convolution layers is configured to perform a respective depthwise separable convolution operation on a feature map that is output by a previous layer; and

inputting the feature map into a full-connection layer of the machine learning model; and

obtaining the picture depth feature of the picture data via the full-connection layer of the machine learning model.

15 . A non-transitory computer-readable storage medium, storing therein at least one computer program, the at least one computer program being loaded and executed by a processor of a computer device and causing the computer device to perform operations including:

determining position information for a plurality of texts and a plurality of pictures of an article;

constructing a relative position encoding matrix based on the position information, wherein the relative position encoding matrix includes a plurality of elements, and each element of the plurality of elements represents a respective relative position information between a text, of the plurality of texts of the article, corresponding to a column to which the respective element belongs and a picture, of the plurality of pictures of the article, corresponding to a row to which the respective element belongs;

determining relative position features between the plurality of texts and the plurality of pictures based on the relative position encoding matrix, the relative position features characterizing sequences and distances between textual data corresponding to the plurality of texts of the article and picture data corresponding to the plurality of pictures of the article, wherein the relative position features include a textual feature and a picture feature of the article;

extracting a textual semantic feature of the textual data, and fusing the textual semantic feature and a textual position feature of the textual data to obtain the textual feature characterizing textual data in the article;

extracting a picture depth feature of the picture data, and fusing the picture depth feature and a picture position feature to obtain the picture feature characterizing picture data in the article;

determining a first interaction feature for the textual feature based on a partial feature associated with the textual feature in the picture feature, wherein the first interaction feature characterizes the textual feature fused with the picture feature;

determining a second interaction feature for the picture feature based on a partial feature associated with the picture feature in the textual feature, wherein the second interaction feature characterizes the picture feature fused with the textual feature;

obtaining a cross-modal interaction feature by fusing the first interaction feature and the second interaction feature; and

determining an article classification to which the article belongs based on the cross-modal interaction feature.

16 . The non-transitory computer-readable storage medium according to claim 15 , wherein determining the article classification to which the article belongs based on the cross-modal interaction feature comprises:

obtaining a full-connection feature of the article from the cross-modal interaction feature;

obtaining a probability prediction result of the article from the full-connection feature, the probability prediction result comprising a plurality of prediction probabilities corresponding to a plurality of classifications; and

determining a classification corresponding to one of the plurality of prediction probabilities meeting a target condition as an article classification to which the article belongs.

17 . The non-transitory computer-readable storage medium according to claim 15 , wherein:

the textual data includes title data and text data; and

extracting the textual semantic feature of the textual data further comprises:

extracting the title data and the text data from the textual data;

extracting a title semantic feature of the title data; and

fusing the title semantic feature and a title position feature to obtain a title feature of the title data.

18 . The non-transitory computer-readable storage medium according to claim 15 , wherein:

a dimension of the picture depth feature is different from a dimension of the picture position feature; and

fusing the picture depth feature and the picture position feature to obtain the picture feature further comprises:

performing a dimension transformation on the picture depth feature to obtain a dimension-transformed picture depth feature whose dimension is the same as the picture position feature; and

fusing the dimension-transformed picture depth feature and the picture position feature to obtain the picture feature.

19 . The non-transitory computer-readable storage medium according to claim 15 , wherein:

the plurality of pictures constitutes a picture sequence; and

extracting the picture depth feature of the picture data further comprises:

performing a convolution operation is performed on the picture sequence to obtain first feature map;

inputting the first feature map into a plurality of cascaded depthwise separable convolution layers of a machine learning model and obtaining a feature map, wherein each depthwise separable convolution layer of the plurality of cascaded depthwise separable convolution layers is configured to perform a respective depthwise separable convolution operation on a feature map that is output by a previous layer; and

inputting the feature map into a full-connection layer of the machine learning model; and

obtaining the picture depth feature of the picture data via the full-connection layer of the machine learning model.