Method and device for processing data based on multi-layer perceptrons

1 . A method for data processing using a phase-aware token mixing (PATM) module in a multi-layer perceptrons (MLP) architecture, the method comprising:

determining, in the MLP architecture, a plurality of tokens for data comprising pixels of an image,

wherein the MLP architecture includes a sub-network including a plurality of branches to process the plurality of tokens in parallel,

wherein a first branch of the plurality of branches is used to aggregate spatial information along height corresponding to the plurality of tokens,

wherein a second branch of the plurality of branches is used to aggregate spatial information along width associated with the plurality of tokens,

wherein a third branch of the plurality of branches is used to preserve original information in the plurality of tokens, and

wherein each token comprises a subset of the pixels describing a patch of the image;

generating, in the MLP architecture, a wave-like token for each of the plurality of tokens, thereby creating a plurality of wave-like tokens,

wherein each wave-like token in the plurality of wave-like tokens includes an amplitude and phase;

generating, by the PATM module, optimized tokens by superposing of the plurality of wave-like tokens, so as to aggregate information contained in the amplitudes and the phases of the plurality of wave-like tokens,

wherein the aggregate information in each optimized token includes information in its phase and amplitude in addition to information regarding modulation of fixed weights provided by the MLP architecture, and

wherein the additional information relates to attributes of the image; and

determining, in the MLP architecture, one or more features in the image based on a plurality of optimized tokens.

2 . The method according to claim 1 , wherein generating the amplitude and the phase for each of the plurality of wave-like tokens further comprises:

extracting information associated with each token for the plurality of tokens; and

determining the amplitude and the phase for the wave-like token based on the information associated with each token.

3 . The method according to claim 2 , wherein the information associated with the token comprises at least one feature of the one or more features included in the data,

wherein the amplitude for the wave-like token represents at least one feature in an associated token, and

wherein the phase for the wave-like token is related to differences between the token and other tokens in the plurality of tokens.

4 . The method according to claim 2 , wherein the extraction of the information associated with each token for the plurality of tokens is performed by a number of fully-connected (FC) layers included in the MLP architecture, and

wherein the number of FC layers are connected by at least one non-linear activation function.

5 . The method according to claim 1 , the method further comprising normalizing the plurality of tokens.

6 . The method according to claim 1 , wherein optimizing the plurality of wave-like tokens by superposing their amplitudes and the phases further comprises determining optimized amplitudes for the optimized tokens based on the phases for the plurality of tokens.

7 . The method according to claim 1 , wherein the method further comprises generating a plurality of feature maps associated with the plurality of tokens, each token carrying information in the corresponding feature map,

wherein each feature map includes spatial information along height, spatial information along width and information of a number of channels associated with the image.

8 . The method according to claim 1 , the plurality of tokens are processed by a number of iterations by using the MLP.

9 . The method according to claim 1 , wherein the method further comprises:

identifying one or more objects in the image based on the one or more features included in the data; and

classifying the image among a plurality of images based on the one or more objects.

10 . The method according to claim 1 , wherein the data is one of audio data, video data, graphic data, and contextual data.

11 . A device for data processing using a phase-aware token mixing (PATM) module in a multi-layer perceptrons (MLP) architecture, the device comprising:

one or more processors; and

a non-transitory computer-readable medium, having computer-executable instructions stored thereon, the computer-executable instructions, when executed by the one or more processors, causing the device to perform:

determining a plurality of tokens for data comprising pixels of an image,

wherein the MLP architecture includes a sub-network including a plurality of branches to process the plurality of tokens in parallel,

wherein a first branch of the plurality of branches is used to aggregate spatial information along height corresponding to the plurality of tokens,

wherein a second branch of the plurality of branches is used to aggregate spatial information along width associated with the plurality of tokens,

wherein a third branch of the plurality of branches is used to preserve original information in the plurality of tokens, and

wherein each token comprises a subset of the pixels describing a patch of the image;

generating a wave-like token for each of the plurality of tokens, thereby creating a plurality of wave-like tokens,

wherein each wave-like token in the plurality of wave-like tokens includes an amplitude and phase;

generating, at the PATM module, optimized tokens by superposing of the plurality of wave-like tokens, so as to aggregate information contained in the amplitudes and the phases of the plurality of wave-like tokens,

wherein the aggregate information in each optimized token includes information in its phase and amplitude in addition to information regarding modulation of fixed weights provided by the MLP architecture, and

wherein the additional information relates to attributes of the image; and

determining one or more features in the image based on a plurality of optimized tokens.

12 . The device according to claim 11 , wherein generating the amplitude and the phase for each of the plurality of wave-like tokens further comprises:

extracting information associated with each token for the plurality of tokens; and

determining the amplitude and the phase for the wave-like token based on the information associated with each token.

13 . The device according to claim 12 , wherein the information associated with the token comprises at least one feature of the one or more features included in the data,

wherein the amplitude for the wave-like token represents at least one feature in an associated token, and

wherein the phase for the wave-like token is related to differences between the token and other tokens in the plurality of tokens.

14 . The device according to claim 12 , wherein the extraction of the information associated with each token for the plurality of tokens is performed by a number of fully-connected (FC) layers included in the MLP architecture, and the number of FC layers are connected by at least one non-linear activation function.

15 . The device according to claim 11 , wherein optimizing the plurality of wave-like tokens by superposing their amplitudes and the phases further comprises determining optimized amplitudes for the optimized tokens based on the phases for the plurality of tokens.

16 . The device according to claim 11 , wherein the computer-executable instructions, when executed by one or more processors, cause the one or more processors to further facilitate:

generating a plurality of feature maps corresponding to the plurality of tokens, each token carrying information in the corresponding feature map,

wherein each feature map includes spatial information along height, spatial information along width and information of a number of channels associated with the image.

17 . The device according to claim 11 , wherein the computer-executable instructions, when executed by one or more processors, cause the one or more processors to further facilitate:

identifying one or more objects in the image based on the one or more features included in the data; and

classifying the image among a plurality of images based on the one or more objects.

18 . A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors based on a multi-layer perceptrons (MLP) architecture that includes a phase-aware token mixing (PATM) module, cause the one or more processors to perform:

determining a plurality of tokens for data comprising pixels of an image,

wherein the MLP architecture includes a sub-network including a plurality of branches to process the plurality of tokens in parallel,

wherein a first branch of the plurality of branches is used to aggregate spatial information along height corresponding to the plurality of tokens,

wherein a second branch of the plurality of branches is used to aggregate spatial information along width associated with the plurality of tokens,

wherein a third branch of the plurality of branches is used to preserve original information in the plurality of tokens, and

wherein each token comprises a subset of the pixels describing a patch of the image;

generating a wave-like token for each of the plurality of tokens, thereby creating a plurality of wave-like tokens,

wherein each wave-like token in the plurality of wave-like tokens includes an amplitude and phase;

generating, by the PATM module, optimized tokens by superposing of the plurality of wave-like tokens, so as to aggregate information contained in the amplitudes and the phases of the plurality of wave-like tokens,

wherein the aggregate information in each optimized token includes information in its phase and amplitude in addition to information regarding modulation of fixed weights provided by the MLP architecture, and

wherein the additional information relates to attributes of the image; and

determining one or more features in the image based on a plurality of optimized tokens.

19 . The non-transitory computer-readable medium according to claim 18 , wherein generating the amplitude and the phase for each of the plurality of wave-like tokens further comprises:

extracting information associated with each token for the plurality of tokens; and

determining the amplitude and the phase for the wave-like token based on the information associated with each token.

20 . The non-transitory computer-readable medium according to claim 18 , wherein optimizing the plurality of wave-like tokens by superposing their amplitudes and the phases further comprises determining optimized amplitudes for the optimized tokens based on the phases for the plurality of tokens.