Contrastive learning by dynamically selecting dropout ratios and locations based on reinforcement learning
A method for contrastive learning by selecting dropout ratios and locations based on reinforcement learning includes receiving training data having a positive sample corresponding to a target and negative samples not corresponding to the target. A dropout policy for a neural network is produced based on the training data, where the dropout policy identifies at least one connection between neurons in the neural network to dropout. The training data is encoded, based on the dropout policy, to form embeddings, where the embeddings include multiple positive sample embeddings corresponding to the positive sample and multiple negative sample embedding corresponding to the negative samples.
1 . A method, comprising:
receiving, by a processor, training data comprising a positive sample corresponding to a target and negative samples not corresponding to the target;
producing, by a dropout controller and based on the training data, a dropout policy for a neural network, wherein the dropout policy identifies at least one connection between neurons in the neural network to dropout, wherein producing the dropout policy further comprises,
producing, by a policy network and based on the training data, a set of dropout policies corresponding to different dropout policies,
selecting, by the dropout controller and based on a short-term reward, a portion of the set of dropout policies, and
selecting, by the dropout controller and based on a long-term reward, one policy from the portion of the set of dropout policies as the dropout policy; and
encoding, by an encoder and based on the dropout policy, the training data to form embeddings,
wherein the embeddings comprise multiple positive sample embeddings corresponding to the positive sample and multiple negative sample embeddings corresponding to the negative samples, and wherein the embeddings are generated based on the dropout policy.
2 . The method of claim 1 , wherein producing the dropout policy comprises:
representing connections between the neurons in the neural network with dropout matrices; and
selecting, based on a reinforcement learning method, one or more of the connections to disconnect to produce the dropout policy.
3 . The method of claim 2 , wherein representing the connections comprises producing N−1 dropout matrices, wherein N comprises any number and corresponds to a number of layers in the neural network, and wherein dimensions of each of the N−1 dropout matrices correspond to a number of neurons in adjacent layers in the neural network.
4 . The method of claim 1 , further comprising calculating both the short-term reward and the long-term reward using a same loss function.
5 . The method of claim 1 , wherein selecting the one policy comprises comparing policies within the portion of the set of dropout policies using a Monte Carlo search tree.
6 . The method of claim 1 , wherein producing the dropout policy comprises producing the dropout policy using transformer networks, wherein the transformer networks comprise a current network, a history network, and a composite network, wherein the current network comprises current parameter information of the current network, wherein the history network comprises previous parameter information of previous networks, and wherein the composite network is configured to merge the current parameter information with the previous parameter information to produce composite parameter information.
7 . A system, comprising:
a non-transitory computer-readable storage memory configured to store instructions; and
a processor coupled to the non-transitory computer-readable storage memory and configured to execute the instructions to cause the system to:
receive, by the processor, training data comprising a positive sample corresponding to a target and negative samples not corresponding to the target;
produce, by a dropout controller and based on the training data, a dropout policy for a neural network, wherein the dropout policy identifies at least one connection between neurons in the neural network to dropout, wherein producing the dropout policy further comprises,
produce, by a policy network and based on the training data, a set of dropout policies corresponding to different dropout policies,
select, by the dropout controller and based on a short-term reward, a portion of the set of dropout policies, and
select, by the dropout controller and based on a long-term reward, one policy from the portion of the set of dropout policies as the dropout policy; and
encode, by an encoder and based on the dropout policy, the training data to form embeddings,
wherein the embeddings comprise multiple positive sample embeddings corresponding to the positive sample and multiple negative sample embeddings corresponding to the negative samples, and wherein the embeddings are generated based on the dropout policy.
8 . The system of claim 7 , wherein the processor is further configured to execute the instructions to cause the system to:
represent connections between the neurons in the neural network with dropout matrices; and
select, based on a reinforcement learning method, one or more of the connections to disconnect to produce the dropout policy.
9 . The system of claim 8 , wherein the processor is further configured to execute the instructions to cause the system to produce N−1 dropout matrices, wherein N comprises any number and corresponds to a number of layers in the neural network, and wherein dimensions of each of the N−1 dropout matrices correspond to a number of neurons in adjacent layers in the neural network.
10 . The system of claim 7 , wherein the processor is further configured to execute the instructions to cause the system to calculate both the short-term reward and the long-term reward using a same loss function.
11 . The system of claim 7 , wherein the processor is further configured to execute the instructions to cause the system to compare policies within the portion of the set of dropout policies using a Monte Carlo search tree.
12 . The system of claim 7 , wherein the processor is further configured to execute the instructions to cause the system to produce the dropout policy using transformer networks, wherein the transformer networks comprise a current network, a history network, and a composite network, wherein the current network comprises current parameter information of the current network, wherein the history network comprises previous parameter information of previous networks, and wherein the composite network is configured to merge the current parameter information with the previous parameter information to produce composite parameter information.
13 . A computer program product comprising instructions stored on a non-transitory computer-readable medium that, when executed by a processor, cause a system to:
receive, by the processor, training data comprising a positive sample corresponding to a target and negative samples not corresponding to the target;
produce, by a dropout controller and based on the training data, a dropout policy for a neural network, wherein the dropout policy identifies at least one connection between neurons in the neural network to dropout, wherein producing the dropout policy further comprises,
produce, by a policy network and based on the training data, a set of dropout policies corresponding to different dropout policies,
select, by the dropout controller and based on a short-term reward, a portion of the set of dropout policies, and
select, by the dropout controller and based on a long-term reward, one policy from the portion of the set of dropout policies as the dropout policy; and
encode, by an encoder and based on the dropout policy, the training data to form embeddings,
wherein the embeddings comprise multiple positive sample embeddings corresponding to the positive sample and multiple negative sample embeddings corresponding to the negative samples, and wherein the embeddings are generated based on the dropout policy.
14 . The computer program product of claim 13 , wherein the instructions further cause the system to:
represent connections between the neurons in the neural network with dropout matrices; and
select, based on a reinforcement learning method, one or more of the connections to disconnect to produce the dropout policy.
15 . The computer program product of claim 14 , wherein the instructions further cause the system to produce N−1 dropout matrices, wherein N comprises any number and corresponds to a number of layers in the neural network, and wherein dimensions of each of the N−1 dropout matrices correspond to a number of neurons in adjacent layers in the neural network.
16 . The computer program product of claim 13 , wherein the instructions further cause the system to calculate both the short-term reward and the long-term reward using a same loss function.
17 . The computer program product of claim 13 , wherein the instructions further cause the system to compare policies within the portion of the set of dropout policies using a Monte Carlo search tree.