Joint training method and apparatus for models, device and storage medium
View Patent ↗The present disclosure provides a joint training method and apparatus for models, a device and a storage medium. The method may include: training a first-party model to be trained using a first sample quantity of first-party training samples to obtain first-party feature gradient information; acquiring second-party feature gradient information and second sample quantity information from a second party, where the second-party feature gradient information is obtained by training, by the second party, a second-party model to be trained using a second sample quantity of second-party training samples; and determining model joint gradient information according to the first-party feature gradient information, the second-party feature gradient information, first sample quantity information and the second sample quantity information, and updating the first-party model and the second-party model according to the model joint gradient information.
1. A joint training method for models, comprising:
training a first-party model to be trained using a first sample quantity of first-party training samples to obtain first-party feature gradient information;
acquiring second-party feature gradient information and second sample quantity information from a second party, wherein the second-party feature gradient information is obtained by training, by the second party, a second-party model to be trained using a second sample quantity of second-party training samples;
determining a model joint gradient cipher text according to a first-party feature gradient cipher text, a second-party feature gradient cipher text, a first sample quantity cipher text and a second sample quantity cipher text; wherein the first-party feature gradient cipher text and the first sample quantity cipher text are obtained by respectively encrypting, by the first party, the first-party feature gradient plain text and the first sample quantity plain text using a first homomorphic encryption key acquired from the second party;
wherein the second-party feature gradient cipher text and the second sample quantity cipher text are obtained by respectively encrypting, by the second party, the second-party feature gradient plain text and the second sample quantity plain text using the first homomorphic encryption key, and wherein the first sample quantity is a sample quantity of the first-party training samples, and the second sample quantity is a sample quantity of the second-party training samples; and
updating the first-party model and the second-party model according to the model joint gradient information.
2. The method according to claim 1 , wherein the first-party training samples are randomly drawn by the first party from first-party sample data, and a quantity of samples randomly drawn from the first-party sample data is used as the first sample quantity; the second-party training samples are randomly drawn by the second party from second-party sample data, and a quantity of samples randomly drawn from the second-party sample data is used as the second sample quantity.
3. The method according to claim 1 , wherein the updating the first-party model and the second-party model according to the model joint gradient information comprises:
sending the model joint gradient cipher text to the second party for instructing the second party to perform operations comprising: decrypting the model joint gradient cipher text using a second homomorphic encryption key to obtain the model joint gradient plain text; and updating the second-party model according to the model joint gradient plain text; and
acquiring the model joint gradient plain text from the second party, and updating the first-party model according to the model joint gradient plain text.
4. The method according to claim 1 , wherein data structures of the first-party training samples and the second-party training samples are same.
5. The method according to claim 4 , further comprising:
determining common feature dimensions between the first-party sample data and the second-party sample data through a security intersection protocol; and
determining the first-party training samples and the second-party training samples based on the common feature dimensions.
6. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used for a computer to execute the method according to claim 1 .
7. A joint training method for models, comprising:
training a second-party model to be trained using a second sample quantity of second-party training samples to obtain second-party feature gradient information; and
sending second sample quantity information and the second-party feature gradient information to a first party for instructing the first party to perform operations comprising:
training a first-party model to be trained using a first sample quantity of first-party training samples to obtain first-party feature gradient information; and determining model joint gradient information according to the first-party feature gradient information, the second-party feature gradient information, first sample quantity information and the second sample quantity information, and updating the first-party model and the second-party model according to the model joint gradient information;
wherein the method further comprises: determining a first homomorphic encryption key and a second homomorphic encryption key, and sending the first homomorphic encryption key to the first party; wherein the sending second sample quantity information and the second-party feature gradient information to the first party comprises:
sending a second sample quantity cipher text and a second-party feature gradient cipher text to the first party for instructing the first party to perform operations comprising:
determining a model joint gradient cipher text according to a first-party feature gradient cipher text, the second-party feature gradient cipher text, a first sample quantity cipher text and the second sample quantity cipher text; wherein the first-party feature gradient cipher text and the first sample quantity cipher text are obtained by respectively encrypting, by the first party, the first-party feature gradient plain text and the first sample quantity plain text using the first homomorphic encryption key; wherein the second-party feature gradient cipher text and the second sample quantity cipher text are obtained by respectively encrypting, by the second party, the second-party feature gradient plain text and the second sample quantity plain text using the first homomorphic encryption key, and wherein the first sample quantity is a sample quantity of the first-party training samples, and the second sample quantity is a sample quantity of the second-party training samples.
8. The method according to claim 7 , wherein the first-party training samples are randomly drawn by the first party from first-party sample data, and a quantity of samples randomly drawn from the first-party sample data is used as the first sample quantity; the second-party training samples are randomly drawn by the second party from second-party sample data, and a quantity of samples randomly drawn from the second-party sample data is used as the second sample quantity.
9. The method according to claim 7 , wherein the updating the first-party model and the second-party model according to the model joint gradient information comprises:
acquiring the model joint gradient cipher text from the first party, decrypting the model joint gradient cipher text using the second homomorphic encryption key to obtain the model joint gradient plain text, and updating the second-party model according to the model joint gradient plain text; and
sending the model joint gradient plain text to the first party for instructing the first party to update the first-party model according to the model joint gradient plain text.
10. The method according to claim 7 , wherein data structures of the first-party training samples and the second-party training samples are same.
11. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used for a computer to execute the method according to claim 7 .
12. An electronic device, comprising:
at least one processor; and
a memory communicatively connected with the at least one processor;
wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, comprising:
training a first-party model to be trained using a first sample quantity of first-party training samples to obtain first-party feature gradient information;
acquiring second-party feature gradient information and second sample quantity information from a second party, wherein the second-party feature gradient information is obtained by training, by the second party, a second-party model to be trained using a second sample quantity of second-party training samples;
determining a model joint gradient cipher text according to a first-party feature gradient cipher text, a second-party feature gradient cipher text, a first sample quantity cipher text and a second sample quantity cipher text; wherein the first-party feature gradient cipher text and the first sample quantity cipher text are obtained by respectively encrypting, by the first party, the first-party feature gradient plain text and the first sample quantity plain text using a first homomorphic encryption key acquired from the second party;
wherein the second-party feature gradient cipher text and the second sample quantity cipher text are obtained by respectively encrypting, by the second party, the second-party feature gradient plain text and the second sample quantity plain text using the first homomorphic encryption key, and wherein the first sample quantity is a sample quantity of the first-party training samples, and the second sample quantity is a sample quantity of the second-party training samples; and
updating the first-party model and the second-party model according to the model joint gradient information.
13. The electronic device according to claim 12 , wherein the first-party training samples are randomly drawn by the first party from first-party sample data, and a quantity of samples randomly drawn from the first-party sample data is used as the first sample quantity; the second-party training samples are randomly drawn by the second party from second-party sample data, and a quantity of samples randomly drawn from the second-party sample data is used as the second sample quantity.
14. The electronic device according to claim 12 , wherein the updating the first-party model and the second-party model according to the model joint gradient information comprises:
sending the model joint gradient cipher text to the second party for instructing the second party to perform operations comprising: decrypting the model joint gradient cipher text using a second homomorphic encryption key to obtain the model joint gradient plain text; and updating the second-party model according to the model joint gradient plain text; and
acquiring the model joint gradient plain text from the second party, and updating the first-party model according to the model joint gradient plain text.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively connected with the at least one processor;
wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, comprising:
training a second-party model to be trained using a second sample quantity of second-party training samples to obtain second-party feature gradient information; and
sending second sample quantity information and the second-party feature gradient information to a first party for instructing the first party to perform operations comprising:
training a first-party model to be trained using a first sample quantity of first-party training samples to obtain first-party feature gradient information; and determining model joint gradient information according to the first-party feature gradient information, the second-party feature gradient information, first sample quantity information and the second sample quantity information, and updating the first-party model and the second-party model according to the model joint gradient information;
wherein the operations further comprise: determining a first homomorphic encryption key and a second homomorphic encryption key, and sending the first homomorphic encryption key to the first party; wherein the sending second sample quantity information and the second-party feature gradient information to the first party comprises:
sending a second sample quantity cipher text and a second-party feature gradient cipher text to the first party for instructing the first party to perform operations comprising: determining a model joint gradient cipher text according to a first-party feature gradient cipher text, the second-party feature gradient cipher text, a first sample quantity cipher text and the second sample quantity cipher text; wherein the first-party feature gradient cipher text and the first sample quantity cipher text are obtained by respectively encrypting, by the first party, the first-party feature gradient plain text and the first sample quantity plain text using the first homomorphic encryption key; wherein the second-party feature gradient cipher text and the second sample quantity cipher text are obtained by respectively encrypting, by the second party, the second-party feature gradient plain text and the second sample quantity plain text using the first homomorphic encryption key, and wherein the first sample quantity is a sample quantity of the first-party training samples, and the second sample quantity is a sample quantity of the second-party training samples.
16. The electronic device according to claim 15 , wherein the first-party training samples are randomly drawn by the first party from first-party sample data, and a quantity of samples randomly drawn from the first-party sample data is used as the first sample quantity; the second-party training samples are randomly drawn by the second party from second-party sample data, and a quantity of samples randomly drawn from the second-party sample data is used as the second sample quantity.