Method of and system for training machine learning algorithm to generate text summary

1. A method for training an extractive machine learning algorithm (MLA) to generate extractive summaries of text documents, the method being executed by a processor, the processor having access to the extractive MLA, the extractive MLA comprising a sentence encoder, a document encoder and a decoder, the method comprising:

receiving a given reference document, the given reference document comprising a plurality of reference sentences;

receiving a reference summary of the given reference document, the reference summary comprising a set of reference summary sentences, the set of reference summary sentences being a subset of the plurality of reference sentences of the given reference document;

training the extractive MLA to generate an extractive summary, said training comprising:

encoding, using the sentence encoder, the plurality of reference sentences to obtain an associated plurality of reference sentence representations;

encoding, using the document encoder, the associated plurality of reference sentence representations to obtain a document representation;

extracting, using the decoder and based on the associated plurality of reference sentence representations and the document representation, a first reference sentence of the plurality of reference sentences to obtain a first extracted sentence of the extractive summary; and

updating, based on the first extracted sentence and the set of reference summary sentences, a given parameter of the extractive MLA to obtain an updated given parameter; and

outputting the trained extractive MLA, the trained extractive MLA comprising the updated given parameter.

2. The method of claim 1 , wherein said training is performed on a set of reference documents by using an associated set of reference summaries as a target, the set of reference documents comprising the given reference document and the set of reference summaries comprising the reference summary.

3. The method of claim 2 , further comprising, prior to said receiving the reference summary of the given reference document, the reference summary comprising the set of reference summary sentences:

receiving a first summary of the given reference document, the first summary comprising a set of first summary sentences, at least one first summary sentence not being included in the plurality of reference sentences;

determining a respective similarity score between the at least one first summary sentence and at least one reference sentence of the plurality of reference sentences; and

extracting, based on the respective similarity score, the at least one reference sentence of the plurality of reference sentences to obtain the reference summary of the given reference document.

4. The method of claim 3 , further comprising, prior to said receiving the given reference document comprising the plurality of reference sentences:

receiving a size of the plurality of reference sentences; and wherein

said receiving the given reference document comprising the plurality of reference sentences is in response to the size of the plurality of reference sentences being above a predetermined threshold.

5. The method of claim 4 , wherein the sentence encoder and the document encoder each comprises a respective bi-directional long short term memory (LSTM) network.

6. The method of claim 5 , wherein the decoder comprises an autoregressive LSTM.

7. The method of claim 6 , further comprising, after said extracting, using the decoder and based on the associated plurality of reference sentence representations and the document representation, the first extracted sentence of the set of reference summary sentences:

extracting, using the decoder, based on the first extracted sentence and the associated plurality of sentence representations, a second first reference sentence of the plurality of reference sentences to obtain a second extracted sentence of the extractive summary.

8. The method of claim 7 , wherein said generating is performed recursively.

9. The method of claim 8 , wherein

the decoder comprises a hidden state, the hidden state comprising the first extracted sentence and the associated plurality of sentence representations; and wherein

said extracting comprises using an attention mechanism on the hidden state of the decoder over a set of document representations comprising the document representation to obtain the extractive summary comprising the first extracted sentence and the second extracted sentence.

10. The method of claim 1 , wherein said training comprises minimizing a cross-entropy of picking a correct sentence at each decoder time step.

11. A system for training an extractive machine learning algorithm (MLA) to generate extractive summaries, the system comprising:

a processor;

a non-transitory storage medium operatively connected to the processor, the non-transitory storage medium comprising computer readable instructions; and

the processor having access to the extractive MLA, the extractive MLA comprising a sentence encoder, a document encoder and a decoder, the processor, upon executing the computer readable instructions, being configured for:

receiving a given reference document, the given reference document comprising a plurality of reference sentences;

receiving a reference summary of the given reference document, the reference summary comprising a set of reference summary sentences, the set of reference summary sentences being a subset of the plurality of reference sentences of the given reference document;

training the extractive MLA to generate an extractive summary, said training comprising:

encoding, using the sentence encoder, the plurality of reference sentences to obtain an associated plurality of reference sentence representations;

encoding, using the document encoder, the associated plurality of reference sentence representations to obtain a document representation;

extracting, using the decoder and based on the associated plurality of reference sentence representations and the document representation, a first reference sentence of the plurality of reference sentences to obtain a first extracted sentence of the extractive summary; and

updating, based on the first extracted sentence and the set of reference summary sentences, a given parameter of the extractive MLA to obtain an updated given parameter; and

outputting the trained extractive MLA, the trained extractive MLA comprising the updated given parameter.

12. The system of claim 11 , wherein said training is performed on a set of reference documents by using an associated set of reference summaries as a target, the set of reference documents comprising the given reference document and the set of reference summaries comprising the reference summary.

13. The system of claim 12 , wherein the processor is further configured for, prior to said receiving the reference summary of the given reference document, the reference summary comprising the set of reference summary sentences:

receiving a first summary of the given reference document, the first summary comprising a set of first summary sentences, at least one first summary sentence not being included in the plurality of reference sentences;

determining a respective similarity score between the at least one first summary sentence and at least one reference sentence of the plurality of reference sentences; and

extracting, based on the respective similarity score, the at least one reference sentence of the plurality of reference sentences to obtain the reference summary of the given reference document.

14. The system of claim 13 , wherein the processor is further configured for, prior to said receiving the given reference document comprising the plurality of reference sentences:

receiving a size of the plurality of reference sentences; and wherein

said receiving the given reference document comprising the plurality of reference sentences is in response to the size of the plurality of reference sentences being above a predetermined threshold.

15. The system of claim 14 , wherein the sentence encoder and the document encoder each comprises a respective bi-directional long short term memory (LSTM) network.

16. The system of claim 15 , wherein the decoder comprises an autoregressive LSTM.

17. The system of claim 16 , wherein the processor is further configured for, after said extracting, using the decoder and based on the associated plurality of reference sentence representations and the document representation, the first extracted sentence of the set of reference summary sentences:

extracting, using the decoder, based on the first extracted sentence and the associated plurality of sentence representations, a second first reference sentence of the plurality of reference sentences to obtain a second extracted sentence of the extractive summary.

18. The system of claim 17 , wherein said generating is performed recursively.

19. The system of claim 18 , wherein

the decoder comprises a hidden state, the hidden state comprising the first extracted sentence and the associated plurality of sentence representations; and wherein

said extracting comprises using an attention mechanism on the hidden state of the decoder over a set of document representations comprising the document representation to obtain the extractive summary comprising the first extracted sentence and the second extracted sentence.

20. The system of claim 19 , wherein said training comprises minimizing a cross-entropy of picking a correct sentence at each decoder time step.