Systems and methods for program synthesis
Embodiments described herein provide a reinforcement learning based framework engaging pretrained language models (LMs) for program synthesis tasks. Specifically, the framework adopts a training strategy that optimizes pretrained LMs for program synthesis tasks in an actor-critic approach.
1. A method of program synthesis, the method comprising:
receiving, via an input interface, a problem specification at a language model pretrained for program synthesis;
extracting one or more unit test input-output pairs from the problem specification;
generating, by the language model, a plurality of program samples from the problem specification;
applying one or more unit tests to the plurality of program samples based on the one or more unit test input-output pairs;
determining, from the plurality of program samples, a first set of program samples that pass the one or more unit tests;
re-generating, by the language model, one or more refined program samples based on information from the first set of program samples, including:
selecting, via critic scoring, one or more sub-sequences from the first set of program samples, each sub-sequence being a truncated version of a program sample; and
re-generating, by the language model, subsequent tokens conditioned on the one or more sub-sequences to form one or more refined program samples.
2. The method of claim 1 , wherein the one or more sub-sequences are selected by:
determining, by a critic model, a value to each token of a first program sample in the first set based on a predicted probability that a subsequence up to the respective token pass the one or more unit tests;
determining a particular token of the first program sample having a highest value; and
selecting a sub-sequence of the first program sample up to the particular token as a sub-sequence.
3. The method of claim 2 , further comprising:
determining that the selected sub-sequence contains a particular token up to which a corresponding sub-sequence has a higher probability to fail than to pass the one or more unit tests; and
chopping the selected sub-sequence at the particular token.
4. The method of claim 1 , further comprising:
determining, from the plurality of program samples, a second set of program samples that are unsuccessful, comprising at least one of compile error, runtime error and failure to pass at least one of the unit tests.
5. The method of claim 4 , further comprising:
determining, by a critic model, a value to a second program sample in the second set based on a predicted probability that the second program sample pass the one or more unit tests;
selecting a subset of program samples with highest values from the second set;
forming an input sequence by concatenating the problem specification, a selected program sample and error information corresponding to the selected program sample;
generating, by a program repair model, a repaired program sample based on the input sequence.
6. The method of claim 5 , wherein the error information comprises any of:
a unit test outcome corresponding to the selected program sample; and
an error subtype during compiling or runtime of the selected program sample.
7. The method of claim 5 , wherein the program repair model is a sequence-to-sequence model.
8. The method of claim 5 , wherein the program repair model is trained by a training objective comparing program samples that fail the unit tests and a ground-truth program corresponding to the problem specification, conditioned on a unit test outcome and/or an error subtype corresponding to the program samples.
9. The method of claim 4 , further comprising:
generating, by a program repair model, one or more repaired program samples using error information from the second set of program samples; and
refining, by the language model, the one or more repaired program samples.
10. A system of program synthesis, the system comprising:
an input interface that receives a problem specification at a language model pretrained for program synthesis;
a memory storing a plurality of processor-executable instructions;
a processor that reads and executes the plurality of processor-executable instructions to perform operations comprising:
extracting one or more unit test input-output pairs from the problem specification;
generating, by the language model, a plurality of program samples from the problem specification;
applying one or more unit tests to the plurality of program samples based on the one or more unit test input-output pairs;
determining, from the plurality of program samples, a first set of program samples that pass the one or more unit tests;
re-generating, by the language model, one or more refined program samples based on information from the first set of program samples, including:
selecting, via critic scoring, one or more sub-sequences from the first set of program samples, each sub-sequence being a truncated version of a program sample; and
re-generating, by the language model, subsequent tokens conditioned on the one or more sub-sequences to form one or more refined program samples.
11. The system of claim 10 , wherein the one or more sub-sequences are selected by:
determining, by a critic model, a value to each token of a first program sample in the first set based on a predicted probability that a subsequence up to the respective token pass the one or more unit tests;
determining a particular token of the first program sample having a highest value; and
selecting a sub-sequence of the first program sample up to the particular token as a sub-sequence.
12. The system of claim 11 , wherein the operations further comprise:
determining that the selected sub-sequence contains a particular token up to which a corresponding sub-sequence has a higher probability to fail than to pass the one or more unit tests; and
chopping the selected sub-sequence at the particular token.
13. The system of claim 10 , wherein the operations further comprise:
determining, from the plurality of program samples, a second set of program samples that are unsuccessful, comprising at least one of compile error, runtime error and failure to pass at least one of the unit tests.
14. The system of claim 13 , wherein the operations further comprise:
determining, by a critic model, a value to a second program sample in the second set based on a predicted probability that the second program sample pass the one or more unit tests;
selecting a subset of program samples with highest values from the second set;
forming an input sequence by concatenating the problem specification, a selected program sample and error information corresponding to the selected program sample;
generating, by a program repair model, a repaired program sample based on the input sequence.
15. The system of claim 14 , wherein the error information comprises any of:
a unit test outcome corresponding to the selected program sample; and
an error subtype during compiling or runtime of the selected program sample.
16. The system of claim 14 , wherein the program repair model is trained by a training objective comparing program samples that fail the unit tests and a ground-truth program corresponding to the problem specification, conditioned on a unit test outcome and/or an error subtype corresponding to the program samples.
17. The system of claim 13 , wherein the operations further comprise:
generating, by a program repair model, one or more repaired program samples using error information from the second set of program samples; and
refining, by the language model, the one or more repaired program samples.
18. A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions for program synthesis, the processor-executable instructions being executed by one or more processors to perform operations comprising:
receiving, via an input interface, a problem specification at a language model pretrained for program synthesis;
extracting one or more unit test input-output pairs from the problem specification;
generating, by the language model, a plurality of program samples from the problem specification;
applying one or more unit tests to the plurality of program samples based on the one or more unit test input-output pairs;
determining, from the plurality of program samples, a first set of program samples that pass the one or more unit tests;
re-generating, by the language model, one or more refined program samples based on information from the first set of program samples, including:
selecting, via critic scoring, one or more sub-sequences from the first set of program samples, each sub-sequence being a truncated version of a program sample; and
re-generating, by the language model, subsequent tokens conditioned on the one or more sub-sequences to form one or more refined program samples.
19. The non-transitory processor-readable storage medium of claim 18 , wherein the one or more sub-sequences are selected by:
determining, by a critic model, a value to each token of a first program sample in the first set based on a predicted probability that a subsequence up to the respective token pass the one or more unit tests;
determining a particular token of the first program sample having a highest value; and
selecting a sub-sequence of the first program sample up to the particular token as a sub-sequence.
20. The method of claim 19 , wherein the operations further comprise:
determining that the selected sub-sequence contains a particular token up to which a corresponding sub-sequence has a higher probability to fail than to pass the one or more unit tests; and
chopping the selected sub-sequence at the particular token.