Large language models for creating a multi-lingual, low-resource code translation dataset
One or more unit-test cases are generated from a monolingual code corpus and the generated unit-test cases are filtered to generate a corpus of unit-test cases which have acceptability scores exceeding one or more predefined thresholds. One or more of the code samples of the monolingual code corpus are translated from a source language to a target language using a pretrained Large Language Model and the generated unit-test cases are translated from the source language to the target language. The LLM-translated code samples are validated using the translated unit-test cases and a parallel-data training corpus comprising the LLM-translated code samples that pass the validation is created. The pretrained large language model (LLM) is fine-tuned using the parallel-data training corpus, a given code segment is translated using the fine-tuned large language model (LLM), the translated given code segment is tested and the tested given code segment is deployed.
1 . A method comprising:
generating, using at least one hardware processor, one or more unit-test cases from a monolingual code corpus;
filtering, using the at least one hardware processor, the generated unit-test cases to generate a corpus of unit-test cases which have acceptability scores exceeding one or more predefined thresholds;
translating, using the at least one hardware processor, one or more code samples of the monolingual code corpus from a source language to a target language using a pretrained large language model (LLM);
translating, using the at least one hardware processor, the generated unit-test cases from the source language to the target language;
validating, using the at least one hardware processor, the LLM-translated code samples using the translated unit-test cases;
creating, using the at least one hardware processor, a parallel-data training corpus comprising the LLM-translated code samples that pass the validation;
fine-tuning, using the at least one hardware processor, the pretrained large language model (LLM) using the parallel-data training corpus;
translating, using the at least one hardware processor, a given code segment using the fine-tuned large language model (LLM);
testing, using the at least one hardware processor, the translated given code segment; and
facilitating, using the at least one hardware processor, deployment of the tested given code segment in the target language.
2 . The method of claim 1 , further comprising:
collecting one or more the one or more of the code samples in one or more programming languages; and
extracting the monolingual code corpus from the collection of code samples.
3 . The method of claim 2 , wherein the filtering of the generated unit-test cases further comprises retaining the generated unit-test cases that have toolkit-metrics exceeding the one or more predefined thresholds.
4 . The method of claim 2 , further comprising performing code translation of the given segment of code in the source language to the target language using the Large Language Model.
5 . The method of claim 2 , wherein the large language model is a multi-billion parameter machine learning model pretrained in an unsupervised manner.
6 . The method of claim 2 , wherein the parallel-data training corpus comprises functionally-equivalent implementation of logic in multiple programming languages.
7 . The method of claim 2 , wherein the generated unit-test cases are expected input-output pairs with assert statements.
8 . The method of claim 2 , further comprising repeating the translating of the one or more of the code samples of the monolingual code corpus, the validating, and the creating operations for all code samples of the monolingual code corpus.
9 . The method of claim 2 , further comprising retraining the large language model using the parallel-data training corpus.
10 . The method of claim 2 , further comprising repeating the translating of the one or more of the code samples of the monolingual code corpus, the validating, and the creating operations for code samples of the monolingual code corpus that failed the validation operation.
11 . The method of claim 2 , further comprising verifying that the code samples of the monolingual code corpus and the translated code are functionally equivalent using the translated unit-test cases.
12 . The method of claim 1 , wherein the acceptability scores are one or more of a coverage score and a mutation score.
13 . The method of claim 1 , further comprising running the deployed tested given code segment.
14 . A computer program product, comprising:
one or more tangible computer-readable storage media and program instructions stored on at least one of the one or more tangible computer-readable storage media, the program instructions executable by a processor, the program instructions comprising:
generating one or more unit-test cases from a monolingual code corpus;
filtering the generated unit-test cases to generate a corpus of unit-test cases which have acceptability scores exceeding one or more predefined thresholds;
translating one or more code samples of the monolingual code corpus from a source language to a target language using a pretrained Large Language Model (LLM);
translating the generated unit-test cases from the source language to the target language;
validating the LLM-translated code samples using the translated unit-test cases;
creating a parallel-data training corpus comprising the LLM-translated code samples that pass the validation;
fine-tuning the pretrained large language model (LLM) using the parallel-data training corpus;
translating a given code segment using the fine-tuned large language model (LLM);
testing the translated given code segment; and
facilitating deployment of the tested given code segment in the target language.
15 . An apparatus comprising:
a memory; and
at least one processor, coupled to said memory, and operative to perform operations comprising:
generating one or more unit-test cases from a monolingual code corpus;
filtering the generated unit-test cases to generate a corpus of unit-test cases which have acceptability scores exceeding one or more predefined thresholds;
translating one or more code samples of the monolingual code corpus from a source language to a target language using a pretrained Large Language Model (LLM);
translating the generated unit-test cases from the source language to the target language using a rules-based translator;
validating the LLM-translated code samples using the translated unit-test cases; and
creating a parallel-data training corpus comprising the LLM-translated code samples that pass the validation;
fine-tuning the pretrained large language model (LLM) using the parallel-data training corpus;
translating a given code segment using the fine-tuned large language model (LLM);
testing the translated given code segment; and
facilitating deployment of the tested given code segment in the target language.
16 . The apparatus of claim 15 , the operations further comprising:
collecting one or more code samples in one or more programming languages; and
extracting the monolingual code corpus from the collection of the one or more of the code samples.
17 . The apparatus of claim 16 , wherein the filtering of the generated unit-test cases further comprises retaining the generated unit-test cases that have toolkit-metrics exceeding the one or more predefined thresholds.
18 . The apparatus of claim 16 , the operations further comprising performing code translation of the given segment of code in the source language to the target language using the large language model.
19 . The apparatus of claim 16 , the operations further comprising retraining the large language model using the parallel-data training corpus.
20 . The apparatus of claim 16 , the operations further comprising verifying that the code samples of the monolingual code corpus and the translated code are functionally equivalent using the translated unit-test cases.