Identifying the translatability of hard-coded strings in source code via POS tagging
A method for identifying hard-coded strings in source code is disclosed. In one embodiment, such a method parses source code and associated localization resource files to identify hard-coded strings and their associated context. The method provides a confidence score for each hard-coded string that indicates whether the hard-coded string is translatable or non-translatable. Based on the confidence score for each hard-coded string, the method transforms each hard-coded string into a single equivalence word. The method then prepares training data by tagging the hard-coded strings in the source code and associated localization resource files as one of translatable and non-translatable. The method then trains a parts-of-speech (POS) tagging model using the training data. At runtime, the method fetches potential hard-coded strings and tags each hard-coded string as one of translatable and non-translatable using the POS tagging model. A corresponding system and computer program product are also disclosed.
1 . A method for localizing software, the method comprising:
preparing a set of training data, wherein the preparing comprises:
parsing source code and associated localization resource files to identify hard-coded strings and their associated context;
providing a confidence score for each hard-coded string indicating a likelihood of the hard-coded string being translatable, wherein the confidence score is determined at least in part based on the associated context;
based on the confidence score for each hard-coded string, transforming each hard-coded string into a single equivalence word from a set of equivalence words mapped to confidence scores;
tagging the hard-coded strings in the source code and associated localization resource files as one of translatable and non-translatable, wherein the tagging tags the single equivalence word of each hard-coded string; and
building positive training data using resourced-out strings from the hard-coded strings tagged as translatable;
training a parts-of-speech (POS) tagging model using the training data;
at runtime, by the POS tagging model, locating translatable hard-coded strings in source code of the software, wherein the locating comprises fetching potential hard-coded strings and their associated contexts and tagging each potential hard-coded string as one of translatable and non-translatable using the POS tagging model;
translating the located translatable hard-coded strings; and
adapting elements of a user interface using the translated hard-coded strings.
2 . The method of claim 1 , wherein parsing the source code and associated localization resource files comprises utilizing a first layer parser to retrieve resourced-out strings and their associated context from the source code and associated localization resource files.
3 . The method of claim 1 , wherein parsing the source code and associated localization resource files comprises utilizing a second layer parser to retrieve intended hard-coded strings and their associated context from the source code.
4 . The method of claim 1 , wherein the context comprises at least one word preceding the hard-coded strings.
5 . The method of claim 1 , wherein the POS tagging model uses one of probabilistic methods and deep learning methods to tag the potential hard-coded strings as one of translatable and non-translatable.
6 . The method of claim 1 , wherein preparing the training data further comprises building negative training data using intended hard-coded strings and their associated context.
7 . A computer program product for localizing software, the computer program product comprising a computer-readable storage medium having computer-usable program code embodied therein, the computer-usable program code configured to perform the following when executed by at least one processor:
prepare a set of training data, wherein the preparing the set of training data comprises:
parsing source code and associated localization resource files to identify hard-coded strings and their associated contexts;
based on the associated contexts, provide a confidence score for each hard-coded string indicating a likelihood of the hard-coded string being translatable, wherein the confidence score is determined at least in part based on the associated context;
based on the confidence score for each hard-coded string, transforming each hard-coded string into a single equivalence word from a set of equivalence words mapped to confidence scores;
tagging the hard-coded strings in the source code and associated localization resource files as one of translatable and non-translatable, wherein the tagging tags the single equivalence word of each hard-coded string; and
build positive training data using resourced-out strings from the hard-coded strings tagged as translatable;
train a parts-of-speech (POS) tagging model using the training data;
at runtime, by the POS tagging model, locate translatable hard-coded strings in source code of the software, wherein the locating comprises fetching potential hard-coded strings and their associated contexts and tagging each potential hard-coded string as one of translatable and non-translatable using the POS tagging model:
translate the located translatable hard-coded strings; and
adapt elements of a user interface using the translated hard-coded strings.
8 . The computer program product of claim 7 , wherein parsing the source code and associated localization resource files comprises utilizing a first layer parser to retrieve the resourced-out strings and their associated context from the source code and associated localization resource files.
9 . The computer program product of claim 7 , wherein parsing the source code and associated localization resource files comprises utilizing a second layer parser to retrieve intended hard-coded strings and their associated context from the source code.
10 . The computer program product of claim 7 , wherein the context comprises at least one word preceding the hard-coded strings.
11 . The computer program product of claim 7 , wherein the POS tagging model uses one of probabilistic methods and deep learning methods to tag the potential hard-coded strings as one of translatable and non-translatable.
12 . The computer program product of claim 7 , wherein preparing the training data further comprises building negative training data using intended hard-coded strings and their associated context.
13 . A system for localizing software, the system comprising:
at least one processor; and
at least one memory device operably coupled to the at least one processor and storing instructions for execution on the at least one processor, the instructions causing the at least one processor to:
prepare a set of training data, wherein the instructions causing the at least one processor to prepare the set of training data comprise instructions causing the at least one processor to:
parse source code and associated localization resource files to identify hard-coded strings and their associated context;
provide a confidence score for each hard-coded string indicating a likelihood of the hard-coded string being translatable, wherein the confidence score is determined at least in part based on the associated context;
based on the confidence score for each hard-coded string, transform each hard-coded string into a single equivalence word from a set of equivalence words mapped to confidence scores;
tag the hard-coded strings in the source code and associated localization resource files as one of translatable and non-translatable, wherein the tagging tags the single equivalence word of each hard-coded string; and
build positive training data using resourced-out strings from the hard-coded strings tagged as translatable;
train a parts-of-speech (POS) tagging model using the training data;
at runtime, by the POS tagging model, locate translatable hard-coded strings in source code of the software, wherein the locating comprises fetching potential hard-coded strings and their associated contexts and tagging each potential hard-coded string as one of translatable and non-translatable using the POS tagging model;
translate the located translatable hard-coded strings; and
adapt elements of a user interface using the translated hard-coded strings.
14 . The system of claim 13 , wherein parsing the source code and associated localization resource files comprises utilizing a first layer parser to retrieve the resourced-out strings and their associated context from the source code and associated localization resource files.
15 . The system of claim 13 , wherein parsing the source code and associated localization resource files comprises utilizing a second layer parser to retrieve intended hard-coded strings and their associated context from the source code.
16 . The system of claim 13 , wherein the context comprises at least one word preceding the hard-coded strings.
17 . The system of claim 13 , wherein the POS tagging model uses one of probabilistic methods and deep learning methods to tag the potential hard-coded strings as one of translatable and non-translatable.
18 . The system of claim 13 , wherein preparing the training data further comprises building negative training data using intended hard-coded strings and their associated context.