Sign-language translation

1 . An apparatus for sign-language translation, the apparatus comprising:

a memory including instructions; and

processing circuitry that, when in operation, is configured by the instructions to:

obtain a video segment that includes one or more signs in sign language;

invoke a first classifier trained to identify individual signs and produce a single symbol based on individual signs;

in response to the first classifier indicating that the video segment does not include a single sign, the processing circuitry is further configured to:

invoke a second classifier to divide the video segment into parts that correspond to discrete signs in a sequence of signs in the video segment;

in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the processing circuitry is further configured to:

invoke a third classifier on the parts of the video segment to produce first labels for the sequence of signs, the third classifier trained on regular continuous signs that are non-fingerspelled signs, each first label indicating either detection of a regular continuous sign or no detection of a regular continuous sign; and

invoke a fourth classifier on the parts of the video segment to produce second labels for the sequence of signs, the fourth classifier trained on fingerspelled signs, the fingerspelled signs corresponding to hand movements representing one or more individual letters, each second label indicating either detection of a fingerspelled sign or no detection of a fingerspelled sign; and

produce a sequence of symbols based on the sequence of signs by merging the first labels and the second labels.

2 . The apparatus of claim 1 , wherein the sequence of symbols is gloss.

3 . The apparatus of claim 1 , wherein the first classifier is trained with discrete single signs.

4 . The apparatus of claim 1 , wherein the third classifier is trained with only regular continuous signs.

5 . The apparatus of claim 1 , wherein the fourth classifier is trained with only fingerspelled signs.

6 . The apparatus of claim 1 , wherein, to merge the first labels and the second labels, the processing circuitry is configured to;

for each discrete sign, select a merged label from a respective first label of the first labels or a respective second label of the second labels, the merged label being selected based on:

one of the respective first label or the respective second label indicating detection when the other of the respective first label or the respective second label indicates no detection; or

one of the respective first label or the respective second label having a higher confidence score than the other of the respective first label or the respective second label.

7 . The apparatus of claim 1 , wherein the processing circuitry is further configured to:

obtain a second video segment that captures a second sign;

invoke the first classifier to determine that the second video segment includes a single sign; and

use the first classifier to produce a symbol from the second sign without invoking the second classifier, the third classifier, and the fourth classifier with respect to the second sign.

8 . The apparatus of claim 1 , wherein the first classifier, the third classifier, or the fourth classifier are configured to recognize custom signs.

9 . The apparatus of claim 8 , wherein a custom sign replaces a standard sign of sign language.

10 . The apparatus of claim 8 , wherein a custom sign is a new sign, and wherein the second classifier is trained with a few-shot technique to recognize the new sign.

11 . At least one non-transitory machine readable medium including instructions for sign-language translation, the instructions, when executed by processing circuitry, cause the processing circuitry to perform operations comprising:

obtaining a video segment that includes one or more signs in sign language;

invoking a first classifier trained to identify individual signs and produce a single symbol based on individual signs;

in response to the first classifier indicating that the video segment does not include a single sign, the operations further comprise:

invoking a second classifier to divide the video segment into parts that correspond to discrete signs in a sequence of signs in the video segment;

in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the operations further comprise:

invoking a third classifier on the parts of the video segment to produce first labels for the sequence of signs, the third classifier trained on regular continuous signs that are non-fingerspelled signs, each first label indicating either detection of a regular continuous sign or no detection of a regular continuous sign; and

invoking a fourth classifier on the parts of the video segment to produce second labels for the sequence of signs, the fourth classifier trained on fingerspelled signs that correspond to hand movements representing one or more individual letters, each second label indicating either detection of a fingerspelled sign or no detection of a fingerspelled sign; and

producing a sequence of symbols based on the sequence of signs by merging the first labels and the second labels.

12 . The at least one non-transitory machine readable medium of claim 11 , wherein the sequence of symbols is gloss.

13 . The at least one non-transitory machine readable medium of claim 11 , wherein the first classifier is trained with discrete single signs.

14 . The at least one non-transitory machine readable medium of claim 11 , wherein the third classifier is trained with only regular continuous signs.

15 . The at least one non-transitory machine readable medium of claim 11 , wherein the fourth classifier is trained with only fingerspelled signs.

16 . The at least one non-transitory machine readable medium of claim 11 , wherein merging the first labels and the second labels includes, for each discrete sign, selecting a merged label from a respective first label of the first labels or a respective second label of the second labels, the merged label being selected based on:

one of the respective first label or the respective second label indicating detection when the other of the respective first label or the respective second label indicates no detection;

one of the respective first label or the respective second label having a higher confidence score than the other of the respective first label or the respective second label.

17 . The at least one non-transitory machine readable medium of claim 11 ,

wherein the operations comprise:

obtaining a second video segment that captures a second sign;

invoking the first classifier to determine that the second video segment includes a single sign; and

using the first classifier to produce a symbol from the second sign without invoking the second classifier, the third classifier, and the fourth classifier with respect to the second sign.

18 . The at least one non-transitory machine readable medium of claim 11 , wherein the first classifier, the third classifier, or the fourth classifier are configured to recognize custom signs.

19 . A system for sign-language translation, the system comprising:

means for obtaining a video segment that includes one or more signs in sign language;

means for invoking a first classifier trained to identify individual signs and produce a single symbol based on individual signs;

means for invoking a second classifier to divide the video segment into parts that correspond to discrete signs in a sequence of signs in the video segment in response to the first classifier indicating that the video segment does not include a single sign;

means for invoking a third classifier on the parts to produce first labels for the sequence of signs in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the third classifier trained on regular continuous signs that are non-fingerspelled signs, each first label indicating either detection of a regular continuous sign or no detection of a regular continuous sign;

means for invoking a fourth classifier on the parts to produce second labels for the sequence of signs in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the fourth classifier trained on fingerspelled signs corresponding to hand movements representing one or more individual letters, each second label indicating either a detection of a fingerspelled sign or no detection of a fingerspelled sign; and

means for producing a sequence of symbols based on the sequence of signs by merging the first labels and the second labels.

20 . The system of claim 19 , comprising:

means for obtaining a second video segment that captures a second sign;

means for invoking the first classifier to determine that the second video segment includes a single sign; and

means for using the first classifier to produce a symbol from the second sign without invoking the second classifier, the third classifier, and the fourth classifier with respect to the second sign.