Object identification and labeling with sign language
Image data are received into a system. The image data include an object. A first gesture of a human hand is detected in the image data. The first gesture identifies the object for processing. A second gesture of the human hand is detected. The second gesture indicates a readiness to accept a third gesture, and then the third gesture is detected. The object is labeled with the third gesture, and the labeled object is stored in a database. The third gesture can be a sign from a sign language.
1 . A computer-implemented method comprising:
receiving image data corresponding to an environment, the environment comprising one or more objects;
detecting a first gesture of a human hand in the image data, the first gesture identifying an object from the one or more objects;
detecting the object;
detecting a second gesture of the human hand, the second gesture indicating a readiness to accept a third gesture;
detecting the third gesture of the human hand;
labeling the object with the third gesture; and
storing the labeled object in a database.
2 . The computer-implemented method of claim 1 , comprising classifying the object.
3 . The computer-implemented method of claim 2 , wherein the classifying the object comprises a one-shot learning algorithm.
4 . The computer-implemented method of claim 1 , comprising transmitting the labeled object to a computer display device for one or more of a sign language translation or a sign language instruction.
5 . The computer-implemented method of claim 1 , wherein the image data comprise a live video or a recorded video.
6 . The computer-implemented method of claim 1 , wherein the detecting the third gesture ceases when no motion of the human hand is detected in the image data.
7 . The computer-implemented method of claim 1 , wherein the storage of the labeled object in the database comprises replacing a prior label for the object.
8 . The computer-implemented method of claim 1 , wherein the detecting a human hand and the detecting the first, second and third gestures of the human hand comprise using a plurality of points on the human hand and on fingers of the human hand.
9 . The computer-implemented method of claim 1 , comprising prompting a user to record the first and second gestures; receiving from the user the first and second gestures, and storing the first and second gestures in the database.
10 . The computer-implemented method of claim 1 , wherein the human hand and the first, second and third gestures of the human hand are detected using computer vision and a machine learning algorithm.
11 . The computer-implemented method of claim 1 , comprising displaying on a computer display device a time period when the readiness to accept the third gesture begins.
12 . The computer-implemented method of claim 1 , wherein the first gesture comprises a finger pointing gesture.
13 . The computer-implemented method of claim 1 , wherein the third human gesture comprises a sign from a sign language.
14 . A non-transitory machine-readable medium comprising instructions that when executed by a processor execute a process comprising:
receiving image data corresponding to an environment, the environment comprising one or more objects;
detecting a first gesture of a human hand in the image data, the first gesture identifying an object from the one or more objects;
detecting the object;
detecting a second gesture of the human hand, the second gesture indicating a readiness to accept a third gesture;
detecting the third gesture of the human hand;
labeling the object with the third gesture; and
storing the labeled object in a database.
15 . The non-transitory machine-readable medium of claim 14 , comprising instructions for classifying the object; wherein the classifying the object comprises a one-shot learning algorithm.
16 . The non-transitory machine-readable medium of claim 14 , comprising instructions for transmitting the labeled object to a computer display device for one or more of a sign language translation or a sign language instruction.
17 . The non-transitory machine-readable medium of claim 14 , wherein the detecting the third gesture ceases when no motion of the human hand is detected in the image data.
18 . The non-transitory machine-readable medium of claim 14 , wherein the detecting a human hand and the detecting the first, second and third gestures of the human hand comprise using a plurality of points on the human hand and on fingers of the human hand.
19 . The non-transitory machine-readable medium of claim 14 , wherein the human hand and the first, second and third gestures of the human hand are detected using computer vision and a machine learning algorithm.
20 . A system comprising:
a computer processor; and
a computer memory coupled to the computer processor;
wherein the computer processor and computer memory are operable for:
receiving image data corresponding to an environment, the environment comprising one or more objects;
detecting a first gesture of a human hand in the image data, the first gesture identifying an object from the one or more objects;
detecting the object;
detecting a second gesture of the human hand, the second gesture indicating a readiness to accept a third gesture;
detecting the third gesture of the human hand;
labeling the object with the third gesture; and
storing the labeled object in a database.