Method and system for recognizing interactive operation, display terminal, and storage medium

1 . A method for recognizing an interactive operation, the method being applied to a display terminal in a system for recognizing an interactive operation and comprising:

displaying an interactive interface;

acquiring control information of the interactive interface from a smart ring in the system, wherein the control information comprises finger motion information and level information, and the level information indicates a pinch state of a finger wearing the smart ring and a finger associated with the finger wearing the smart ring;

determining an interactive operation corresponding to the control information according to an interaction mode for the interactive interface, the control information and a recognition model library, wherein the recognition model library comprises a recognition model corresponding to an interaction mode of at least one interaction mode; and

displaying, on the interactive interface, the interactive operation corresponding to the control information;

wherein determining the interactive operation corresponding to the control information according to the interaction mode for the interactive interface, the control information and the recognition model library comprises:

searching the recognition model library to determine a target recognition model matching the interaction mode, wherein the interaction mode comprises at least one of a gesture mode, a write mode or a touch mode; and

determining the interactive operation corresponding to the control information according to the target recognition model, the finger motion information and the level information.

2 . The method of claim 1 , wherein

in response to the interaction mode being the gesture mode, the target recognition model is a gesture classification model; and

determining the interactive operation corresponding to the control information according to the target recognition model, the finger motion information and the level information comprises:

inputting the level information and the finger motion information into the gesture classification model to determine a gesture action formed by the finger wearing the smart ring and the finger associated with the finger wearing the smart ring;

searching a preset gesture-function mapping table to determine a control function corresponding to the gesture action; and

using the control function as the interactive operation corresponding to the control information.

3 . The method of claim 1 , wherein

in response to the interaction mode being the write mode, the target recognition model is a character recognition model; and

determining the interactive operation corresponding to the control information according to the target recognition model, the finger motion information and the level information comprises:

inputting the level information and the finger motion information into the character recognition model to recognize written content corresponding to the finger wearing the smart ring and the finger associated with the finger wearing the smart ring which satisfy a writing posture condition; and

using the written content as the interactive operation corresponding to the control information.

4 . The method of claim 1 , wherein

in response to the interaction mode being the touch mode, the target recognition model is a regression recognition model; and

determining the interactive operation corresponding to the control information according to the target recognition model, the finger motion information and the level information comprises:

inputting the level information and the finger motion information into the regression recognition model to determine a finger swipe trajectory of the finger wearing the smart ring on the finger associated with the finger wearing the smart ring; and

using the finger swipe trajectory as the interactive operation corresponding to the control information.

5 . The method of claim 4 , wherein displaying, on the interactive interface, the interactive operation corresponding to the control information comprises:

converting the finger swipe trajectory into a pointer movement trajectory of a pointer in the interactive interface, and controlling the pointer to move along the pointer movement trajectory.

6 . A system for recognizing an interactive operation, comprising a smart ring and a display terminal;

wherein the display terminal is configured to display an interactive interface;

wherein the smart ring is configured to acquire control information of the interactive interface, wherein the control information comprises finger motion information and level information, and the level information indicates a pinch state of a finger wearing the smart ring and a finger associated with the finger wearing the smart ring;

wherein the display terminal is configured to acquire the control information of the interactive interface from the smart ring and determine an interactive operation corresponding to the control information according to an interaction mode for the interactive interface, the control information and a recognition model library, wherein the recognition model library comprises a recognition model corresponding to an interaction mode of at least one interaction mode; and

wherein the display terminal is configured to display, on the interactive interface, the interactive operation corresponding to the control information;

wherein the display terminal is configured to determine the interactive operation corresponding to the control information according to the interaction mode for the interactive interface, the control information and the recognition model library by:

searching the recognition model library to determine a target recognition model matching the interaction mode, wherein the interaction mode comprises at least one of a gesture mode, a write mode or a touch mode; and

determining the interactive operation corresponding to the control information according to the target recognition model, the finger motion information and the level information.

7 . A display terminal, comprising:

at least one processor; and

a memory communicatively connected to the at least one processor;

wherein the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform:

displaying an interactive interface;

acquiring control information of the interactive interface from a smart ring in a system for recognizing an interactive operation, wherein the control information comprises finger motion information and level information, and the level information indicates a pinch state of a finger wearing the smart ring and a finger associated with the finger wearing the smart ring;

determining an interactive operation corresponding to the control information according to an interaction mode for the interactive interface, the control information and a recognition model library, wherein the recognition model library comprises a recognition model corresponding to an interaction mode of at least one interaction mode; and

displaying, on the interactive interface, the interactive operation corresponding to the control information;

wherein the at least one processor is further configured to determine the interactive operation corresponding to the control information by:

searching the recognition model library to determine a target recognition model matching the interaction mode, wherein the interaction mode comprises at least one of a gesture mode, a write mode or a touch mode; and

determining the interactive operation corresponding to the control information according to the target recognition model, the finger motion information and the level information.

8 . The display terminal of claim 7 , wherein

in response to the interaction mode being the gesture mode, the target recognition model is a gesture classification model; and

the at least one processor is further configured to determine the interactive operation corresponding to the control information by:

inputting the level information and the finger motion information into the gesture classification model to determine a gesture action formed by the finger wearing the smart ring and the finger associated with the finger wearing the smart ring;

searching a preset gesture-function mapping table to determine a control function corresponding to the gesture action; and

using the control function as the interactive operation corresponding to the control information.

9 . The display terminal of claim 7 , wherein

in response to the interaction mode being the write mode, the target recognition model is a character recognition model; and

the at least one processor is further configured to determine the interactive operation corresponding to the control information by:

inputting the level information and the finger motion information into the character recognition model to recognize written content corresponding to the finger wearing the smart ring and the finger associated with the finger wearing the smart ring which satisfy a writing posture condition; and

using the written content as the interactive operation corresponding to the control information.

10 . The display terminal of claim 7 , wherein

in response to the interaction mode being the touch mode, the target recognition model is a regression recognition model; and

the at least one processor is further configured to determine the interactive operation corresponding to the control information by:

inputting the level information and the finger motion information into the regression recognition model to determine a finger swipe trajectory of the finger wearing the smart ring on the finger associated with the finger wearing the smart ring; and

using the finger swipe trajectory as the interactive operation corresponding to the control information.

11 . The display terminal of claim 10 , wherein the at least one processor is further configured to display, on the interactive interface, the interactive operation corresponding to the control information by:

converting the finger swipe trajectory into a pointer movement trajectory of a pointer in the interactive interface, and controlling the pointer to move along the pointer movement trajectory.

12 . A non-transitory computer-readable storage medium storing computer instructions configured to, when executed by a processor, cause the processor to perform:

displaying an interactive interface;

acquiring control information of the interactive interface from a smart ring in a system for recognizing an interactive operation, wherein the control information comprises finger motion information and level information, and the level information indicates a pinch state of a finger wearing the smart ring and a finger associated with the finger wearing the smart ring;

determining an interactive operation corresponding to the control information according to an interaction mode for the interactive interface, the control information and a recognition model library, wherein the recognition model library comprises a recognition model corresponding to an interaction mode of at least one interaction mode; and

displaying, on the interactive interface, the interactive operation corresponding to the control information;

wherein the processor is further configured to determine the interactive operation corresponding to the control information by:

searching the recognition model library to determine a target recognition model matching the interaction mode, wherein the interaction mode comprises at least one of a gesture mode, a write mode or a touch mode; and

determining the interactive operation corresponding to the control information according to the target recognition model, the finger motion information and the level information.

13 . The non-transitory computer-readable storage medium of claim 12 , wherein

in response to the interaction mode being the gesture mode, the target recognition model is a gesture classification model; and

the processor is further configured to determine the interactive operation corresponding to the control information by:

inputting the level information and the finger motion information into the gesture classification model to determine a gesture action formed by the finger wearing the smart ring and the finger associated with the finger wearing the smart ring;

searching a preset gesture-function mapping table to determine a control function corresponding to the gesture action; and

using the control function as the interactive operation corresponding to the control information.

14 . The non-transitory computer-readable storage medium of claim 12 , wherein

in response to the interaction mode being the write mode, the target recognition model is a character recognition model; and

the processor is further configured to determine the interactive operation corresponding to the control information by:

inputting the level information and the finger motion information into the character recognition model to recognize written content corresponding to the finger wearing the smart ring and the finger associated with the finger wearing the smart ring which satisfy a writing posture condition; and

using the written content as the interactive operation corresponding to the control information.

15 . The non-transitory computer-readable storage medium of claim 12 , wherein

in response to the interaction mode being the touch mode, the target recognition model is a regression recognition model; and

the processor is further configured to determine the interactive operation corresponding to the control information by:

inputting the level information and the finger motion information into the regression recognition model to determine a finger swipe trajectory of the finger wearing the smart ring on the finger associated with the finger wearing the smart ring; and

using the finger swipe trajectory as the interactive operation corresponding to the control information.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein the processor is further configured to display, on the interactive interface, the interactive operation corresponding to the control information by:

converting the finger swipe trajectory into a pointer movement trajectory of a pointer in the interactive interface, and controlling the pointer to move along the pointer movement trajectory.