IP Library Granted Patent US 11,778,263
Granted Patent B2
US 11,778,263 · App. 17/783,267 · Granted Oct 3, 2023

Live streaming video interaction method and apparatus, and computer device

Inventors: Zixin Tang (Shanghai, CN); Dewei Xue (Shanghai, CN)
Assignee: SHANGHAI HODE INFORMATION TECHNOLOGY CO., LTD.
H04N21/431H04N21/2187H04N21/42204H04N21/44008H04N21/4788H04N21/47205
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,778,263
App. No.
17/783,267
Granted
Oct 3, 2023
Kind
B2
Abstract

The present application discloses techniques of interacting with live videos. The techniques comprise obtaining a streaming video of a live streamer and images of a user captured in real time by a user terminal, and displaying the streaming video and the image of the user in a same video play box; obtaining and recognizing a first gesture of a user in the images of the user, and comparing the first gesture with a second gesture included in a preset table, wherein the preset table comprises information indicating corresponding relationships between gestures and special effects; obtaining a first special effect corresponding to the second gesture by querying the preset table when the first gesture matches with the second gesture; and displaying the first special effect in the video play box.

Claims (80)

1. A method of interacting with live videos, comprising:

obtaining a streaming video of a live streamer;

obtaining images of a user captured in real time by a user terminal;

displaying the streaming video and the images of the user in a same video play box;

obtaining and recognizing a first gesture of the user in the images;

comparing the first gesture with a second gesture included in a preset table, wherein the preset table comprises information indicating corresponding relationships between gestures and special effects;

obtaining a first special effect corresponding to the second gesture by querying the preset table in response to determining that the first gesture matches the second gesture; and

displaying the first special effect in the video play box.

2. The method of claim 1 , wherein the displaying the streaming video and the images of the user in a same video play box comprises:

playing the streaming video and the images of the user in a first part and a second part of the video play box respectively in a seamless stitching manner.

3. The method of claim 1 , wherein the displaying the first special effect in the video play box comprises:

obtaining a first palm position associated with the first gesture;

obtaining a first face position associated with a face of the live streamer in the streaming video; and

displaying the first special effect by extending from the first palm position as a start point to the first face position as an end point.

4. The method of claim 3 , further comprising:

monitoring whether a hand palm associated with the first gesture moves;

in response to determining that the hand palm associated with the first gesture moves, obtaining a second palm position associated with the first gesture after a movement of the hand palm; and

changing the start point of the first special effect from the first palm position to the second palm position.

5. The method of claim 3 , further comprising:

monitoring whether the face of the live streamer moves;

in response to determining that the face of the live streamer moves, obtaining a second face position associated with the live streamer after a movement of the face of the live streamer; and

changing the end point of the first special effect from the first face position to the second face position.

6. The method of claim 1 , wherein the displaying the first special effect in the video play box comprises:

obtaining a face image of the live streamer in the streaming video;

synthesizing the face image and a facial special effect corresponding to the first gesture; and

displaying a synthesized image comprising the face image and the facial special effect, wherein the facial special effect comprises at least one of a clothing special effect and an expression special effect.

7. The method of claim 1 , further comprising:

determining a gesture duration of the first gesture; and

triggering a second special effect in response to determining that the gesture duration is greater than or equal to a predetermined duration threshold.

8. A computer device, comprising a memory and a processor, wherein the memory stores computer-readable instructions operable on the processor, and wherein the computer-readable instructions, upon being executed by the processor, cause the processor to implement operations comprising:

obtaining a streaming video of a live streamer;

obtaining images of a user captured in real time by a user terminal;

displaying the streaming video and the images of the user in a same video play box;

obtaining and recognizing a first gesture of the user in the images;

comparing the first gesture with a second gesture included in a preset table, wherein the preset table comprises information indicating corresponding relationships between gestures and special effects;

obtaining a first special effect corresponding to the second gesture by querying the preset table in response to determining that the first gesture matches the second gesture; and

displaying the first special effect in the video play box.

9. The computer device of claim 8 , wherein the displaying the streaming video and the images of the user in a same video play box comprises:

playing the streaming video and the images of the user in a first part and a second part of the video play box respectively in a seamless stitching manner.

10. The computer device of claim 8 , wherein the displaying the first special effect in the video play box comprises:

obtaining a first palm position associated with the first gesture;

obtaining a first face position associated with a face of the live streamer in the streaming video; and

displaying the first special effect by extending from the first palm position as a start point to the first face position as an end point.

11. The computer device of claim 10 , the computer-readable instructions, upon being executed by the processor, further cause the processor to implement operations comprising:

monitoring whether a hand palm associated with the first gesture moves;

in response to determining that the hand palm associated with the first gesture moves, obtaining a second palm position associated with the first gesture after a movement of the hand palm; and

changing the start point of the first special effect from the first palm position to the second palm position.

12. The computer device of claim 10 , the computer-readable instructions, upon being executed by the processor, further cause the processor to implement operations comprising:

monitoring whether the face of the live streamer moves;

in response to determining that the face of the live streamer moves, obtaining a second face position associated with the live streamer after a movement of the face of the live streamer; and

changing the end point of the first special effect from the first face position to the second face position.

13. The computer device of claim 8 , wherein the displaying the first special effect in the video play box comprises:

obtaining a face image of the live streamer in the streaming video;

synthesizing the face image and a facial special effect corresponding to the first gesture; and

displaying a synthesized image comprising the face image and the facial special effect, wherein the facial special effect comprises at least one of a clothing special effect and an expression special effect.

14. The computer device of claim 8 , the computer-readable instructions, upon being executed by the processor, further cause the processor to implement operations comprising:

determining a gesture duration of the first gesture; and

triggering a second special effect in response to determining that the gesture duration is greater than or equal to a predetermined duration threshold.

15. A non-transitory computer-readable storage medium computer-readable instructions, the computer-readable instructions being executable by at least one processor to cause the at least one processor to perform operations comprising:

obtaining a streaming video of a live streamer;

obtaining images of a user captured in real time by a user terminal;

displaying the streaming video and the images of the user in a same video play box;

obtaining and recognizing a first gesture of the user in the images;

comparing the first gesture with a second gesture included in a preset table, wherein the preset table comprises information indicating corresponding relationships between gestures and special effects;

obtaining a first special effect corresponding to the second gesture by querying the preset table in response to determining when the first gesture matches the second gesture; and

displaying the first special effect in the video play box.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the displaying the streaming video and the images of the user in a same video play box comprises:

playing the streaming video and the images of the user in a first part and a second part of the video play box respectively in a seamless stitching manner.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the displaying the first special effect in the video play box comprises:

obtaining a first palm position associated with the first gesture;

obtaining a first face position associated with a face of the live streamer in the streaming video; and

displaying the first special effect by extending from the first palm position as a start point to the first face position as an end point.

18. The non-transitory computer-readable storage medium of claim 17 , the computer-readable instructions, upon being executed by the processor, further cause the at least one processor to perform operations comprising:

monitoring whether a hand palm associated with the first gesture moves, obtaining a second palm position associated with the first gesture after a movement of the hand palm in response to determining that the hand palm associated with the first gesture moves, and changing the start point of the first special effect from the first palm position to the second palm position; or

monitoring whether the face of the live streamer moves, obtaining a second face position associated with the live streamer after a movement of the face of the live streamer in response to determining that the face of the live streamer moves, and changing the end point of the first special effect from the first face position to the second face position; or

determining a gesture duration of the first gesture and triggering a second special effect in response to determining that the gesture duration is greater than or equal to a predetermined duration threshold.

19. The non-transitory computer-readable storage medium of claim 15 , wherein the displaying the first special effect in the video play box comprises:

obtaining a face image of the live streamer in the streaming video;

synthesizing the face image and a facial special effect corresponding to the first gesture; and

displaying a synthesized image comprising the face image and the facial special effect, wherein the facial special effect comprises at least one of a clothing special effect and an expression special effect.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2022
From: TANG, ZIXIN; XUE, DEWEI
To: SHANGHAI HODE INFORMATION TECHNOLOGY CO., LTD.
Reel/Frame 060143/0890 →
Priority Claims (1)
CN 201911251115.7 · Dec 9, 2019 · national
Continuity (1)
Related Publication 20230012089A1 · Jan 12, 2023