IP Library Granted Patent US 10,129,476
Granted Patent B1
US 10,129,476 · App. 15/962,347 · Granted Nov 13, 2018

Subject stabilisation based on the precisely detected face position in the visual input and computer systems and computer-implemented methods for implementing thereof

Inventors: Yury Hushchyn (Vilnius, LT); Aliaksei Sakolski (Minsk, BY)
Assignee: Banuba Limited
H04N5/23254G06K9/00228G06T7/248H04N5/23267G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,129,476
App. No.
15/962,347
Granted
Nov 13, 2018
Kind
B1
Abstract

In some embodiments, the present invention provides for an exemplary computer system that may include: a camera component configured to acquire a visual content, wherein the visual content having a plurality of frames with a visual representation of a face of a person; a processor configured to: apply, for each frame, a multi-dimensional face detection regressor for fitting at least one meta-parameter to detect or to track a plurality of multi-dimensional landmarks representative of a face; apply a face movement detection algorithm to identify each displacement of each respective multi-dimensional landmark between frames; and apply a face movement compensation algorithm to generate a face movement compensated output that stabilizes the visual representation of the face.

Claims (38)

1. A computer-implemented method, comprising:

obtaining, by at least one processor, a plurality of frames having a visual representation of a face of at least one person;

applying, by the at least one processor, for each frame, at least one multi-dimensional face detection regressor for fitting at least one meta-parameter to detect or to track a plurality of multi-dimensional landmarks that are representative of a presence of a face of at least one person in each respective frame;

separating, by the at least one processor, for each frame in the plurality of frames, the face of the at least one person from a background based on utilizing at least one deep learning algorithm;

applying, by the at least one processor, for each frame in the plurality of frames, at least one face movement detection algorithm to identify each displacement of each respective multi-dimensional landmark of the plurality of multi-dimensional landmarks between frames;

applying, by the at least one processor, for each two sequential frames in the plurality of frames, at least one face movement compensation algorithm that is configured to at least:

i) determine that a current displacement value of at least one respective multi-dimensional landmark of the plurality of multi-dimensional landmarks between two sequential frames exceeds a pre-determined threshold value, and

ii) re-draw the face of the at least one person for a particular frame of the two sequential frames, in which the current displacement value exceeds the pre-determined threshold value, to reduce the current displacement value of the at least one respective multi-dimensional landmark to an updated displacement value that is less than the pre-determined threshold value to generate a re-drawn face of the at least one person;

wherein the pre-determined threshold value is between 1 and 20 Hz; and

combining, by the at least one processor, the re-drawn face of the at least one person in the particular frame of the two sequential frames with the background to generate a face movement compensated output that stabilizes the visual representation of the face of the at least one person between the two sequential frames of the plurality of frames.

2. The method of claim 1 , wherein the plurality of frames is part of a video stream.

3. The method of claim 2 , wherein the video stream is a real-time video stream.

4. The method of claim 3 , wherein the real-time video stream is a live video stream.

5. The method of claim 1 , wherein the pre-determined threshold value is between 10 and 20 Hz.

6. The method of claim 1 , wherein the method further comprising:

applying, by the at least one processor, at least one visual encoding algorithm to transform the plurality of face movement compensated frames into a visual encoded output.

7. The method of claim 6 , wherein the at least one visual encoding algorithm comprises a perceptual coding compression based on a human visual system model to remove a perceptual redundancy.

8. The method of claim 1 , wherein the plurality of frames is obtained by a camera of a portable electronic device and wherein the at least one processor is a processor of the portable electronic device.

9. A system, comprising:

a camera component, wherein the camera component is configured to acquire a visual content, wherein the visual content comprises a plurality of frames having a visual representation of a face of at least one person;

at least one processor;

a non-transitory computer memory, storing a computer program that, when executed by the at least one processor, causes the at least one processor to:

apply, for each frame of the plurality of frames, at least one multi-dimensional face detection regressor for fitting at least one meta-parameter to detect or to track a plurality of multi-dimensional landmarks that are representative of a presence of a face of at least one person in each respective frame;

separate, for each frame in the plurality of frames, the face of the at least one person from a background based on utilizing at least one deep learning algorithm;

apply, for each frame in the plurality of frames, at least one face movement detection algorithm to identify each displacement of each respective multi-dimensional landmark of the plurality of multi-dimensional landmarks between frames;

apply, for each two sequential frames in the plurality of frames, at least one face movement compensation algorithm that is configured to at least:

i) determine that a current displacement value of at least one respective multi-dimensional landmark of the plurality of multi-dimensional landmarks between two sequential frames exceeds a pre-determined threshold value, and

ii) re-draw the face of the at least one person for a particular frame of the two sequential frames, in which the current displacement value exceeds the pre-determined threshold value, to reduce the current displacement value of the at least one respective multi-dimensional landmark to an updated displacement value that is less than the pre-determined threshold value to generate a re-drawn face of the at least one person;

wherein the pre-determined threshold value is between 1 and 20 Hz; and

combine the re-drawn face of the at least one person in the particular frame of the two sequential frames with the background to generate a face movement compensated output that stabilizes the visual representation of the face of the at least one person between the two sequential frames of the plurality of frames.

10. The system of claim 9 , wherein the plurality of frames is part of a video stream.

11. The system of claim 10 , wherein the video stream is a real-time video stream.

12. The system of claim 11 , wherein the real-time video stream is a live video stream.

13. The system of claim 9 , wherein the pre-determined threshold value is between 10 and 20 Hz.

14. The system of claim 9 , wherein the at least one processor is further configured to:

apply at least one visual encoding algorithm to transform the plurality of face movement compensated frames into a visual encoded output.

15. The system of claim 14 , wherein the at least one visual encoding algorithm comprises a perceptual coding compression based on a human visual system model to remove a perceptual redundancy.

16. The system of claim 9 , wherein the plurality of frames is obtained by a camera of a portable electronic device and wherein the at least one processor is a processor of the portable electronic device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 12, 2025
From: BANUBA LIMITED
To: BANUBA FZCO
Reel/Frame 070195/0149 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2018
From: HUSHCHYN, YURY; SAKOLSKI, ALIAKSEI
To: BANUBA LIMITED
Reel/Frame 046982/0667 →
Continuity (1)
Provisional Application 62490433 · Apr 26, 2017
Cited By (5)
US 12,401,911 US 12,401,912 US 12,418,727 US 12,445,736 US 12,666,159