IP Library Granted Patent US 10,460,487
Granted Patent B2
US 10,460,487 · App. 15/205,082 · Granted Oct 29, 2019

Automatic image synthesis method

Inventors: Dajun Ding (Shanghai, CN); Lili Zhao (Shanghai, CN)
Assignee: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
G06T11/60G06K9/4604G06K9/6202G06T3/40G06T5/50G06T7/11G06T7/12H04N9/78G06T7/194G06T2200/28G06T2207/10004G06T2207/10016G06T2207/20032G06T2207/20216G06T2207/20221G06T2207/20224
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,460,487
App. No.
15/205,082
Granted
Oct 29, 2019
Kind
B2
Abstract

Methods and apparatus for automatically synthesizing images are disclosed. The methods may include receiving a plurality of input frames with a common background. The methods may also include determining a number of the input frames. The methods may also include selecting, based on the number, a method to detect foregrounds of the input frames. The methods may further include using the selected method to generate an output frame comprising a combination of a plurality of the foregrounds.

Claims (90)

1. An image synthesis method, comprising:

receiving a plurality of input frames with a common background;

determining a number of the input frames;

selecting, based on the number, a method to detect foregrounds of the input frames; and

using the selected method to generate an output frame comprising a combination of a plurality of the foregrounds,

wherein generating an output frame comprising a combination of a plurality of the foregrounds comprises:

determining image positions of the foregrounds;

determining, based on the image positions, amounts of overlap among the foregrounds; and

selecting the foregrounds for the combination, based on the amounts of overlap;

wherein determining the amounts of overlap comprises:

determining a percentage of a first one of the foregrounds that overlaps with other ones of the foregrounds; and

selecting the one or more foregrounds for the combination comprises:

when the percentage is higher than a second predetermined threshold, excluding the first foreground from the combination.

2. The method of claim 1 , wherein using the method to detect the foregrounds comprises:

when the number of input frames is greater than a first predetermined threshold, determining median pixel values of corresponding pixels of the input frames;

generating a background frame comprising pixels having the median pixel values;

subtracting the background frame from the input frames to generate respective difference frames; and

detecting the foregrounds of the input frames based on the difference frames.

3. The method of claim 2 , wherein the first predetermined threshold is equal to 6.

4. The method of claim 1 , wherein using the method to detect the foregrounds comprises:

when the number of input frames is less than 6 but greater than 2, generating a first difference frame based on a first input frame and a second input frame, the first difference frame including a first region representing a foreground;

generating a second difference frame based on the first input frame and a third input frame, the second difference frame including a second region representing a foreground; and

determining a common portion of the first and second region as a foreground of the first input frame.

5. The method of claim 1 , wherein using the method to detect the foregrounds comprises:

when the number of input frames is equal to 2, generating a difference frame based on the two input frames;

determining a foreground in the difference frame;

determining whether an edge of the foreground is present in the two input frames; and

when it is determined that a first input frame includes at least part of the edge, determining that the foreground is a foreground of the first input frame.

6. The method of claim 5 , wherein determining whether the edge of the foreground is present in the two input frames comprises:

running an edge filter in regions of the two input frames that encompass the edge, the edge filter returning a first value in the first input frame and a second value in the second input frame;

comparing the first and second values; and

when the first value is greater than the second value, determining that the first input frame includes at least part of the edge.

7. The method of claim 1 , wherein generating an output frame comprising a combination of one or more foregrounds comprises:

determining image sizes of the foregrounds; and

selecting the foregrounds for the combination, based on the image sizes.

8. The method of claim 1 , wherein generating an output frame comprising a combination of a plurality of the foregrounds:

determining image positions of the detected foregrounds;

determining a sequence based on the image positions; and

combining the detected foregrounds according to the sequence.

9. The method of claim 1 , wherein using the selected method to detect the foregrounds comprises:

reducing image sizes of the input frames to generate reduced frames; and

detecting the foregrounds based on the reduced frames.

10. A device, comprising:

a memory storing instructions; and

a processor configured to execute the instructions to:

receive a plurality of input frames with a common background;

determine a number of the input frames;

select, based on the number, a method to detect foregrounds of the input frames; and

use the selected method to generate an output frame comprising a combination of a plurality of the foregrounds,

wherein the processor is further configured to execute the instructions to:

determining a percentage of a first one of the foregrounds that overlaps with other ones of the foregrounds; and

when the percentage is higher than a second predetermined threshold, excluding the first foreground from the combination.

11. The device of claim 10 , wherein the processor is further configured to execute the instructions to:

when the number of input frames is greater than a first predetermined threshold, determine median pixel values of corresponding pixels of the input frames;

generate a background frame comprising pixels having the median pixel values;

subtract the background frame from the input frames to generate respective difference frames; and

detect the foregrounds of the input frames based on the difference frames.

12. The device of claim 10 , wherein the processor is further configured to execute the instructions to:

when the number of input frames is less than 6 but greater than 2, generate a first difference frame based on a first input frame and a second input frame, the first difference frame including a first region representing a foreground;

generate a second difference frame based on the first input frame and a third input frame, the second difference frame including a second region representing a foreground; and

determine a common portion of the first and second region as a foreground of the first input frame.

13. The device of claim 10 , wherein the processor is further configured to execute the instructions to:

when the number of input frames is equal to 2, generate a difference frame based on the two input frames;

determine a foreground in the difference frame;

determine whether an edge of the foreground is present in the two input frames; and

when it is determined that a first input frame includes at least part of the edge, determine that the foreground is a foreground of the first input frame.

14. The device of claim 13 , wherein the processor is further configured to execute the instructions to:

run an edge filter in regions of the two input frames that encompass the edge, the edge filter returning a first value in the first input frame and a second value in the second input frame;

compare the first and second values; and

when the first value is greater than the second value, determine that the first input frame includes at least part of the edge.

15. The device of claim 10 , wherein the processor is further configured to execute the instructions to:

determine image positions of the foregrounds;

determine, based on the image positions, amounts of overlap among the foregrounds; and

select the foregrounds for the combination, based on the amounts of overlap.

16. The device of claim 10 , wherein the processor is further configured to execute the instructions to:

determining image sizes of the foregrounds; and

selecting the foregrounds for the combination, based on the image sizes.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a device, cause the processor to perform an image-synthesis method, the method comprising:

receiving a plurality of input frames with a common background;

determining a number of the input frames;

selecting, based on the number, a method to detect foregrounds of the input frames; and

using the selected method to generate an output frame comprising a combination of a plurality of the foregrounds,

wherein generating an output frame comprising a combination of a plurality of the foregrounds comprises:

determining image positions of the foregrounds;

determining, based on the image positions, amounts of overlap among the foregrounds; and

selecting the foregrounds for the combination, based on the amounts of overlap;

wherein determining the amounts of overlap comprises:

determining a percentage of a first one of the foregrounds that overlaps with other ones of the foregrounds; and

selecting the one or more foregrounds for the combination comprises:

when the percentage is higher than a second predetermined threshold, excluding the first foreground from the combination.

Assignments (5)
RELEASE OF SECURITY INTEREST Recorded Nov 5, 2024
From: EAST WEST BANK
To: KAMI VISION INCORPORATED
Reel/Frame 070792/0551 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 28, 2022
From: KAMI VISION INCORPORATED
To: EAST WEST BANK
Reel/Frame 059512/0101 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2022
From: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
To: KAMI VISION INC.
Reel/Frame 059275/0652 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY NAME PREVIOUSLY RECORDED AT REEL: 039106 FRAME: 0494. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded Jan 31, 2018
From: DING, DAJUN; ZHAO, LILI
To: SHANGHAI XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 045203/0747 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 8, 2016
From: DING, DAJUN; ZHAO, LILI
To: XIAOYI TECHNOLOGY CO., LTD.
Reel/Frame 039106/0494 →
Priority Claims (1)
CN 2016 1 0481886 · Jun 27, 2016 · national
Continuity (1)
Related Publication 20170372498A1 · Dec 28, 2017