Granted Patent B2

US 12,744,861 · App. 18/254,575 · Granted Sep 22, 2026

Inventors: Yingyi Chen (Beijing, CN); Shaohui Jiao (Beijing, CN)

Assignee: Beijing Bytedance Network Technology Co., Ltd.

H04N5/272G06T7/194G06V10/16G06V10/44G06V10/60G06V10/751

Abstract

Claims (75)

1 . A method for replacing a background in a picture, comprising:

acquiring a basic background image of a preset scene, wherein the basic background image is an image obtained by shooting the preset scene in advance;

acquiring a captured picture obtained by currently shooting the preset scene, and determining a current background image of the preset scene according to the basic background image and the captured picture; and

replacing the current background image in the captured picture with a preset background image to obtain a final captured picture,

wherein the determining the current background image of the preset scene according to the basic background image and the captured picture comprises:

extracting basic feature point information of the basic background image, and extracting shooting feature point information in the captured picture; and

determining a matching relationship between each piece of the basic feature point information and each piece of the shooting feature point information, and determining the current background image in the captured picture according to the matching relationship,

wherein the determining the current background image in the captured picture according to the matching relationship comprises:

stitching a background portion in the captured picture and the basic background image according to the matching relationship, to obtain the current background image of the preset scene in the captured picture,

wherein the method further comprises:

extending a boundary of the current background image to obtain an extended background image; and

replacing the extended background image in the captured picture with the preset background image correspondingly according to the extended background image, to obtain the final captured picture,

wherein the method further comprises:

performing any of the following enhancement processing on the extended background image to obtain an enhanced background image:

stitching seam feathering, light transformation; and

replacing the enhanced background image in the captured picture with the preset background image correspondingly according to the enhanced background image, to obtain the final captured picture, and

wherein the extended background image and the enhanced background image are images extended based on a current shooting angle.

2 . The method according to claim 1 , wherein the replacing the current background image in the captured picture with the preset background image to obtain the final captured picture comprises:

determining a foreground image comprised in the captured picture according to the current background image and the captured picture; and

generating the final captured picture according to the foreground image and the preset background image, wherein a background of the final captured picture is the preset background image, and a foreground of the final captured picture is the foreground image.

3 . The method according to claim 2 , wherein the determining the foreground image comprised in the captured picture according to the current background image and the captured picture comprises:

inputting the current background image and the captured picture into a preset segmentation model, wherein the preset segmentation model is used to output the foreground image comprised in the captured picture;

or, inputting the extended background image, which is determined according to the current background image, and the captured picture into a preset segmentation model, wherein the preset segmentation model is used to output the foreground image comprised in the captured picture;

or, inputting the enhanced background image, which is determined according to the current background image, and the captured picture into a preset segmentation model, wherein the preset segmentation model is used to output the foreground image comprised in the captured picture.

4 . The method according to claim 1 , further comprising:

updating the basic background image to the enhanced background image or the extended background image.

5 . The method according to claim 1 , further comprising:

sending the final captured picture.

6 . An apparatus for replacing a background in a picture, comprising:

at least one processor, and

a memory communicatively connected to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor is configured to:

acquire a basic background image of a preset scene, wherein the basic background image is an image obtained by shooting the preset scene in advance;

acquire a captured picture obtained by currently shooting the preset scene;

determine a current background image of the preset scene according to the basic background image and the captured picture; and

replace the current background image in the captured picture with a preset background image to obtain a final captured picture,

wherein the at least one processor is further configured to:

extract basic feature point information of the basic background image, and extract shooting feature point information in the captured picture; and

determine a matching relationship between each piece of the basic feature point information and each piece of the shooting feature point information, and determine the current background image in the captured picture according to the matching relationship,

wherein the at least one processor is further configured to:

stitch a background portion in the captured picture and the basic background image according to the matching relationship, to obtain the current background image of the preset scene in the captured picture,

wherein the at least one processor is further configured to:

extend a boundary of the current background image to obtain an extended background image; and

replace the extended background image in the captured picture with the preset background image correspondingly according to the extended background image, to obtain the final captured picture,

wherein the at least one processor is further configured to:

perform any of the following enhancement processing on the extended background image to obtain an enhanced background image:

stitching seam feathering, light transformation; and

replace the enhanced background image in the captured picture with the preset background image correspondingly according to the enhanced background image, to obtain the final captured picture, and

wherein the extended background image and the enhanced background image are images extended based on a current shooting angle.

7 . The apparatus according to claim 6 , wherein the at least one processor is configured to:

determine a foreground image comprised in the captured picture according to the current background image and the captured picture; and

generate the final captured picture according to the foreground image and the preset background image, wherein a background of the final captured picture is the preset background image, and a foreground of the final captured picture is the foreground image.

8 . The apparatus according to claim 7 , wherein the at least one processor is configured to:

input the current background image and the captured picture into a preset segmentation model, wherein the preset segmentation model is used to output the foreground image comprised in the captured picture;

or, input the extended background image, which is determined according to the current background image, and the captured picture into a preset segmentation model, wherein the preset segmentation model is used to output the foreground image comprised in the captured picture;

or, input the enhanced background image, which is determined according to the current background image, and the captured picture into a preset segmentation model, wherein the preset segmentation model is used to output the foreground image comprised in the captured picture.

9 . The apparatus according to claim 6 , wherein the at least one processor is configured to:

send the final captured picture.

10 . A non-transitory computer-readable storage medium, wherein the computer-readable storage medium has a computer-executed instruction stored thereon, and when a processor executes the computer-executed instruction, the processor executes steps of:

acquiring a basic background image of a preset scene, wherein the basic background image is an image obtained by shooting the preset scene in advance;

acquiring a captured picture obtained by currently shooting the preset scene, and determining a current background image of the preset scene according to the basic background image and the captured picture; and

replacing the current background image in the captured picture with a preset background image to obtain a final captured picture,

wherein the computer-executed instruction is further used to perform the following steps:

extracting basic feature point information of the basic background image, and extracting shooting feature point information in the captured picture; and

determining a matching relationship between each piece of the basic feature point information and each piece of the shooting feature point information, and determining the current background image in the captured picture according to the matching relationship,

wherein the computer-executed instruction is further used to perform the following steps:

stitching a background portion in the captured picture and the basic background image according to the matching relationship, to obtain the current background image of the preset scene in the captured picture,

wherein the computer-executed instruction is further used to perform the following steps:

extending a boundary of the current background image to obtain an extended background image; and

replacing the extended background image in the captured picture with the preset background image correspondingly according to the extended background image, to obtain the final captured picture,

wherein the computer-executed instruction is further used to perform the following steps:

performing any of the following enhancement processing on the extended background image to obtain an enhanced background image:

stitching seam feathering, light transformation; and

replacing the enhanced background image in the captured picture with the preset background image correspondingly according to the enhanced background image, to obtain the final captured picture, and

wherein the extended background image and the enhanced background image are images extended based on a current shooting angle.

Assignments (3)

Priority Claims (1)

Continuity (1)

References Cited (62)