IP Library Granted Patent US 12711571
Granted Patent B2
US 12711571 · App. 18/813,778 · Granted Aug 18, 2026

Image processing method, electronic device and storage medium

Inventors: Ziyao Xu (Beijing, CN); Yi Liu (Beijing, CN); Shijie Zhao (Beijing, CN); Junlin Li (Los Angeles, CA); Li Zhang (Los Angeles, CA)
Assignees: Douyin Vision Co., Ltd.; Lemon Inc.
G06T3/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12711571
App. No.
18/813,778
Granted
Aug 18, 2026
Kind
B2
Abstract

An image processing method, an electronic device and a storage medium are provided. The method includes acquiring a first image; performing down-sampling processing on the first image to obtain a first down-sampled image; performing average pooling processing on pixels in the first down-sampled image to obtain a first pooled image; determining a plurality of pixel merging weights corresponding to pixels in the first down-sampled image; and performing pixel merging processing on the first down-sampled image and the first pooled image based on the plurality of pixel merging weights to obtain a second image.

Claims (93)

1 . An image processing method, comprising:

acquiring a first image;

performing down-sampling processing on the first image to obtain a first down-sampled image;

performing average pooling processing on pixels in the first down-sampled image to obtain a first pooled image;

determining a plurality of pixel merging weights corresponding to the pixels in the first down-sampled image; and

performing pixel merging processing on the first down-sampled image and the first pooled image based on the plurality of pixel merging weights to obtain a second image.

2 . The method according to claim 1 , wherein performing average pooling processing on the pixels in the first down-sampled image to obtain the first pooled image, comprising:

determining N image blocks from the first down-sampled image, wherein N is an integer greater than 1;

determining an average pixel value corresponding to a plurality of pixels in each image block of the N image blocks; and

generating the first pooled image based on the average pixel value corresponding to each image block, wherein the first pooled image comprises N pixels, and pixel values of the N pixels are in one-to-one correspondence with average pixel values of the N image blocks.

3 . The method according to claim 2 , wherein determining the N image blocks from the first down-sampled image, comprising:

determining a sliding window, wherein a size of the sliding window is as same as that of the image block;

controlling the sliding window to slide for N times in the first down-sampled image based on a preset sliding step size to obtain N groups of pixels; and

determining each group of pixels of the N groups of pixels as one image block, to obtain the N image blocks.

4 . The method according to claim 1 , wherein determining the plurality of pixel merging weights corresponding to the pixels in the first down-sampled image, comprising:

determining a variance associated with the pixels in the first down-sampled image;

determining a covariance associated with the pixels in the first down-sampled image; and

determining the plurality of pixel merging weights corresponding to the pixels in the first down-sampled image based on the variance and the covariance.

5 . The method according to claim 4 , wherein determining the variance associated with the pixels in the first down-sampled image, comprising:

performing squared processing on a pixel value of each of the pixels in the first down-sampled image to obtain a squared first down-sampled image;

performing average pooling processing on the squared first down-sampled image to obtain a second pooled image;

performing squared processing on a pixel value of each of pixels in the first pooled image to obtain a squared first pooled image; and

determining the variance associated with the pixels in the first down-sampled image based on the squared first pooled image and the second pooled image.

6 . The method according to claim 5 , wherein determining the covariance associated with the pixels in the first down-sampled image, comprising:

performing squared processing on a pixel value of each of pixels in the first image to obtain a squared first image, and performing average pooling processing on the squared first image to obtain a third image;

performing down-sampling processing on the third image to obtain a third down-sampled image associated with the third image;

performing average pooling processing on the third down-sampled image associated with the third image to obtain a third pooled image; and

determining the covariance associated with the pixels in the first down-sampled image based on the third pooled image and the squared first pooled image.

7 . The method according to claim 1 , wherein performing pixel merging processing on the first down-sampled image and the first pooled image based on the plurality of pixel merging weights to obtain the second image, comprising:

for each pixel of a plurality of pixels in the first down-sampled image, obtaining one or more target pixel merging weights determined based on the pixel from the plurality of pixel merging weights, to obtain a plurality of target pixel merging weights corresponding to the plurality of pixels in the first down-sampled image;

determining a target pixel in the first pooled image corresponding to each pixel in the first down-sampled image, to obtain a plurality of target pixels corresponding to the plurality of pixels in the first down-sampled image; and

determining the second image based on the first down-sampled image, the plurality of target pixel merging weights, and the plurality of target pixels.

8 . An electronic device, comprising: at least one processor and a memory on which computer-readable instructions are stored; wherein

the at least one processor is configured to execute the computer-executable instructions stored in the memory, so that the at least one processor executes an image processing method, comprising:

acquiring a first image;

performing down-sampling processing on the first image to obtain a first down-sampled image;

performing average pooling processing on pixels in the first down-sampled image to obtain a first pooled image;

determining a plurality of pixel merging weights corresponding to the pixels in the first down-sampled image; and

performing pixel merging processing on the first down-sampled image and the first pooled image based on the plurality of pixel merging weights to obtain a second image.

9 . The electronic device according to claim 8 , wherein in the image processing method, performing average pooling processing on the pixels in the first down-sampled image to obtain the first pooled image, comprising:

determining N image blocks from the first down-sampled image, wherein N is an integer greater than 1;

determining an average pixel value corresponding to a plurality of pixels in each image block of the N image blocks; and

generating the first pooled image based on the average pixel value corresponding to each image block, wherein the first pooled image comprises N pixels, and pixel values of the N pixels are in one-to-one correspondence with average pixel values of the N image blocks.

10 . The electronic device according to claim 9 , wherein in the image processing method, determining the N image blocks from the first down-sampled image, comprising:

determining a sliding window, wherein a size of the sliding window is as same as that of the image block;

controlling the sliding window to slide for N times in the first down-sampled image based on a preset sliding step size to obtain N groups of pixels; and

determining each group of pixels of the N groups of pixels as one image block, to obtain the N image blocks.

11 . The electronic device according to claim 8 , wherein in the image processing method, determining the plurality of pixel merging weights corresponding to the pixels in the first down-sampled image, comprising:

determining a variance associated with the pixels in the first down-sampled image;

determining a covariance associated with the pixels in the first down-sampled image; and

determining the plurality of pixel merging weights corresponding to the pixels in the first down-sampled image based on the variance and the covariance.

12 . The electronic device according to claim 11 , wherein in the image processing method, determining the variance associated with the pixels in the first down-sampled image, comprising:

performing squared processing on a pixel value of each of the pixels in the first down-sampled image to obtain a squared first down-sampled image;

performing average pooling processing on the squared first down-sampled image to obtain a second pooled image;

performing squared processing on a pixel value of each of pixels in the first pooled image to obtain a squared first pooled image; and

determining the variance associated with the pixels in the first down-sampled image based on the squared first pooled image and the second pooled image.

13 . The electronic device according to claim 12 , wherein in the image processing method, determining the covariance associated with the pixels in the first down-sampled image, comprising:

performing squared processing on a pixel value of each of pixels in the first image to obtain a squared first image, and performing average pooling processing on the squared first image to obtain a third image;

performing down-sampling processing on the third image to obtain a third down-sampled image associated with the third image;

performing average pooling processing on the third down-sampled image associated with the third image to obtain a third pooled image; and

determining the covariance associated with the pixels in the first down-sampled image based on the third pooled image and the squared first pooled image.

14 . The electronic device according to claim 8 , wherein in the image processing method, performing pixel merging processing on the first down-sampled image and the first pooled image based on the plurality of pixel merging weights to obtain the second image, comprising:

for each pixel of a plurality of pixels in the first down-sampled image, obtaining one or more target pixel merging weights determined based on the pixel from the plurality of pixel merging weights, to obtain a plurality of target pixel merging weights corresponding to the plurality of pixels in the first down-sampled image;

determining a target pixel in the first pooled image corresponding to each pixel in the first down-sampled image, to obtain a plurality of target pixels corresponding to the plurality of pixels in the first down-sampled image; and

determining the second image based on the first down-sampled image, the plurality of target pixel merging weights, and the plurality of target pixels.

15 . A non-transient computer-readable storage medium on which computer-executable instructions are stored, wherein the computer-executable instructions, when executed by a processor, are configured to cause the processer to execute an image processing method, comprising:

acquiring a first image;

performing down-sampling processing on the first image to obtain a first down-sampled image;

performing average pooling processing on pixels in the first down-sampled image to obtain a first pooled image;

determining a plurality of pixel merging weights corresponding to the pixels in the first down-sampled image; and

performing pixel merging processing on the first down-sampled image and the first pooled image based on the plurality of pixel merging weights to obtain a second image.

16 . The non-transient computer-readable storage medium according to claim 15 , wherein in the image processing method, performing average pooling processing on the pixels in the first down-sampled image to obtain the first pooled image, comprising:

determining N image blocks from the first down-sampled image, wherein N is an integer greater than 1;

determining an average pixel value corresponding to a plurality of pixels in each image block of the N image blocks; and

generating the first pooled image based on the average pixel value corresponding to each image block, wherein the first pooled image comprises N pixels, and pixel values of the N pixels are in one-to-one correspondence with average pixel values of the N image blocks.

17 . The non-transient computer-readable storage medium according to claim 16 , wherein in the image processing method, determining the N image blocks from the first down-sampled image, comprising:

determining a sliding window, wherein a size of the sliding window is as same as that of the image block;

controlling the sliding window to slide for N times in the first down-sampled image based on a preset sliding step size to obtain N groups of pixels; and

determining each group of pixels of the N groups of pixels as one image block, to obtain the N image blocks.

18 . The non-transient computer-readable storage medium according to claim 15 , wherein in the image processing method, determining the plurality of pixel merging weights corresponding to the pixels in the first down-sampled image, comprising:

determining a variance associated with the pixels in the first down-sampled image;

determining a covariance associated with the pixels in the first down-sampled image; and

determining the plurality of pixel merging weights corresponding to the pixels in the first down-sampled image based on the variance and the covariance.

19 . The non-transient computer-readable storage medium according to claim 18 , wherein in the image processing method, determining the variance associated with the pixels in the first down-sampled image, comprising:

performing squared processing on a pixel value of each of the pixels in the first down-sampled image to obtain a squared first down-sampled image;

performing average pooling processing on the squared first down-sampled image to obtain a second pooled image;

performing squared processing on a pixel value of each of pixels in the first pooled image to obtain a squared first pooled image; and

determining the variance associated with the pixels in the first down-sampled image based on the squared first pooled image and the second pooled image.

20 . The non-transient computer-readable storage medium according to claim 19 , wherein in the image processing method, determining the covariance associated with the pixels in the first down-sampled image, comprising:

performing squared processing on a pixel value of each of pixels in the first image to obtain a squared first image, and performing average pooling processing on the squared first image to obtain a third image;

performing down-sampling processing on the third image to obtain a third down-sampled image associated with the third image;

performing average pooling processing on the third down-sampled image associated with the third image to obtain a third pooled image; and

determining the covariance associated with the pixels in the first down-sampled image based on the third pooled image and the squared first pooled image.