IP Library › Granted Patent US 11,714,921
Granted Patent B2
US 11,714,921 · App. 16/916,872 · Granted Aug 1, 2023

Image processing method with ash code on local feature vectors, image processing device and storage medium

Inventors: Yigeng Fang (Beijing, CN); Xiaojun Tang (Beijing, CN); Yadong Mu (Beijing, CN)
Assignees: BOE TECHNOLOGY GROUP CO., LTD.; PEKING UNIVERSITY
G06F21/64G06N3/08G06V10/454G06V10/764G06V10/82H04L9/0643H04L9/3247
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,714,921
App. No.
16/916,872
Granted
Aug 1, 2023
Kind
B2
Abstract

Provided are an image processing method, an image matching method, a device, and a storage medium. The image processing method includes: obtaining an image feature of an input image; determining a plurality of local image features of the image feature; determining a plurality of local feature vectors corresponding to the plurality of local image features respectively; determining a hash code of the input image based on the plurality of local feature vectors.

Claims (41)

1. An image processing method, executed by at least one hardware processor, comprising:

processing an input image to obtain an image feature of the input image;

processing the image feature to determine a plurality of local image features of the image feature;

determining a plurality of local feature vectors corresponding to the plurality of local image features respectively;

determining a hash code of the input image based on the plurality of local feature vectors;

wherein the determining a hash code of the input image based on the plurality of local feature vectors, comprises:

mapping each of the plurality of local feature vectors into each of a plurality of sub-hash codes of an equal length respectively; and

combining the plurality of sub-hash codes to obtain the hash code of the input image, wherein the combining the plurality of sub-hash codes to obtain the hash code of the input image comprises:

arranging the plurality of sub-hash codes according to a spatial arrangement mode of a local region corresponding each sub-hash code, to obtain the hash code of the image, wherein the hash code includes a plurality of layers, an i-th layer of the hash code includes an i-th hash bit of each sub-hash code, and the i-th layer is a matrix formed by arranging the i-th hash bit of the sub-hash code according to the spatial arrangement mode of the local region corresponding to each sub-hash code, where i is an integer greater than or equal to 1 and less than or equal to K, and K is a length of each sub-hash code.

2. The image processing method according to claim 1 , wherein the processing the image feature to determine a plurality of local image features of the image feature comprises:

segmenting the image feature at a preset size, to determine a plurality of local regions of the image feature;

determining information of the image feature that corresponds to each local region in the plurality of local regions as a local image feature corresponding to the local region respectively.

3. The image processing method according to claim 2 , wherein the image feature includes feature maps of a plurality of channels, and the local image feature corresponding to each local region includes a channel local feature of a corresponding local region of the feature map of each channel in the feature maps of the plurality of channels.

4. The image processing method according to claim 3 , wherein the determining a plurality of local feature vectors corresponding to the plurality of local image features respectively comprises:

pooling the channel local feature of the corresponding local region in a feature map of each channel for each local image feature, to determine the local feature vector corresponding to the local image feature.

5. The image processing method according to claim 1 , wherein processing an input image to obtain an image feature of the input image comprises:

extracting the image feature of the input image by a trained neural network.

6. An image data processing device, the device comprising a memory and a processor, wherein the memory has instructions stored thereon, and when the processor is used to execute the instructions, the processor is caused to execute an image processing method, comprising:

processing an input image to obtain an image feature of the input image;

processing the image feature to determine a plurality of local image features of the image feature;

determining a plurality of local feature vectors corresponding to the plurality of local image features respectively;

determining a hash code of the input image based on the plurality of local feature vectors,

wherein the determining a hash code of the input image based on the plurality of local feature vectors, comprises:

mapping each of the plurality of local feature vectors into each of a plurality of sub-hash codes of an equal length respectively; and

combining the plurality of sub-hash codes to obtain the hash code of the input image, wherein the combining the plurality of sub-hash codes to obtain the hash code of the input image comprises:

arranging the plurality of sub-hash codes according to a spatial arrangement mode of a local region corresponding each sub-hash code, to obtain the hash code of the image, wherein the hash code includes a plurality of layers, an i-th layer of the hash code includes an i-th hash bit of each sub-hash code, and the i-th layer is a matrix formed by arranging the i-th hash bit of the sub-hash code according to the spatial arrangement mode of the local region corresponding to each sub-hash code, where i is an integer greater than or equal to 1 and less than or equal to K, and K is a length of each sub-hash code.

7. The image data processing device according to claim 6 , wherein the processing the image feature to determine a plurality of local image features of the image feature comprises:

segmenting the image feature at a preset size, to determine a plurality of local regions of the image feature;

determining information of the image feature that corresponds to each local region in the plurality of local regions as a local image feature corresponding to the local region, respectively.

8. The image data processing device according to claim 7 , wherein the image feature include feature maps of a plurality of channels, the local image feature corresponding to each local region includes a channel local feature of a corresponding local region of a feature map of each channel in the feature maps of the plurality of channels.

9. The image data processing device according to claim 8 , wherein the determining a plurality of local feature vectors respectively corresponding to the plurality of local image features comprises:

pooling the channel local feature of a corresponding local region in the feature map of each channel with respect to each local image feature, to determine the local feature vector corresponding to the local image feature.

10. A computer-readable non-transitory storage medium, having instructions stored thereon, wherein the instructions, when executed by a processor, causes the processor to execute an image processing method, comprising:

processing an input image to obtain an image feature of the input image;

processing the image feature to determine a plurality of local image features of the image feature;

determining a plurality of local feature vectors corresponding to the plurality of local image features respectively;

determining a hash code of the input image based on the plurality of local feature vectors,

wherein the determining a hash code of the input image based on the plurality of local feature vectors, comprises:

mapping each of the plurality of local feature vectors into each of a plurality of sub-hash codes of an equal length respectively; and

combining the plurality of sub-hash codes to obtain the hash code of the input image, wherein the combining the plurality of sub-hash codes to obtain the hash code of the input image comprises:

arranging the plurality of sub-hash codes according to a spatial arrangement mode of a local region corresponding each sub-hash code, to obtain the hash code of the image, wherein the hash code includes a plurality of layers, an i-th layer of the hash code includes an i-th hash bit of each sub-hash code, and the i-th layer is a matrix formed by arranging the i-th hash bit of the sub-hash code according to the spatial arrangement mode of the local region corresponding to each sub-hash code, where i is an integer greater than or equal to 1 and less than or equal to K, and K is a length of each sub-hash code.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2020
From: FANG, YIGENG
To: BOE TECHNOLOGY GROUP CO., LTD.
Reel/Frame 053089/0241 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2020
From: TANG, XIAOJUN
To: BOE TECHNOLOGY GROUP CO., LTD.
Reel/Frame 053089/0303 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2020
From: MU, YADONG
To: PEKING UNIVERSITY
Reel/Frame 053089/0482 →
Priority Claims (1)
CN 201910923691.5 · Sep 27, 2019 · national
Continuity (1)
Related Publication 20210099310A1 · Apr 1, 2021
Cited By (1)
US 12,731,369