Granted Patent B2

US 12,657,878 · App. 17/941,060 · Granted Jun 16, 2026

Inventor: Takato Kimura (Kanagawa, JP)

Assignee: CANON KABUSHIKI KAISHA

G06V10/7715G06T7/10G06V10/761G06V10/764G06V10/774G06V10/82G06V40/168G06T2207/20021G06T2207/20081G06T2207/20084G06T2207/30201

Abstract

Claims (86)

1 . An image processing apparatus, comprising:

a processor; and

a memory including instructions, which when executed by the processor, cause the image processing apparatus to perform operations comprising:

acquiring an input image;

dividing the input image into a plurality of partial images;

converting the partial images into tokens having fixed dimensional vectors;

adding two or more class tokens having a vector of a dimension same as the tokens having fixed dimensional vectors to the tokens having fixed dimensional vectors;

updating the two or more class tokens based on a relevance between the two or more class tokens to obtain encoded representations;

acquiring class-token encoded representations corresponding to each of the two or more class tokens in the encoded representations; and

combining the two or more class encoded representations to obtain a feature vector of the input image, wherein

a learning is performed with a loss function including a regularization term that decreases at least one of a degree of similarity between the two or more class tokens and a degree of similarity between the class-token encoded representations.

2 . The apparatus according to claim 1 , wherein

combining the class-token encoded representations includes outputting a weighted sum of the class-token encoded representations as the feature vector of the input image.

3 . The apparatus according to claim 1 , wherein

combining the class-token encoded representations includes outputting a basic statistic of the class-token encoded representations as the feature vector of the input image.

4 . The apparatus according to claim 1 , wherein

combining the class-token encoded representations includes combining the class-token encoded representations by neural network to obtain the feature vector of the input image.

5 . The apparatus according to claim 1 , wherein

combining the class-token encoded representations includes:

adding one feature vector token to the class-token encoded representations;

obtaining second encoded representations from the class-token encoded representations; and

acquiring a feature vector encoded representation corresponding to a feature vector token in the second encoded representations as the feature vector of the input image.

6 . The apparatus according to claim 1 , wherein

updating the tokens includes calculating an attention map including a matrix of the number of tokens x the number of tokens having relationships between the respective tokens as elements, and

a learning is performed with a loss function including a regularization term, and the regularization term decreases at least one of a degree of similarity between the two or more class tokens, a degree of similarity between the class-token encoded representations, and a degree of similarity between the attention maps corresponding to the two or more class tokens.

7 . The apparatus according to claim 1 , wherein

the respective two or more class tokens are learned with common supervisory data.

8 . The apparatus according to claim 1 , wherein

the respective two or more class tokens are initialized with random values having values different from one another for learning.

9 . The apparatus according to claim 8 , wherein

the respective two or more class tokens are initialized so as to be orthogonal to one another for learning.

10 . A method of controlling an image processing apparatus, comprising the steps of:

(a) acquiring an input image;

(b) dividing the input image into a plurality of partial images;

(c) converting the partial images into tokens having fixed dimensional vectors to convert partial images divided in the dividing (b) into tokens;

(d) adding two or more class tokens class tokens having a vector of a dimension same as the tokens to the tokens obtained in the converting (c);

(e) updating the tokens two or more class tokens obtained in the adding (d) based on a relevance between the tokens two or more class tokens to obtain encoded representations;

(f) acquiring class tokens class tokens encoded representations corresponding to each of the respective class-tokens in the encoded representations obtained in the updating (e) as class-token encoded representations; and

(g) combining the class-token encoded representations to obtain a feature vector of the input image, wherein

a learning is performed with a loss function including a regularization term that decreases at least one of a degree of similarity between the two or more class tokens and a degree of similarity between the class-token encoded representations.

11 . A non-transitory computer-readable storage medium storing a program which, when read and executed by a computer, causes the computer to perform steps of a method controlling an image processing apparatus, the method comprising the steps of:

(a) acquiring an input image;

(b) dividing the input image into a plurality of partial images;

(c) converting the partial images into tokens having fixed dimensional vectors to convert partial images divided in the dividing (b) into tokens having fixed dimensional vectors;

(d) adding two or more class tokens having a vector of a dimension same as the tokens to the tokens obtained in the converting (c);

(e) updating the two or more class tokens obtained in the adding (d) based on a relevance between the two or more class tokens to obtain encoded representations;

(f) acquiring class-token encoded representations corresponding to each of the respective class tokens in the encoded representations obtained in the updating (e) as class-token encoded representations; and

(g) combining the class-token encoded representations to obtain a feature vector of the input image, wherein

a learning is performed with a loss function including a regularization term that decreases at least one of a degree of similarity between the two or more class tokens and a degree of similarity between the class-token encoded representations.

12 . An image processing apparatus, comprising:

a processor; and

a memory including instructions, which when executed by the processor, cause the image processing apparatus to perform operations comprising:

acquiring an input image;

dividing the input image into a plurality of partial images;

converting the partial images into tokens having fixed dimensional vectors;

adding two or more class tokens having a vector of a dimension same as the tokens having fixed dimensional vectors to the tokens having fixed dimensional vectors;

updating the tokens based on a relevance between the tokens to obtain encoded representations;

acquiring class-token encoded representations corresponding to each of the respective two or more class tokens in the encoded representations; and

combining the class-token encoded representations to obtain a feature vector of the input image, wherein

combining the class-token encoded representations includes:

adding one feature vector token to the class-token encoded representations;

obtaining second encoded representations from the class-token encoded representations; and

acquiring a feature vector encoded representation corresponding to a feature vector token in the second encoded representations as the feature vector of the input image.

13 . An image processing apparatus, comprising:

a processor; and

a memory including instructions, which when executed by the processor, cause the image processing apparatus to perform operations comprising:

acquiring an input image;

dividing the input image into a plurality of partial images;

converting the partial images into tokens having fixed dimensional vectors;

adding two or more class tokens having a vector of a dimension same as the tokens having fixed dimensional vectors to the tokens having fixed dimensional vectors;

updating the tokens based on a relevance between the tokens to obtain encoded representations;

acquiring class-token encoded representations corresponding to each of the respective two or more class tokens in the encoded representations; and

combining the class-token encoded representations to obtain a feature vector of the input image, wherein

updating the tokens includes calculating an attention map including a matrix of the number of tokens x the number of tokens having relationships between the respective tokens as elements, and

a learning is performed with a loss function including a regularization term, and the regularization term decreases at least one of a degree of similarity between the two or more class tokens, a degree of similarity between the class-token encoded representations, and a degree of similarity between the attention maps corresponding to the class tokens.

14 . An image processing apparatus, comprising:

a processor; and

a memory including instructions, which when executed by the processor, cause the image processing apparatus to perform operations comprising:

acquiring an input image;

dividing the input image into a plurality of partial images;

converting the partial images into tokens having fixed dimensional vectors;

adding two or more class tokens having a vector of a dimension same as the tokens to the tokens;

updating the tokens based on a relevance between the tokens to obtain encoded representations;

acquiring class-token encoded representations corresponding to each of the respective two or more class tokens in the encoded representations; and

combining the class-token encoded representations to obtain a feature vector of the input image, wherein

the respective two or more class tokens are initialized with random values having values different from one another, and so as to be orthogonal to one another for learning.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (28)