IP Library › Granted Patent US 11,250,292
Granted Patent B2
US 11,250,292 · App. 16/959,605 · Granted Feb 15, 2022

Method and apparatus for generating information

Inventor: Lei Wang (Beijing, CN)
Assignees: Beijing Jingdong Shangke Information Technology Co., Ltd.; Beijing Jingdong Century Trading Co., Ltd.
G06K9/6215G06K9/00671G06K9/6256G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,250,292
App. No.
16/959,605
Granted
Feb 15, 2022
Kind
B2
Abstract

Disclosed by the present disclosure are a method and apparatus for generating information. A specific embodiment of the method comprises: obtaining a first image and a second image; inputting the first image and the second image respectively into a pre-trained detection and recognition model, to obtain an annotated first image and an annotated second image, where an annotation comprises an image box surrounding a target object in the image, and the detection and recognition model is configured to represent the correspondence relationship between an image and an annotated image; and inputting the annotated first image and the annotated second image to a pre-trained matching model to obtain a matching degree between the annotated first image and the annotated second image, where the matching model is used to characterize a corresponding relationship between a pair of images and the matching degree between the images.

Claims (47)

1. A method for generating information, the method comprising:

acquiring a first image and a second image;

inputting, respectively, the first image and the second image into a pre-trained detection and recognition model, to obtain an annotated first image and an annotated second image, wherein an annotation includes an image box surrounding a target object in an image, and the detection and recognition model is used to characterize a correspondence relationship between the image and an annotated image; and

inputting the annotated first image and the annotated second image to a pre-trained matching model to obtain a matching degree between the annotated first image and the annotated second image, wherein the matching model is used to characterize a corresponding relationship between a pair of images and the matching degree between the images in the pair of images.

2. The method for generating information according to claim 1 , wherein the detection and recognition model is trained by:

acquiring a plurality of sample images and an annotated sample image corresponding to each of the plurality of sample images; and

training, by using a machine learning method, with each sample image in the plurality of sample images as an input and the annotated sample image corresponding to each sample image in the plurality of sample images as an output, to obtain the detection and recognition model.

3. The method for generating information according to claim 2 , wherein the matching model is trained by:

acquiring a plurality of pairs of annotated sample images and a matching degree between images in each pair of annotated sample images; and

training, by using a machine learning method, with each pair of annotated sample images in the plurality of pairs of annotated sample images as an input and the matching degree between the images in each pair of annotated sample images as an output, to obtain the matching model.

4. The method for generating information according to claim 1 , wherein the matching model comprises a first convolutional neural network and a second convolutional neural network.

5. The method for generating information according to claim 4 , wherein the inputting the annotated first image and the annotated second image to the pre-trained matching model to obtain the matching degree between the annotated first image and the annotated second image comprises:

inputting the annotated first image to the first convolutional neural network, to obtain an output feature vector of a target object in the annotated first image;

inputting the annotated second image to the second convolutional neural network, to obtain an output feature vector of a target object in the annotated second image; and

performing a distance calculation on the two obtained output feature vectors, to obtain a matching degree between the annotated first image and the annotated second image.

6. An apparatus for generating information, wherein the apparatus comprises:

at least one processor; and

a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring a first image and a second image;

respectfully inputting the first image and the second image into a pre-trained detection and recognition model, to obtain an annotated first image and an annotated second image, wherein an annotation includes an image box surrounding a target object in the image, and the detection and recognition model is used to characterize a correspondence relationship between the image and an annotated image; and

inputting the annotated first image and the annotated second image to a pre-trained matching model to obtain a matching degree between the annotated first image and the annotated second image, wherein the matching model is used to characterize a corresponding relationship between a pair of images and the matching degree between the images in the pair of images.

7. The apparatus for generating information according to claim 6 , wherein the detection and recognition model is trained by:

acquiring a plurality of sample images and an annotated sample image corresponding to each of the plurality of sample images; and

training, by using a machine learning method, with each sample image in the plurality of sample images as an input and the annotated sample image corresponding to each sample image in the plurality of sample images as an output, to obtain the detection and recognition model.

8. The apparatus for generating information according to claim 7 , wherein the matching model is trained by:

acquiring a plurality of pairs of annotated sample images and a matching degree between images in each pair of annotated sample images; and

training, by using a machine learning method, with each pair of annotated sample images in the plurality of pairs of annotated sample images as an input and the matching degree between the images in each pair of annotated sample images as an output, to obtain the matching model.

9. The apparatus for generating information according to claim 6 , wherein the matching model comprises a first convolutional neural network and a second convolutional neural network, the convolutional neural network comprises a convolutional layer, a pooling layer, and a fully connected layer.

10. The apparatus for generating information according to claim 9 , wherein the inputting the annotated first image and the annotated second image to the pre-trained matching model to obtain the matching degree between the annotated first image and the annotated second image comprises:

inputting the annotated first image to the first convolutional neural network, to obtain an output feature vector of a target object in the annotated first image;

inputting the annotated second image to a second convolutional neural network, to obtain an output feature vector of a target object in the annotated second image; and

performing a distance calculation on the two obtained feature vectors, to obtain a matching degree between the annotated first image and the annotated second image.

11. A non-transitory computer readable storage medium, storing a computer program thereon, the program, when executed by a processor, cause the processor to perform operations, the operations comprising:

acquiring a first image and a second image;

inputting, respectively, the first image and the second image into a pre-trained detection and recognition model, to obtain an annotated first image and an annotated second image, wherein an annotation includes an image box surrounding a target object in an image, and the detection and recognition model is used to characterize a correspondence relationship between the image and an annotated image; and

inputting the annotated first image and the annotated second image to a pre-trained matching model to obtain a matching degree between the annotated first image and the annotated second image, wherein the matching model is used to characterize a corresponding relationship between a pair of images and the matching degree between the images in the pair of images.

12. The medium according to claim 11 , wherein the detection and recognition model is trained by:

acquiring a plurality of sample images and an annotated sample image corresponding to each of the plurality of sample images; and

training, by using a machine learning method, with each sample image in the plurality of sample images as an input and the annotated sample image corresponding to each sample image in the plurality of sample images as an output, to obtain the detection and recognition model.

13. The medium according to claim 12 , wherein the matching model is trained by:

acquiring a plurality of pairs of annotated sample images and a matching degree between images in each pair of annotated sample images; and

training, by using a machine learning method, with each pair of annotated sample images in the plurality of pairs of annotated sample images as an input and the matching degree between the images in each pair of annotated sample images as an output, to obtain the matching model.

14. The medium according to claim 11 , wherein the matching model comprises a first convolutional neural network and a second convolutional neural network.

15. The medium according to claim 14 , wherein the inputting the annotated first image and the annotated second image to the pre-trained matching model to obtain the matching degree between the annotated first image and the annotated second image comprises:

inputting the annotated first image to the first convolutional neural network, to obtain an output feature vector of a target object in the annotated first image;

inputting the annotated second image to the second convolutional neural network, to obtain an output feature vector of a target object in the annotated second image; and

performing a distance calculation on the two obtained output feature vectors, to obtain a matching degree between the annotated first image and the annotated second image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2020
From: WANG, LEI
To: BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY CO., LTD; BEIJING JINGDONG CENTURY TRADING CO., LTD.
Reel/Frame 053116/0987 →
Priority Claims (1)
CN 201810102618.7 · Feb 1, 2018 · national
Continuity (1)
Related Publication 20200387744A1 · Dec 10, 2020