IP Library Granted Patent US 10,762,607
Granted Patent B2
US 10,762,607 · App. 16/820,535 · Granted Sep 1, 2020

Method and device for sensitive data masking based on image recognition

Inventors: Fuheng Xie (Hangzhou, CN); Kun Lian (Hangzhou, CN); Qingcheng Wei (Hangzhou, CN)
Assignee: Alibaba Group Holding Limited
G06T5/004G06K9/4604G06K9/6256G06T2207/20081G06T2207/20132
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,762,607
App. No.
16/820,535
Granted
Sep 1, 2020
Kind
B2
Abstract

A method for masking sensitive data based on image recognition comprises: extracting initial image features of a to-be-processed image in multiple dimensions of features; reducing dimensions for the initial image features in the multiple dimensions of features, to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking; identifying sensitive features of the to-be-processed image based on the image features in the at least one dimension of sensitive data masking; masking the sensitive data for the sensitive features of the to-be-processed image. The method for masking sensitive data based on image recognition realizes the sensitive data masking for images based on image recognition, and the efficiency of sensitive data masking is relatively high.

Claims (74)

1. A method for masking sensitive data based on image recognition, comprising:

extracting initial image features of a to-be-processed image in multiple dimensions of features by extracting feature vectors corresponding to the initial image features of the to-be-processed image in the multiple dimensions of features, wherein a numerical value of vector dimensions of the feature vectors is equal to a numerical value of the multiple dimensions of features;

reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking by performing a dimension-reduction calculation to obtain the feature vectors of the to-be-processed image in the at least one dimension of sensitive data masking;

identifying sensitive features of the to-be-processed image based on the image features in the at least one dimension of sensitive data masking; and

masking the sensitive features of the to-be-processed image.

2. The method for masking sensitive data based on image recognition according to claim 1 , wherein the at least one dimension of sensitive data masking comprises at least one of a dimension of image color, a dimension of image feature position, and a dimension of image feature identifier.

3. The method for masking sensitive data based on image recognition according to claim 1 , wherein the method, after the extracting initial image features of a to-be-processed image in multiple dimensions of features, and before the reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking, further comprises:

preprocessing the to-be-processed image based on the initial image features in the multiple dimensions of features.

4. The method for masking sensitive data based on image recognition according to claim 3 , wherein, if the multiple dimensions of features include a dimension of image feature position, the preprocessing the to-be-processed image based on the initial image features in the multiple dimensions of features comprises:

determining, according to the initial image features in the dimension of image feature position, position information of the image features of the to-be-processed image; and

cropping the to-be-processed image based on the position information of the image features of the to-be-processed image.

5. The method for masking sensitive data based on image recognition according to claim 1 , wherein the method, before the extracting initial image features of a to-be-processed image in multiple dimensions of features, further comprises:

acquiring the to-be-processed image included in materials of a project submitted by a project member during project participation.

6. The method for masking sensitive data based on image recognition according to claim 1 , wherein the reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking comprises:

obtaining, by training, a dimension-reduction model of image features, and

inputting, into the trained dimension-reduction model of image features, the initial image features of the to-be-processed image in the multiple dimensions of the features to perform features dimension reduction, and to output the image features of the to-be-processed image in the at least one dimension of sensitive data masking.

7. The method for masking sensitive data based on image recognition according to claim 6 , wherein the obtaining, by training, a dimension-reduction model of image features comprises:

acquiring an initial image set including initial images on which sensitive data is masked;

extracting, from the initial image set, initial image features of the initial images of the initial image set in the multiple dimensions of features;

reducing dimensions for the initial image features in the multiple dimensions of features, to obtain real image features of the initial images in the at least one dimension of sensitive data masking; and

training, with first training samples including the initial image features in the multiple dimensions of the features and the real image features in the at least one dimension of sensitive data masking, a pre-built dimension-reduction model of the initial image features to obtain the dimension-reduction model of the image features.

8. The method for masking sensitive data based on image recognition according to claim 7 , wherein the identifying sensitive features of the to-be-processed image based on the image features in the at least one dimension of sensitive data masking comprises:

obtaining, by training, an image recognition model, and

inputting, into the trained image recognition model, the to-be-processed image having the image features in the at least one dimension of sensitive data masking to identify the sensitive features, and to output the sensitive features of the to-be-processed image.

9. The method for masking sensitive data based on image recognition according to claim 8 , wherein the obtaining, by training, an image recognition model comprises:

training a pre-built initial image recognition model with second training samples including the initial images having the real image features in the at least one dimension of sensitive data masking and the initial images having the real sensitive features to obtain the image recognition model.

10. A computing device, comprising:

one or more processors; and

a memory storing instructions executable by the one or more processors to causes the computing device to perform operations comprising:

extracting initial image features of a to-be-processed image in multiple dimensions of features by extracting feature vectors corresponding to the initial image features of the to-be-processed image in the multiple dimensions of features, wherein a numerical value of vector dimensions of the feature vectors is equal to a numerical value of the multiple dimensions of features;

reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking by performing a dimension-reduction calculation to obtain the feature vectors of the to-be-processed image in the at least one dimension of sensitive data masking;

identifying sensitive features of the to-be-processed image based on the image features in the at least one dimension of sensitive data masking; and

masking the sensitive features of the to-be-processed image.

11. The computing device according to claim 10 , wherein the operations further comprise:

acquiring an initial image set including initial images on which sensitive data is masked;

extracting, from the initial image set, initial image features of the initial images of the initial image set in the multiple dimensions of features;

reducing dimensions for the initial image features in the multiple dimensions of features, to obtain real image features of the initial images in at least one dimension of sensitive data masking;

training, with first training samples including the initial image features in the multiple dimensions of features and the real image features in the at least one dimension of sensitive data masking, a pre-built dimension-reduction model of image features to obtain the dimension-reduction model of the image features;

training a pre-built image recognition model with second training sample including the initial images having the real image features in the at least one dimension of sensitive data masking and the initial images having real sensitive features.

12. The computing device according to claim 10 , wherein the at least one dimension of sensitive data masking comprises at least one of a dimension of image color, a dimension of image feature position, and a dimension of image feature identifier.

13. A non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

extracting initial image features of a to-be-processed image in multiple dimensions of features by extracting feature vectors corresponding to the initial image features of the to-be-processed image in the multiple dimensions of features, wherein a numerical value of vector dimensions of the feature vectors is equal to a numerical value of the multiple dimensions of features;

reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking by performing a dimension-reduction calculation to obtain the feature vectors of the to-be-processed image in the at least one dimension of sensitive data masking;

identifying sensitive features of the to-be-processed image based on the image features in the at least one dimension of sensitive data masking; and

masking for the sensitive features of the to-be-processed image.

14. The non-transitory computer-readable storage medium according to claim 13 , wherein the operations further comprise:

acquiring an initial image set including initial images corresponding to images on which sensitive data is masked;

extracting, from the initial image set, initial image features of the initial images in the multiple dimensions of features;

reducing dimensions for the initial image features in the multiple dimensions of features, to obtain real image features of the initial images in at least one dimension of sensitive data masking;

training, with first training samples including the initial image features in the multiple dimensions of features and the real image features in the at least one dimension of sensitive data masking, a pre-built dimension-reduction model of image features to obtain the dimension-reduction model of the image features;

train a pre-built image recognition model with second training sample including the initial images having the real image features in the at least one dimension of sensitive data masking and the initial images having real sensitive features.

15. The non-transitory computer-readable storage medium according to claim 13 , wherein

the extracting initial image features of a to-be-processed image in multiple dimensions of features comprises: extracting feature vectors corresponding to the initial image features of the to-be-processed image in the multiple dimensions of features; and

a numerical value of vector dimensions of the feature vectors is equal to a numerical value of the multiple dimensions of features.

16. The computing device according to claim 15 , wherein the reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking comprises: performing a dimension-reduction calculation to obtain the feature vectors of the to-be-processed image in the at least one dimension of sensitive data masking.

17. A non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

extracting initial image features of a to-be-processed image in multiple dimensions of features, wherein the multiple dimensions of features include a dimension of image feature position;

preprocessing the to-be-processed image based on the initial image features in the multiple dimensions of features by:

determining, according to the initial image features in the dimension of image feature position, position information of the image features of the to-be-processed image; and

cropping the to-be-processed image based on the position information of the image features of the to-be-processed image;

reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking;

identifying sensitive features of the to-be-processed image based on the image features in the at least one dimension of sensitive data masking; and

masking the sensitive features of the to-be-processed image.

18. A non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

extracting initial image features of a to-be-processed image in multiple dimensions of features;

reducing dimensions for the initial image features in the multiple dimensions of features to obtain image features of the to-be-processed image in at least one dimension of sensitive data masking by:

obtaining, by training, a dimension-reduction model of image features, and

inputting, into the trained dimension-reduction model of image features, the initial image features of the to-be-processed image in the multiple dimensions of the features to perform features dimension reduction, and to output the image features of the to-be-processed image in the at least one dimension of sensitive data masking;

identifying sensitive features of the to-be-processed image based on the image features in the at least one dimension of sensitive data masking; and

masking the sensitive features of the to-be-processed image, wherein the obtaining, by training, a dimension-reduction model of image features comprises:

acquiring an initial image set including initial images on which sensitive data is masked,

extracting, from the initial image set, initial image features of the initial images of the initial image set in the multiple dimensions of features,

reducing dimensions for the initial image features in the multiple dimensions of features, to obtain real image features of the initial images in the at least one dimension of sensitive data masking, and

training, with first training samples including the initial image features in the multiple dimensions of the features and the real image features in the at least one dimension of sensitive data masking, a pre-built dimension-reduction model of the initial image features to obtain the dimension-reduction model of the image features.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 16, 2020
From: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
To: ADVANCED NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053796/0281 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 3, 2020
From: ALIBABA GROUP HOLDING LIMITED
To: ADVANTAGEOUS NEW TECHNOLOGIES CO., LTD.
Reel/Frame 053702/0392 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 7, 2020
From: XIE, FUHENG; LIAN, KUN; WEI, QINGCHENG
To: ALIBABA GROUP HOLDING LIMITED
Reel/Frame 052603/0856 →