IP Library Granted Patent US 11,080,388
Granted Patent B2
US 11,080,388 · App. 16/150,213 · Granted Aug 3, 2021

Automatic extraction of information from obfuscated image regions

Inventors: Raoul Christopher Johnson (San Jose, CA); Bradley Wardman (Phoenix, AZ); Sai Raghavendra Maddhuri Venkata Subramaniya (West Lafayette, IN)
Assignee: PayPal, Inc.
G06F21/45G06T5/001G06T7/10H04L63/1408G06T2207/20076
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,080,388
App. No.
16/150,213
Granted
Aug 3, 2021
Kind
B2
Abstract

Images related to one or more attacks to a service provider system may be analyzed to improve the security of the service provider system. Each of the images may be segmented into multiple segments. Each of the segments is analyzed independently to determine whether the segment includes obfuscated data and if so, which one of the data obfuscation techniques was used to generate the obfuscated data. Additional information regarding the obfuscated data may be derived from other segments that include unobfuscated data and from the metadata of the image. A data restoration algorithm may be configured accordingly to restore the obfuscated data. The restored data, as well as a context derived for the image, may be used to adjust one or more security parameters of the service provider system to improve the security of the service provider system.

Claims (57)

1. A system, comprising:

a non-transitory memory; and

one or more hardware processors coupled with the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:

segmenting an image into a plurality of image segments;

determining that at least a first image segment in the plurality of image segments comprises obfuscated data;

deriving attributes associated with unobfuscated data in a subset of image segments in the plurality of image segments;

determining a layout of the image based on the unobfuscated data in the subset of image segments and relative positions of the subset of image segments within the image;

generating restored data for the obfuscated data based on the layout of the image and the derived attributes associated with the unobfuscated data;

determining that the image comprises transaction data associated with one or more electronic transactions based on the restored data; and

configuring a risk engine to assess a risk level of incoming electronic transaction requests based at least in part on the transaction data.

2. The system of claim 1 , wherein the derived attributes comprise at least one of a data type associated with the unobfuscated data, a font type associated with the unobfuscated data, a font size associated with the unobfuscated data, or a particular character existing in the unobfuscated data.

3. The system of claim 1 , wherein the operations further comprise:

determining a first data obfuscation algorithm used to generate the obfuscated data; and

selecting, from a plurality of data restoration algorithms, a first data restoration algorithm corresponding to the first data obfuscation algorithm.

4. The system of claim 3 , wherein the obfuscated data is first obfuscated data, wherein the restored data is first restored data, and wherein the operations further comprise:

determining that a second image segment in the plurality of image segments comprises second obfuscated data;

determining a second data obfuscation algorithm used to generate the second obfuscated data, wherein the second data obfuscation algorithm is different from the first data obfuscation algorithm;

selecting, from the plurality of data restoration algorithms, a second data restoration algorithm corresponding to the second data obfuscation algorithm; and

generating second restored data for the second obfuscated data using the second data restoration algorithm based on the layout of the image and the derived attributes.

5. The system of claim 3 , wherein the operations further comprise extracting metadata from the image, wherein the first data restoration algorithm is selected from the plurality of data restoration algorithms based at least in part on the extracted metadata.

6. The system of claim 5 , wherein the metadata comprises data indicating a computing environment in which the image was captured and/or edited.

7. The system of claim 3 , wherein the determining the first data obfuscation algorithm used to generate the obfuscated data comprises:

calculating a variance measure based on pixel data in the first image segment; and

determining that the variance measure is within a predetermined variance range corresponding to a blurriness measure.

8. The system of claim 3 , wherein the determining the first data obfuscation algorithm used to generate the obfuscated data comprises:

calculating a standard deviation based on pixel data within the first image segment; and

determining that the standard deviation is below a predetermined threshold corresponding to a masking measure.

9. The system of claim 1 , wherein the operations further comprise modifying an authentication setting associated with a user account based at least in part on the restored data.

10. A method comprising:

segmenting, by one or more hardware processors, an image into a plurality of image segments;

determining, by the one or more hardware processors, that at least a first image segment in the plurality of image segments comprises obfuscated data;

deriving, by the one or more hardware processors, attributes associated with unobfuscated data in a subset of image segments in the plurality of image segments;

determining, by the one or more hardware processors, a layout of the image based on the unobfuscated data in the subset of image segments and relative positions of the subset of image segments within the image;

generating, by the one or more hardware processors, restored data for the obfuscated data based on the layout of the image and the derived attributes;

determining that the image comprises transaction data associated with one or more electronic transactions based on the restored data; and

configuring a risk engine to assess a risk level of incoming electronic transaction requests based at least in part on the transaction data.

11. The method of claim 10 , further comprising:

determining a computing environment in which the image was captured and/or edited.

12. The method of claim 11 , further comprising extracting metadata from the image, wherein the computing environment is determined based on the extracted metadata.

13. The method of claim 11 , wherein the restored data is generated further based on the computing environment.

14. The method of claim 10 , wherein the derived attributes comprise at least one of a data type associated with the unobfuscated data, a font type associated with the unobfuscated data, or a font size associated with the unobfuscated data.

15. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

segmenting an image into a plurality of image segments;

determining that at least a first image segment in the plurality of image segments comprises obfuscated data;

analyzing unobfuscated data in a subset of image segments in the plurality of image segments;

deriving attributes associated with the unobfuscated data based on the analyzing;

determining a layout of the image based on the unobfuscated data in the subset of image segments and relative positions of the subset of image segments within the image;

generating restored data for the obfuscated data based on the layout of the image and the derived attributes;

determining that the image comprises transaction data associated with one or more electronic transactions based on the restored data; and

configuring a risk engine to assess a risk level of incoming electronic transaction requests based at least in part on the transaction data.

16. The non-transitory machine-readable medium of claim 15 , wherein the restored data comprises text data.

17. The non-transitory machine-readable medium of claim 15 , wherein the image corresponds to a webpage associated with a service provider.

18. The non-transitory machine-readable medium of claim 15 , wherein the derived attributes comprise at least one of a data type associated with the unobfuscated data, a font type associated with the unobfuscated data, a font size associated with the unobfuscated data, or a particular character existing in the unobfuscated data.

19. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:

determining a data obfuscation algorithm used to generate the obfuscated data; and

selecting, from a plurality of data restoration algorithms, a data restoration algorithm corresponding to the data obfuscation algorithm.

20. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise modifying an authentication setting associated with a user account based at least in part on the restored data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2018
From: JOHNSON, RAOUL CHRISTOPHER; WARDMAN, BRADLEY; MADDHURI VENKATA SUBRAMANIYA, SAI RAGHAVENDRA
To: PAYPAL, INC.
Reel/Frame 047167/0415 →
Continuity (1)
Related Publication 20200159907A1 · May 21, 2020