IP Library › Granted Patent US 10,853,362
Granted Patent B2
US 10,853,362 · App. 15/131,904 · Granted Dec 1, 2020

Optimized full-spectrum loglog-based cardinality estimation

Inventors: Jason Jinshui Qin (Great Falls, VA); Denys Kim (Fairfax, VA); Yumei Tung (Vienna, VA)
Assignee: Verizon Media Inc.
G06F16/2453G06F16/2255
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,853,362
App. No.
15/131,904
Granted
Dec 1, 2020
Kind
B2
Abstract

Systems and methods are disclosed for optimizing full-spectrum cardinality approximations on big data utilizing an optimized LogLog counting technique. To accomplish the foregoing, a multiset of objects that each corresponds to one of a plurality of objects associated with a resource are obtained. A compound data object is populated at least in part with data that is derived based on generated hash values that correspond to each object in the obtained multiset. The populated compound data object is processed with a full-spectrum harmonic mean estimation operation that can accurately determine a cardinality estimate for the obtained multiset using less resources and time when compared to traditional techniques. The determination is further made without the need to employ linear counting or bias correction operations on low or high cardinalities. An estimated number of unique objects in the obtained multiset is determined as a result of the processing, and subsequently provided for display or further manipulation.

Claims (42)

1. A computer-implemented method comprising:

receiving a request for a number of unique objects in a plurality of objects associated with a resource, wherein the plurality of objects comprise at least two of an IP address associated with the resource, cookie data associated with the resource, an email address associated with the resource, a username associated with the resource or a file name associated with the resource;

obtaining a multiset of objects that each corresponds to a different one of the plurality of objects associated with the resource, wherein the multiset is at least some of the plurality of objects associated with the resource, wherein the obtaining comprises obtaining a first object of the plurality of objects associated with the resource and obtaining a second object of the plurality of objects associated with the resource, wherein the first object is different than the second object, wherein the first object comprises the IP address associated with the resource, wherein the second object comprises the cookie data associated with the resource;

generating a hash value for each object in the multiset, wherein the generating comprises generating a first hash value for the first object associated with the resource and generating a second hash value for the second object associated with the resource;

populating at least portions of a compound data object with data based on generated hash values, including the first hash value and the second hash value, that correspond to each object in the multiset including the first object and the second object;

processing the compound data object, after the populating, with a single procedure to determine an estimated number of unique objects in the multiset, wherein the single procedure is a self-adjusting procedure configured to automatically adjust processing of the compound data object for any actual number of unique objects in the multiset; and

providing the estimated number of unique objects in the multiset.

2. The method of claim 1 , wherein the plurality of objects comprise the IP address associated with the resource.

3. The method of claim 1 , wherein the plurality of objects comprise the cookie data associated with the resource.

4. The method of claim 1 , wherein the request includes a particular time period, and wherein each object of the multiset corresponds to one of the plurality of objects associated with the resource having a timestamp corresponding to the particular time period included in the request.

5. The method of claim 1 , further comprising initializing the compound data object having a plurality of compound object portions, each portion associated with the compound data object being unpopulated.

6. The method of claim 5 , each portion associated with the compound data object being one of the plurality of compound object portions associated with the compound data object.

7. The method of claim 1 , the compound data object having a total number of portions based on an average accuracy requirement, the total number of portions being a size of the compound data object.

8. The method of claim 7 , populating at least portions of the compound data object with data comprising:

determining, for each generated hash value, a first value that is based on a first portion of a hash,

obtaining, for each generated hash value, a second value that is stored in a corresponding portion of the compound data object, the corresponding portion being based on a second portion of the hash, and

storing, for each generated hash value, at least one of the first value or the second value into the corresponding portion of the compound data object based on a comparison of the first value and the second value.

9. The method of claim 8 , the first portion of the hash being a latter portion of the hash, and the second portion of the hash being a former portion of the hash, wherein both the first portion and the second portion of the hash are based on the size of the compound data object.

10. The method of claim 1 , wherein the plurality of objects comprise the email address associated with the resource.

11. The method of claim 1 , wherein the plurality of objects comprise the username associated with the resource.

12. The method of claim 1 , wherein the single procedure includes at least a balancing factor to facilitate processing of the compound data object for any actual number of unique objects in the multiset.

13. The method of claim 12 , wherein the balancing factor is based on a number of populated portions of the compound data object.

14. The method of claim 1 , the single procedure being a single complex operation for determining the estimated number of unique objects in the multiset.

15. The method of claim 14 , the processing not including a secondary procedure for determining the estimated number of unique objects in the multiset, the secondary procedure including at least one of a bias correction operation and a linear counting operation.

16. The method of claim 1 , wherein the plurality of objects comprise the file name associated with the resource.

17. A non-transitory computer storage medium storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:

receiving a request for a number of unique objects in a plurality of objects associated with a resource, the request including a particular time period, wherein the plurality of objects comprise at least one of an IP address associated with the resource, cookie data associated with the resource, an email address associated with the resource, a username associated with the resource or a file name associated with the resource;

obtaining a multiset of objects that each has a timestamp corresponding to the particular time period included in the request and further corresponds to one of the plurality of objects associated with the resource, wherein the multiset is at least some of the plurality of objects associated with the resource, wherein the obtaining comprises obtaining a first object of the plurality of objects associated with the resource and obtaining a second object of the plurality of objects associated with the resource, wherein the first object is different than the second object;

generating a hash value for each object in the multiset, wherein the generating comprises generating a first hash value for the first object associated with the resource and generating a second hash value for the second object associated with the resource;

populating at least portions of a compound data object based on generated hash values, including the first hash value and the second hash value, that correspond to each object in the multiset including the first object and the second object;

processing the compound data object, after the populating, with a self-adjusting procedure implementing a full-spectrum cardinality formula to determine an estimated number of unique objects in the multiset, the estimated number of unique objects being substantially equivalent to an actual number of unique objects in the multiset; and

providing the estimated number of unique objects in the multiset.

18. The non-transitory computer storage medium of claim 17 , the compound data object having a number of portions that is based on an accuracy requirement.

19. A computerized system comprising:

one or more processors;

one or more computer storage media comprising instructions that when executed by the one or more processors perform operations comprising:

receiving a request for a number of unique objects in a plurality of objects associated with a resource, wherein the plurality of objects comprise at least two of an IP address associated with the resource, cookie data associated with the resource, an email address associated with the resource, a username associated with the resource or a file name associated with the resource;

obtaining a multiset of objects that each corresponds to a different one of the plurality of objects associated with the resource, wherein the multiset is at least some of the plurality of objects associated with the resource, wherein the obtaining comprises obtaining a first object of the plurality of objects associated with the resource and obtaining a second object of the plurality of objects associated with the resource, wherein the first object is different than the second object, wherein the first object comprises the IP address associated with the resource, wherein the second object comprises at least one of the cookie data associated with the resource, the email address associated with the resource, the username associated with the resource or the file name associated with the resource;

generating a hash value for each object in the multiset, wherein the generating comprises generating a first hash value for the first object associated with the resource and generating a second hash value for the second object associated with the resource;

populating at least portions of a compound data object with data based on generated hash values, including the first hash value and the second hash value, that correspond to each object in the multiset including the first object and the second object; and

processing the compound data object, after the populating, with a single procedure to determine an estimated number of unique objects in the multiset, wherein the single procedure is configured to automatically adjust processing of the compound data object for any actual number of unique objects in the multiset.

20. The computerized system of claim 19 , the operations comprising providing the estimated number of unique objects in the multiset.

Assignments (6)
PATENT SECURITY AGREEMENT (FIRST LIEN) Recorded Sep 29, 2022
From: YAHOO ASSETS LLC
To: ROYAL BANK OF CANADA, AS COLLATERAL AGENT
Reel/Frame 061571/0773 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2021
From: YAHOO AD TECH LLC (FORMERLY VERIZON MEDIA INC.)
To: YAHOO ASSETS LLC
Reel/Frame 058982/0282 →
CHANGE OF NAME Recorded Feb 24, 2020
From: OATH (AMERICAS) INC.
To: VERIZON MEDIA INC.
Reel/Frame 051999/0720 →
CHANGE OF NAME Recorded Dec 20, 2017
From: AOL ADVERTISING INC.
To: OATH (AMERICAS) INC.
Reel/Frame 044957/0456 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE ADDRESS PREVIOUSLY RECORDED ON REEL 038321 FRAME 0888. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 7, 2017
From: QUIN, JASON JINSHUI; KIM, DENYS; TUNG, YUMEI
To: AOL ADVERTISING INC.
Reel/Frame 041903/0831 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 19, 2016
From: QIN, JASON JINSHUI; KIM, DENYS; TUNG, YUMEI
To: AOL ADVERTISING INC.
Reel/Frame 038321/0888 →
Continuity (1)
Related Publication 20170300528A1 · Oct 19, 2017