IP Library Granted Patent US 10,943,175
Granted Patent B2
US 10,943,175 · App. 15/359,971 · Granted Mar 9, 2021

Methods, systems and apparatus to improve multi-demographic modeling efficiency

Inventors: Michael Sheppard (Holland, MI); Ludo Daemen (Duffel, BE); Xiaoqi Cui (Darien, IL)
Assignee: The Nielsen Company (US), LLC
G06N7/005G06Q30/0201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,943,175
App. No.
15/359,971
Granted
Mar 9, 2021
Kind
B2
Abstract

Methods, apparatus, systems and articles of manufacture are disclosed to improve multi-demographic modeling efficiency. An example apparatus includes a feature set aggregator to segregate training data based on feature sets of interest, and to identify households that participate in at least one of the feature sets of interest, a class enumerator to reduce multi-demographic model iterations by enumerating demographic combinations for the identified households, the enumerated demographic combinations including a single identifier to represent a combination of two or more demographic categories, and a modeling engine to generate training coefficients associated with respective ones of the enumerated demographic combinations.

Claims (43)

1. An apparatus to reduce computational resources for a multi-demographic training model, comprising:

a feature set aggregator to:

segregate training data based on feature sets of interest; and

identify households that participate in at least one of the feature sets of interest;

a class enumerator to reduce multi-demographic model iterations by enumerating demographic combinations for the identified households, the enumerated demographic combinations including a single identifier to represent a combination of two or more demographic categories;

a linear combiner to:

select a demographic category of interest from the two or more demographic categories;

discriminate which ones of the enumerated demographic combinations are to be selected based on whether the selected demographic category of interest is present therein; and

when the selected demographic category of interest is present, select the respective ones of the enumerated demographic combinations based on the single identifier, the single identifier including cartesian representations that reflect occurrences or non-occurrences of the selected demographic category of interest; and

a modeling engine to generate training coefficients corresponding to the selected ones of the enumerated demographic combinations.

2. The apparatus as defined in claim 1 , further including a household data associator to associate demographic combinations with respective ones of the identified households.

3. The apparatus as defined in claim 1 , wherein the modeling engine is to perform a multinomial logistic regression with the segregated training data and the selected ones of the enumerated demographic combinations to generate the training coefficients.

4. The apparatus as defined in claim 1 , wherein the modeling engine is to generate probability values for the selected ones of the enumerated demographic combinations.

5. The apparatus as defined in claim 4 , wherein the linear combiner is to revert the probability values for respective ones of the enumerated demographic combinations to probability values associated with individual demographic components.

6. The apparatus as defined in claim 1 , further including a data retriever to retrieve a third party model, the modeling engine to generate probability values based on the segregated training data for the selected ones of the enumerated demographic combinations associated with (a) a training model and (b) the third party model.

7. The apparatus as defined in claim 6 , wherein the linear combiner is to determine probability value differences between the training model and the third party model.

8. The apparatus as defined in claim 7 , wherein the linear combiner is to compare the probability value differences to a threshold to determine a trust metric for the third party model.

9. A computer-implemented method to reduce computational resources for a multi-demographic training model, the method comprising:

segregating, by executing an instruction with a processor, training data based on feature sets of interest;

identifying, by executing an instruction with the processor, households that participate in at least one of the feature sets of interest;

reducing, by executing an instruction with the processor, multi-demographic model iterations by enumerating demographic combinations for the identified households, the enumerated demographic combinations including a single identifier to represent a combination of two or more demographic categories;

selecting a demographic category of interest from the two or more demographic categories;

discriminating which ones of the enumerated demographic combinations to select based on whether the selected demographic category of interest is present therein;

when the selected demographic category of interest is present, selecting respective ones of the enumerated demographic combinations based on the single identifier, the single identifier including cartesian representations that reflect occurrences or non-occurrences of the selected demographic category of interest; and

generating, by executing an instruction with the processor, training coefficients corresponding to the selected ones of the enumerated demographic combinations.

10. A method as defined in claim 9 , further including associating demographic combinations with respective ones of the identified households.

11. A method as defined in claim 9 , further including performing a multinomial logistic regression with the segregated training data and the selected ones of the enumerated demographic combinations to generate the training coefficients.

12. A method as defined in claim 9 , further including generating probability values for the selected ones of the enumerated demographic combinations.

13. A method as defined in claim 12 , further including reverting the probability values for respective ones of the enumerated demographic combinations to probability values associated with individual demographic components.

14. A method as defined in claim 9 , further including retrieving a third party model, and wherein the generating probability values is based on the segregated training data for the selected ones of the enumerated demographic combinations associated with (a) a training model and (b) the third party model.

15. A method as defined in claim 14 , further including determining probability value differences between the training model and the third party model.

16. A method as defined in claim 15 , further including comparing the probability value differences to a threshold to determine a trust metric for the third party model.

17. A tangible computer-readable storage medium comprising instructions that, when executed, cause a processor to, at least:

segregate training data based on feature sets of interest;

identify households that participate in at least one of the feature sets of interest;

reduce multi-demographic model iterations by enumerating demographic combinations for the identified households, the enumerated demographic combinations including a single identifier to represent a combination of two or more demographic categories;

select a demographic category of interest from the two or more demographic categories;

discriminate which ones of the enumerated demographic combinations to select based on whether the selected demographic category of interest is present therein;

when the selected demographic category of interest is present, select respective ones of the enumerated demographic combination based on the single identifier, the single identifier including cartesian representations that reflect occurrences or non-occurrences of the selected demographic category of interest; and

generate training coefficients corresponding to the selected ones of the enumerated demographic combinations.

18. The computer-readable medium as defined in claim 17 including instructions that, when executed, cause the processor to associate demographic combinations with respective ones of the identified households.

19. The computer-readable medium as defined in claim 17 including instructions that, when executed, cause the processor to perform a multinomial logistic regression with the segregated training data and the selected ones of the enumerated demographic combinations to generate the training coefficients.

20. The computer-readable medium as defined in claim 17 including instructions that, when executed, cause the processor to generate probability values for the selected ones of the enumerated demographic combinations.

Assignments (8)
RELEASE (REEL 054066 / FRAME 0064) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063605/0001 →
RELEASE (REEL 053473 / FRAME 0001) Recorded May 11, 2023
From: CITIBANK, N.A.
To: A. C. NIELSEN COMPANY, LLC; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE MEDIA SERVICES, LLC; THE NIELSEN COMPANY (US), LLC; NETRATINGS, LLC
Reel/Frame 063603/0001 →
SECURITY INTEREST Recorded May 8, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: ARES CAPITAL CORPORATION
Reel/Frame 063574/0632 →
SECURITY INTEREST Recorded Apr 28, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: CITIBANK, N.A.
Reel/Frame 063561/0381 →
SECURITY AGREEMENT Recorded Jan 31, 2023
From: GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; GRACENOTE, INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 063560/0547 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENTS LISTED ON SCHEDULE 1 RECORDED ON 6-9-2020 PREVIOUSLY RECORDED ON REEL 053473 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE SUPPLEMENTAL IP SECURITY AGREEMENT. Recorded Oct 7, 2020
From: A.C. NIELSEN (ARGENTINA) S.A.; A.C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A
Reel/Frame 054066/0064 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Jun 9, 2020
From: A. C. NIELSEN COMPANY, LLC; ACN HOLDINGS INC.; ACNIELSEN CORPORATION; ACNIELSEN ERATINGS.COM; AFFINNOVA, INC.; ART HOLDING, L.L.C.; ATHENIAN LEASING CORPORATION; CZT/ACN TRADEMARKS, L.L.C.; EXELATE, INC.; GRACENOTE, INC.; GRACENOTE DIGITAL VENTURES, LLC; GRACENOTE MEDIA SERVICES, LLC; NETRATINGS, LLC; NIELSEN AUDIO, INC.; NIELSEN CONSUMER INSIGHTS, INC.; NIELSEN CONSUMER NEUROSCIENCE, INC.; NIELSEN FINANCE CO.; NIELSEN FINANCE LLC; NIELSEN INTERNATIONAL HOLDINGS, INC.; NIELSEN MOBILE, LLC; NIELSEN UK FINANCE I, LLC; NMR INVESTING I, INC.; TCG DIVESTITURE INC.; TNC (US) HOLDINGS, INC.; THE NIELSEN COMPANY (US), LLC; VIZU CORPORATION; VNU MARKETING INFORMATION, INC.; NMR LICENSING ASSOCIATES, L.P.; NIELSEN HOLDING AND FINANCE B.V.; THE NIELSEN COMPANY B.V.; VNU INTERNATIONAL B.V.
To: CITIBANK, N.A.
Reel/Frame 053473/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 13, 2017
From: SHEPPARD, MICHAEL; DAEMEN, LUDO; CUI, XIAOQI
To: THE NIELSEN COMPANY (US), LLC
Reel/Frame 040967/0329 →
Continuity (1)
Related Publication 20180144267A1 · May 24, 2018