IP Library › Granted Patent US 10,958,704
Granted Patent B2
US 10,958,704 · App. 16/557,558 · Granted Mar 23, 2021

Feature generation for online/offline machine learning

Inventors: Mohammad Hossein Taghavi (Sunnyvale, CA); Prasanna Padmanabhan (Fremont, CA); Dong-Bang Tsai (San Jose, CA); Faisal Zakaria Siddiqi (San Jose, CA); Justin Derrick Basilico (Saratoga, CA)
Assignee: Netflix, Inc.
H04L65/602G06N5/04G06N20/00H04L65/607H04L67/22G06F16/335G06F16/9535H04L67/303H04L67/306H04N21/2407H04N21/252H04N21/25891
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,958,704
App. No.
16/557,558
Granted
Mar 23, 2021
Kind
B2
Abstract

A system for utilizing models derived from offline historical data in online applications is provided. The system includes a processor and a memory storing machine-readable instructions for determining a set of contexts of the usage data, and for each of the contexts within the set of contexts, collecting service data from services supporting the media service and storing that service data in a database. The system performing an offline testing process by fetching service data for a defined context from the database, generating a first set of feature vectors based on the fetched service data, and providing the first set to a machine-learning module. The system performs an online testing process by fetching active service data from the services supporting the media streaming service, generating a second set of feature vectors based on the fetched active service data, and providing the second set to the machine-learning module.

Claims (52)

1. A system comprising:

a processor; and

a memory storing machine-readable instructions that when executed by the processor, cause the processor to perform operations comprising:

collecting service data from a media service;

performing an offline testing process comprising:

fetching at least some of the collected service data,

generating a first set of features based on the fetched service data using a specified feature encoder that is configured to encode the first set of features in a specified manner, and

providing the first set of features to a machine-learning module; and

performing an online testing process comprising:

fetching active service data from the media service,

generating a second set of features based on the fetched active service data using the same specified feature encoder the same encoding allowing the system to dynamically transition between the offline testing process and the online testing process, and

providing the second set of features to the machine-learning module.

2. The system of claim 1 , wherein the offline testing process further implements one or more portions of snapshot usage data to generate the first set of features.

3. The system of claim 1 , wherein the offline testing process further creates one or more data maps according to a feature model designed to test specified features, the data maps being configured to map data keys to specific data objects that are used to generate the first set of features.

4. The system of claim 1 , wherein the machine-readable instructions further cause the system to perform operations comprising determining a confidence value for a snapshot of collected usage data.

5. The system of claim 1 , wherein collecting usage data further comprises transforming service objects from the services into a condensed format.

6. The system of claim 5 , wherein fetching service data comprises restoring the service objects to a non-condensed format.

7. The system of claim 1 , wherein performing the offline testing process further comprises receiving label data defining contexts to be fetched.

8. The system of claim 7 , wherein the label data further defines items to be scored for the offline testing process.

9. The system of claim 1 , wherein the service data includes context-dependent data.

10. The system of claim 1 , wherein the service data includes context-independent data.

11. A method comprising:

collecting service data from a primary service;

performing an offline testing process comprising:

fetching at least some of the collected service data for a defined context from the database,

generating a first set of features based on the fetched service data using a specified feature encoder that is configured to encode the first set of features in a specified manner, and

providing the first set of features to a machine-learning function; and

performing an online testing process comprising:

fetching active service data from the primary service,

generating a second set of features based on the fetched active service data using the same specified feature encoder, the same encoding allowing dynamic transitions between the offline testing process and the online testing process, and

providing the second set of features to the machine-learning function.

12. The method of claim 11 , further comprising:

performing an online testing process comprising:

fetching active service data from the secondary services;

generating a second set of features based on the fetched active service data; and

providing the second set of features to the machine-learning function.

13. The method of claim 11 , wherein the primary service is a media streaming service.

14. The method of claim 13 , wherein the media streaming service utilizes a service-oriented architecture.

15. The method of claim 11 , further comprising, determining a confidence value for a snapshot of collected usage data.

16. The method of claim 11 , wherein the secondary services include at least one of: a viewing history service, a user media list service, or a ratings service.

17. The method of claim 11 , wherein the contexts include one of: user profiles, device types, or time period.

18. A method comprising:

performing an offline testing process comprising:

fetching service data from a primary service,

generating a first set of features based on the fetched service data using a specified feature encoder that is configured to encode the first set of features in a specified manner, and

providing the first set of features into a machine-learning function; and

performing an online testing process comprising:

fetching active service data from secondary services supporting the primary service,

generating a second set of features based on the fetched active service data using the same specified feature encoder, the same encoding allowing dynamic transitions between the offline testing process and the online testing process, and

providing the second set of features to the machine-learning function.

19. The method of claim 18 , wherein the machine-learning function generates a likelihood that a specified user will accept a recommended piece of content based on the offline and online testing using the first and second sets of features.

20. The method of claim 18 , wherein the offline testing process is initiated before the online testing process begins fetching active service data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2019
From: TAGHAVI, MOHAMMAD HOSSEIN; PADMANABHAN, PRASANNA; TSAI, DONG-BANG; SIDDIQI, FAISAL ZAKARIA; BASILICO, JUSTIN DERRICK
To: NETFLIX, INC
Reel/Frame 050655/0925 →
Continuity (3)
Continuation 15331106 · Oct 21, 2016
Provisional Application 62295462 · Feb 15, 2016
Related Publication 20190394252A1 · Dec 26, 2019