IP Library Granted Patent US 11,989,248
Granted Patent B2
US 11,989,248 · App. 18/073,160 · Granted May 21, 2024

Content item selection in a digital transaction management platform

Inventors: Shrinivas Kiran Kaza (Fremont, CA); Taiwo Raphael Alabi (San Jose, CA); Farzaneh Rajabi (Stanford, CA); Ashwath Mohan (San Ramon, CA); Kim Cuong Phung (Hayward, CA)
Assignee: DOCUSIGN, INC.
G06F16/9538G06F3/0482G06F9/451G06F16/24578G06F16/93G06F16/9535G06F40/166G06N20/00G06Q10/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,989,248
App. No.
18/073,160
Granted
May 21, 2024
Kind
B2
Abstract

A centralized document system identifies content items for presentation to a user based initially on a cold-start algorithm and subsequently based on machine-learned models. The system detects a first access by the user. The system generates a user attribute vector for the user and a content vector for each content item. The system selects a first content item based on the initial cold-start algorithm and modifies a user interface to include the first content item. The system identifies an interaction with the first content item by the user. The system detects a second access by the user. The system selects a machine-learned model based on a set of interactions by the user with displayed content items. The system applies the selected machine-learned model to the set of interactions and the user attribute vector to identify a second content item and modifies the user interface to include the second content item.

Claims (81)

1. A method comprising:

detecting, by a centralized document system, a first access by a user to the centralized document system;

generating, by the centralized document system, a user attribute vector for the user, each user attribute vector entry corresponding to a characteristic of the user;

generating, by the centralized document system, a content vector associated with each of a plurality of content items, each content vector entry corresponding to a characteristic of the associated content item;

selecting, by the centralized document system, a first content item of the plurality of content items based on the user attribute vector and the content vectors by:

for each content vector of the content vectors, determining one or more matches between the user attribute vector entries of the user attribute vector and the content vector entries of the content vector;

ranking the plurality of content items based on a number of determined matches; and

selecting a top ranked content item as the first content item;

modifying, by the centralized document system, a user interface displayed to the user to include the first content item;

identifying, by the centralized document system, an interaction by the user with the first content item;

detecting, by the centralized document system, a second access by the user to the centralized document system;

selecting, by the centralized document system and from a plurality of machine-learned models, a machine-learned model based on the identified interaction taken by the user with the first content item;

applying, by the centralized document system, the selected machine-learned model to a set of interactions and the user attribute vector to identify a second content item of the plurality of content items, the set of interactions including the identified interaction; and

modifying, by the centralized document system, the user interface displayed to the user to include the second content item.

2. The method of claim 1 , wherein the plurality of machine-learned models includes one or more of: a deep and wide neural network, a deep factorization machine neural network, a graph-based neural network, and a reinforcement learning model.

3. The method of claim 1 , wherein the first access by the user to the centralized document system is a first login to an account of the user after establishing the account with the centralized document system.

4. The method of claim 1 , further comprising:

accessing a training set of historic data comprising a historic user attribute vector, historic content vectors, and historic user interactions;

training the plurality of machine-learned models using the training set of historic data to select content items to present to the user; and

re-training the plurality of machine-learned models using the set of interactions by the user with displayed content items within the centralized document system.

5. The method of claim 1 , wherein selecting the machine-learned model comprises:

determining a predictive performance of each of the plurality of machine-learned models based on the set of interactions by the user with displayed content items within the centralized document system; and

selecting a best performing machine-learned model of the plurality of machine-learned models.

6. The method of claim 1 , wherein the user had not previously accessed the centralized document system prior to the first access.

7. The method of claim 6 , further comprising:

weighting one or more content vector entries of the content vector; and

for any weighted content vector entry without a matching user attribute vector entry, removing the content vector entry from the content vector before ranking the plurality of content items.

8. Non-transitory computer readable storage media comprising computer executable instructions that, when executed, causes one or more processors of a centralized document system to:

detect a first access by a user to the centralized document system;

generate a user attribute vector for the user, each user attribute vector entry corresponding to a characteristic of the user;

generate a content vector associated with each of a plurality of content items, each content vector entry corresponding to a characteristic of the associated content item;

select a first content item of the plurality of content items based on the user attribute vector and the content vectors, wherein to select the first content item, the computer executable instructions cause the one or more processors to:

for each content vector of the content vectors, determine one or more matches between the user attribute vector entries of the user attribute vector and the content vector entries of the content vector;

rank the plurality of content items based on a number of determined matches; and

select a top ranked content item as the first content item;

modify a user interface displayed to the user to include the first content item;

identify an interaction by the user with the first content item;

detect a second access by the user to the centralized document system;

select, from a plurality of machine-learned models, a machine-learned model based on the identified interaction taken by the user with the first content item;

apply the selected machine-learned model to a set of interactions and the user attribute vector to identify a second content item of the plurality of content items, the set of interactions including the identified interaction; and

modify the user interface displayed to the user to include the second content item.

9. The non-transitory computer-readable media of claim 8 , wherein the plurality of machine-learned models includes one or more of: a deep and wide neural network, a deep factorization machine neural network, a graph-based neural network, and a reinforcement learning model.

10. The non-transitory computer-readable media of claim 8 , wherein the first access by the user to the centralized document system is a first login to an account of the user after establishing the account with the centralized document system.

11. The non-transitory computer-readable media of claim 8 , wherein the computer executable instructions further cause the one or more processors to:

access a training set of historic data comprising a historic user attribute vector, historic content vectors, and historic user interactions;

train the plurality of machine-learned models using the training set of historic data to select content items to present to the user; and

re-train the plurality of machine-learned models using the set of interactions by the user with displayed content items within the centralized document system.

12. The non-transitory computer-readable media of claim 8 , wherein to select the machine-learned model, the computer executable instructions cause the one or more processors to:

determine a predictive performance of each of the plurality of machine-learned models based on the set of interactions by the user with displayed content items within the centralized document system; and

select a best performing machine-learned model of the plurality of machine-learned models.

13. The non-transitory computer-readable media of claim 8 , wherein the user had not previously accessed the centralized document system prior to the first access.

14. The non-transitory computer-readable media of claim 13 , wherein the computer executable instructions further cause the one or more processors to:

weigh one or more content vector entries of the content vector; and

for any weighted content vector entry without a matching user attribute vector entry,

remove the content vector entry from the content vector before ranking the plurality of content items.

15. A computer system comprising:

one or more computer processors; and

non-transitory computer readable storage media comprising computer executable instructions that, when executed, causes one or more processors of a centralized document system to:

detect a first access by a user to the centralized document system;

generate a user attribute vector for the user, each user attribute vector entry corresponding to a characteristic of the user;

generate a content vector associated with each of a plurality of content items, each content vector entry corresponding to a characteristic of the associated content item;

select a first content item of the plurality of content items based on the user attribute vector and the content vectors, wherein to select the first content item, the computer executable instructions cause the one or more processors to:

for each content vector of the content vectors, determine one or more matches between the user attribute vector entries of the user attribute vector and the content vector entries of the content vector;

rank the plurality of content items based on a number of determined matches; and

select a top ranked content item as the first content item;

modify a user interface displayed to the user to include the first content item;

identify an interaction by the user with the first content item;

detect a second access by the user to the centralized document system;

select, from a plurality of machine-learned models, a machine-learned model based on the identified interaction taken by the user with the first content item;

apply the selected machine-learned model to a set of interactions and the user attribute vector to identify a second content item of the plurality of content items, the set of interactions including the identified interaction; and

modify the user interface displayed to the user to include the second content item.

16. The system of claim 15 , wherein the plurality of machine-learned models includes one or more of: a deep and wide neural network, a deep factorization machine neural network, a graph-based neural network, and a reinforcement learning model.

17. The system of claim 15 , wherein the first access by the user to the centralized document system is a first login to an account of the user after establishing the account with the centralized document system.

18. The system of claim 15 , wherein the computer executable instructions further cause the one or more processors to:

access a training set of historic data comprising a historic user attribute vector, historic content vectors, and historic user interactions;

train the plurality of machine-learned models using the training set of historic data to select content items to present to the user; and

re-train the plurality of machine-learned models using the set of interactions by the user with displayed content items within the centralized document system.

19. The system of claim 15 , wherein to select the machine-learned model, the computer executable instructions cause the one or more processors to:

determine a predictive performance of each of the plurality of machine-learned models based on the set of interactions by the user with displayed content items within the centralized document system; and

select a best performing machine-learned model of the plurality of machine-learned models.

20. The system of claim 15 , wherein the user had not previously accessed the centralized document system prior to the first access.

Assignments (2)
PATENT SECURITY AGREEMENT Recorded May 23, 2025
From: DOCUSIGN, INC.
To: BANK OF AMERICA, N.A.
Reel/Frame 071337/0240 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2022
From: KAZA, SHRINIVAS KIRAN; ALABI, TAIWO RAPHAEL; RAJABI, FARZANEH; MOHAN, ASHWATH; PHUNG, KIM CUONG
To: DOCUSIGN, INC.
Reel/Frame 062000/0588 →