IP Library Granted Patent US 11,567,812
Granted Patent B2
US 11,567,812 · App. 17/065,266 · Granted Jan 31, 2023

Utilizing a natural language model to determine a predicted activity event based on a series of sequential tokens

Inventors: Ranjitha Gurunath Kulkarni (Sunnyvale, CA); Xingyu Xiang (San Francisco, CA); Jongmin Baek (Foster City, CA); Ermo Wei (San Bruno, CA)
Assignee: Dropbox, Inc.
G06F9/542G06F40/284G06N3/08G06N5/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,567,812
App. No.
17/065,266
Granted
Jan 31, 2023
Kind
B2
Abstract

The present disclosure relates to systems, methods, and non-transitory computer-readable media that can leverage a natural language model to determine a most probable candidate sequence of tokens and thereby generate a predicted user activity. In particular, the disclosed systems can tokenize activity event vectors to generate a series of sequential tokens that correspond to recent user activity of one or more user accounts. In addition, the disclosed systems can, for each candidate (e.g., hypothetical) user activity, augment the series of sequential tokens to include a corresponding token. Based on respective probability scores for each of the augmented series of sequential tokens, the disclosed systems can identify as the predicted user activity, a candidate user activity corresponding to one of the augmented series of sequential tokens associated with a highest probability score. Based on the predicted user activity, the disclosed systems can surface one or more suggestions to a client device.

Claims (61)

1. A system comprising:

at least one processor; and

at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:

identify, from an events database, a sequence of activity events, each activity event within the sequence of activity events comprising raw event data associated with a user account of a content management system;

generate a sequence of activity event vectors by generating an activity event vector for each activity event based on the raw event data for each respective activity event;

determine, utilizing a natural language processing (NLP) model, a predicted activity event based on the sequence of activity event vectors; and

perform an action based on the predicted activity event.

2. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to perform the action by providing, for display on a client device associated with the user account, one or more suggestions based on the predicted activity event.

3. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to generate the sequence of activity event vectors by:

generating feature embeddings for features from the raw event data of each activity event within the sequence of activity events;

concatenating a first set of feature embeddings corresponding to a first activity event to generate a first activity event vector; and

concatenating a second set of feature embeddings corresponding to a second activity event subsequent to the first activity event to generate a second activity event vector.

4. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:

generate, utilizing a hashing function, respective tokens for each activity event vector in the sequence of activity event vectors; and

generate, based on the respective tokens, a series of sequential tokens for the sequence of activity event vectors.

5. The system of claim 4 , further comprising instructions that, when executed by the at least one processor, cause the system to generate, utilizing the NLP model, the predicted activity event by:

determining, for a set of candidate sequences combining the series of sequential tokens with respective tokens of candidate activity events, a set of probability scores; and

selecting, as the predicted activity event, a candidate activity event corresponding to a candidate sequence associated with a highest probability score.

6. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to group the sequence of activity event vectors for generating a series of sequential tokens by grouping, into the sequence of activity event vectors, activity event vectors based on timestamps of corresponding feature embeddings.

7. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:

generate, from the events database, a corpus of series of sequential tokens associated with multiple user accounts of a content management system; and

apply the corpus of series of sequential tokens to the NLP model to identify one or more learned parameters.

8. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:

determine a set of predicted activity events based on the sequence of activity event vectors; and

perform multiple actions based on the set of predicted activity events, each action corresponding to a respective predicted activity event of the set of predicted activity events.

9. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to perform the action based on the predicted activity event by predicting a user segment for the user account.

10. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to identify, from the events database, the sequence of activity events by:

analyzing, from the events database, raw event data to identify timestamps for activity events for a threshold time period; and

determining, based on the identified timestamps, that a subset of the activity events occurring during the threshold time period corresponds to the sequence of activity events.

11. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:

identify, from an events database, a sequence of activity events, each activity event within the sequence of activity events comprising raw event data associated with a user account of a content management system;

generate a sequence of activity event vectors by generating an activity event vector for each activity event based on the raw event data for each respective activity event;

provide the sequence of activity event vectors to a natural language processing (NLP) model;

receive, from the NLP model, one or more predicted activity events; and

provide, to a client device associated with the user account, one or more suggestions based on the one or more predicted activity events.

12. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to provide, for display on the client device associated with the user account, the one or more suggestions comprising:

a suggested team workspace accessible by the user account and one or more other user accounts of the content management system;

a suggested digital file to digitally share with the one or more other user accounts; or

a suggested access privilege or a suggested software tool based on a user-account classification for the user account.

13. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate a sequence of activity event vectors by generating feature embeddings based on features from the raw event data of each activity event within the sequence of activity events.

14. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to provide, for display on the client device associated with the user account, the one or more suggestions comprising an action suggestion that includes one or more of:

downloading, accessing, or editing a digital content item;

creating a calendar invite;

generating a digital reminder; or

generating a task list.

15. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to provide the sequence of activity event vectors to the NLP model by generating a token for each activity event vector in the sequence of activity event vectors.

16. A computer-implemented method comprising:

identifying, from an events database, a sequence of activity events, each activity event within the sequence of activity events comprising raw event data associated with a group of user accounts of a content management system;

generating a sequence of activity event vectors by generating an activity event vector for each activity event based on the raw event data for each respective activity event;

generating, utilizing a natural language processing (NLP) model, a predicted activity event based on the sequence of activity event vectors; and

performing an action based on the predicted activity event.

17. The computer-implemented method of claim 16 , wherein the NLP model comprises a long short-term memory neural network.

18. The computer-implemented method of claim 16 , further comprising generating one or more activity highlights pertaining to digital activity in a team workspace accessible by the group of user accounts.

19. The computer-implemented method of claim 16 , wherein generating the predicted activity event comprises:

generating a first predicted activity event for a first user account of the group of user accounts; and

generating a second predicted activity event for a second user account of the group of user accounts, wherein the second predicted activity event differs from the first predicted activity event.

20. The computer-implemented method of claim 19 , wherein performing the action based on the predicted activity event comprises:

providing, for display on a first client device associated with the first user account, a first action suggestion related to a digital content item accessible by the group of user accounts, wherein the first action suggestion is based on the first predicted activity event; and

providing, for display on a second client device associated with the second user account, a second action suggestion related to the digital content item accessible by the group of user accounts, wherein:

the second action suggestion differs from the first action suggestion; and

the second action suggestion is based on the second predicted activity event.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Dec 13, 2024
From: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
To: DROPBOX, INC.
Reel/Frame 069635/0332 →
SECURITY INTEREST Recorded Dec 12, 2024
From: DROPBOX, INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 069604/0611 →
PATENT SECURITY AGREEMENT Recorded Mar 10, 2021
From: DROPBOX, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 055670/0219 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 19, 2021
From: KULKARNI, RANJITHA GURUNATH; XIANG, XINGYU; BAEK, JONGMIN; WEI, ERMO
To: DROPBOX, INC.
Reel/Frame 054951/0139 →