IP Library Granted Patent US 10,409,859
Granted Patent B2
US 10,409,859 · App. 15/595,745 · Granted Sep 10, 2019

Video heat maps personalized for online system users

Inventors: Saurabh Gupta (Sunnyvale, CA); Aditya Palnitkar (Mountain View, CA); Yash Gupta (Mountain View, CA)
Assignee: Facebook, Inc.
G06F16/5866G06F16/437G06F16/70G06F16/907G06F16/9535G06F16/9537G06K9/00711G06Q30/02G06Q50/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,409,859
App. No.
15/595,745
Granted
Sep 10, 2019
Kind
B2
Abstract

An online system generates personalized video heat map for a target user, which visually indicates segments of a video likely to be of interest to the target user. The online system divides the video into the segments and identifies actions performed by users other than the target user on each of the segments. The online system determines embedding vectors describing each segment as represented by the identified actions performed on the segment and also determines an embedding vector describing the target user. Based on those embedding vectors, a personalized score for the segment is determined using a trained model. The online system uses the personalized score for each segment of the video to generate the personalized heat map. The personalized heat map can be provided to the target user along with the video.

Claims (54)

1. A method comprising:

determining a user vector for a target user of an online system, the user vector being an embedding vector that describes the target user in latent space;

dividing a video into a plurality of segments, the video to be presented by the online system to the target user;

identifying one or more actions performed on each of the segments the video by other users of the online system over a period of time;

determining an embedding vector of each identified action for each segment of the video;

determining a personalized score for each segment indicating a likelihood of the target user's engagement with the segment based on the embedding vector of each identified action and the user vector of the target user;

generating a personalized heat map for the target user based on the personalized score of each segment, the personalized heat map indicating visually segments of the video likely to be of interest to the target user as indicated by each segment's personalized score; and

providing, for display to a client device of the target user, the video with the personalized heat map indicating segments of interest to the target user.

2. The method of claim 1 , wherein determining the personalized score for each segment comprises:

determining a dot product of the embedding vector of each identified action and the user vector of the target user; and

applying the dot product to a model trained to determine the personalized score, the model trained based on positive label actions or negative label actions taken by the other users on each of the segments of the video, the positive label actions indicating a positive sentiment toward the segment and the negative label actions indicating a negative sentiment toward the segment.

3. The method of claim 2 , wherein the trained model is a logistic regression model.

4. The method of claim 1 , wherein determining the user vector for the target user comprises:

retrieving past interactions of the target user with other content presented by the online system to the target user, the past interactions providing information about the types of content that are of interest to the user; and

determining the user vector based on a weighted sum of the embedding vectors of the other content.

5. The method of claim 1 , wherein dividing a video into a plurality of segments comprises dividing the video into segments that each have a fixed size.

6. The method of claim 1 , wherein the identified actions on each segment of the video include at least one of the following: playing the segment of the video, liking or reacting to the video while playing the segment, sharing the video while playing the segment, commenting on the video while playing the segment, disliking the video while playing the segment, hiding the video while playing the segment, and leaving the video while playing the segment; and wherein playing, liking, sharing, or commenting are given a positive label, while disliking, hiding, and leaving are given a negative label in terms of the user's interest in that segment, the positive and negative labels used in training a machine learning model with regard to the segments.

7. The method of claim 1 , wherein the personalized heat map comprises a plurality of sections, each section associated with a segment of the video and associated with an indicator that indicates the personalized score of the segment for the target user.

8. A non-transitory computer readable medium storing executable computer program instructions, the computer program instructions comprising instructions that when executed cause a computer processor to:

determine a user vector for a target user of an online system, the user vector being an embedding vector that describes the target user in latent space;

divide a video into a plurality of segments, the video to be presented by the online system to the target user;

identify one or more actions performed on each of the segments the video by other users of the online system over a period of time;

determine an embedding vector of each identified action for each segment of the video;

determine a personalized score for each segment indicating a likelihood of the target user's engagement with the segment based on the embedding vector of each identified action and the user vector of the target user;

generate a personalized heat map for the target user based on the personalized score of each segment, the personalized heat map indicating visually segments of the video likely to be of interest to the target user as indicated by each segment's personalized score; and

provide, for display to a client device of the target user, the video with the personalized heat map indicating segments of interest to the target user.

9. The computer readable medium of claim 8 , wherein the computer program instructions for determining the personalized score for each segment comprise instructions that when executed cause the computer processor to:

determine a dot product of the embedding vector of each identified action and the user vector of the target user; and

apply the dot product to a model trained to determine the personalized score, the model trained based on positive label actions or negative label actions taken by the other users on each of the segments of the video, the positive label actions indicating a positive sentiment toward the segment and the negative label actions indicating a negative sentiment toward the segment.

10. The computer readable medium of claim 9 , wherein the trained model is a logistic regression model.

11. The computer readable medium of claim 8 , wherein the computer program instructions for determining the user vector for the target user comprise instructions that when executed cause the computer processor to:

retrieve past interactions of the target user with other content presented by the online system to the target user, the past interactions providing information about the types of content that are of interest to the user; and

determine the user vector based on a weighted sum of the embedding vectors of the other content.

12. The computer readable medium of claim 8 , wherein the computer program instructions for dividing a video into a plurality of segments comprise instructions that when executed cause the computer processor to divide the video into segments that each have a fixed size.

13. The computer readable medium of claim 8 , wherein the identified actions on each segment of the video include at least one of the following: playing the segment of the video, liking or reacting to the video while playing the segment, sharing the video while playing the segment, commenting on the video while playing the segment, disliking the video while playing the segment, hiding the video while playing the segment, and leaving the video while playing the segment; and wherein playing, liking, sharing, or commenting are given a positive label, while disliking, hiding, and leaving are given a negative label in terms of the user's interest in that segment, the positive and negative labels used in training a machine learning model with regard to the segments.

14. The computer readable medium of claim 8 , wherein the personalized heat map comprises a plurality of sections, each section associated with a segment of the video and associated with an indicator that indicates the personalized score of the segment for the target user.

15. A computer system comprising:

a non-transitory computer-readable storage medium storing executable computer program instructions, the computer program instructions comprising instructions that when executed cause a computer processor to perform steps, comprising:

determining a user vector for a target user of an online system, the user vector being an embedding vector that describes the target user in latent space;

dividing a video into a plurality of segments, the video to be presented by the online system to the target user;

identifying one or more actions performed on each of the segments the video by other users of the online system over a period of time;

determining an embedding vector of each identified action for each segment of the video;

determining a personalized score for each segment indicating a likelihood of the target user's engagement with the segment based on the embedding vector of each identified action and the user vector of the target user;

generating a personalized heat map for the target user based on the personalized score of each segment, the personalized heat map indicating visually segments of the video likely to be of interest to the target user as indicated by each segment's personalized score; and

providing, for display to a client device of the target user, the video with the personalized heat map indicating segments of interest to the target user.

16. The system of claim 15 , wherein determining the personalized score for each segment comprises:

determining a dot product of the embedding vector of each identified action and the user vector of the target user; and

applying the dot product to a model trained to determine the personalized score, the model trained based on positive label actions or negative label actions taken by the other users on each of the segments of the video, the positive label actions indicating a positive sentiment toward the segment and the negative label actions indicating a negative sentiment toward the segment.

17. The system of claim 16 , wherein the trained model is a logistic regression model.

18. The system of claim 15 , wherein determining the user vector for the target user comprises:

retrieving past interactions of the target user with other content presented by the online system to the target user, the past interactions providing information about the types of content that are of interest to the user; and

determining the user vector based on a weighted sum of the embedding vectors of the other content.

19. The system of claim 15 , wherein the identified actions on each segment of the video include at least one of the following: playing the segment of the video, liking or reacting to the video while playing the segment, sharing the video while playing the segment, commenting on the video while playing the segment, disliking the video while playing the segment, hiding the video while playing the segment, and leaving the video while playing the segment; and wherein playing, liking, sharing, or commenting are given a positive label, while disliking, hiding, and leaving are given a negative label in terms of the user's interest in that segment, the positive and negative labels used in training a machine learning model with regard to the segments.

20. The system of claim 15 , wherein the personalized heat map comprises a plurality of sections, each section associated with a segment of the video and associated with an indicator that indicates the personalized score of the segment for the target user.

Assignments (2)
CHANGE OF NAME Recorded Nov 18, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058897/0824 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 30, 2017
From: GUPTA, SAURABH; PALNITKAR, ADITYA; GUPTA, YASH
To: FACEBOOK, INC.
Reel/Frame 042532/0073 →
Continuity (1)
Related Publication 20180329928A1 · Nov 15, 2018
Cited By (1)
US 12,671,667