Real-time online learning for short-form content ranking

1 . A computer-implemented method for real-time online learning for short-form content ranking, the computer-implemented method comprising:

receiving, by at least one computer processor, a user feedback event that includes user feedback on a short-form content item played using a media device;

publishing a message based on the user feedback event to a publish-subscribe (pub/sub) topic via a pub/sub stream, the pub/sub topic comprising queues organized as partitions each held by a broker server of a pub/sub distributed messaging system, each broker server holding one or more partitions;

retrieving the message via a pull operation;

saving the user feedback event from the retrieved message to a real-time data store;

triggering a short-form content recommendation engine to generate a playlist batch of short-form content recommendations based on the user feedback event and on other user feedback events retrieved from the real-time data store by the short-form content recommendation engine; and

transmitting short-form content items specified in the playlist batch to the media device for playback on a short-form content presentation user interface of the media device,

wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within five seconds.

2 . The computer-implemented method of claim 1 , wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within three seconds.

3 . The computer-implemented method of claim 1 , wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within one second.

4 . The computer-implemented method of claim 1 , wherein the user feedback event is received from the media device as a Hypertext Transfer Protocol (HTTP) request.

5 . The computer-implemented method of claim 4 , wherein the message is published to the pub/sub topic by an event relay service without any processing of the HTTP request prior to placing the message into the pub/sub stream, and without any persistence of the HTTP request within the event relay service.

6 . The computer-implemented method of claim 1 , wherein the generating the playlist batch of short-form content recommendations by the short-form content recommendation engine comprises:

retrieving a short-form content events user history and a main-asset events user history from the real-time data store, the short-form content events user history comprising the user feedback event;

detecting, based on the user feedback event, whether the user feedback on the short-form content item indicates like or dislike of the short-form content item; and

identifying user-relevant short-form content items by ranking machine-learning model item embeddings using an approximate nearest neighbor algorithm based on the detected like or dislike of the short-form content item, the main-asset events user history, and a seed identifying an initially selected short-form content item that is selected by a user from a short-form content selection user interface, the machine-learning model item embeddings each representative of a respective content item, the machine-learning model item embeddings generated by inferencing of an offline sequence model,

wherein the playlist batch comprises identifiers of the user-relevant short-form content items, identifiers of short-form content items determined to be popular among a plurality of users, and identifiers of randomly-selected short-form content items.

7 . The computer-implemented method of claim 1 , wherein the generating the playlist batch of short-form content recommendations by the short-form content recommendation engine comprises:

retrieving a short-form content events user history from the real-time data store, the short-form content events user history comprising the user feedback event;

determining long-form content items associated with short-form content items specified in the short-form content events user history, each of the long-form content items having a longer playback duration than respective ones of the short-form content items specified in the short-form content events user history;

retrieving a long-form content events user history from the real-time data store based on the long-form content items;

identifying user-relevant long-form content items by ranking candidate long-form content items using an online sequence model by providing as inferencing inputs to the online sequence model the long-form content events user history and a seed identifying an initially selected short-form content item that is selected by a user from a short-form content selection user interface; and

determining user-relevant short-form content items associated with the user-relevant long-form content items,

wherein the playlist batch comprises identifiers of the user-relevant short-form content items.

8 . A system, comprising:

one or more memories; and

at least one processor each coupled to at least one of the memories and configured to perform operations comprising:

receiving a user feedback event that includes user feedback on a short-form content item played using a media device;

publishing a message based on the user feedback event to a publish-subscribe (pub/sub) topic via a pub/sub stream, the pub/sub topic comprising queues organized as partitions each held by a broker server of a pub/sub distributed messaging system, each broker server holding one or more partitions;

retrieving the message via a pull operation;

saving the user feedback event from the retrieved message to a real-time data store;

triggering a short-form content recommendation engine to generate a playlist batch of short-form content recommendations based on the user feedback event and on other user feedback events retrieved from the real-time data store by the short-form content recommendation engine; and

transmitting short-form content items specified in the playlist batch to the media device for playback on a short-form content presentation user interface of the media device,

wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within five seconds.

9 . The system of claim 8 , wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within three seconds.

10 . The system of claim 8 , wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within one second.

11 . The system of claim 8 , wherein the user feedback event is received from the media device as a Hypertext Transfer Protocol (HTTP) request.

12 . The system of claim 11 , wherein the message is published to the pub/sub topic by an event relay service without any processing of the HTTP request prior to placing the message into the pub/sub stream, and without any persistence of the HTTP request within the event relay service.

13 . The system of claim 8 , wherein the generating the playlist batch of short-form content recommendations by the short-form content recommendation engine comprises:

retrieving a short-form content events user history and a main-asset events user history from the real-time data store, the short-form content events user history comprising the user feedback event;

detecting, based on the user feedback event, whether the user feedback on the short-form content item indicates like or dislike of the short-form content item; and

identifying user-relevant short-form content items by ranking machine-learning model item embeddings using an approximate nearest neighbor algorithm based on the detected like or dislike of the short-form content item, the main-asset events user history, and a seed identifying an initially selected short-form content item that is selected by a user from a short-form content selection user interface, the machine-learning model item embeddings each representative of a respective content item, the machine-learning model item embeddings generated by inferencing of an offline sequence model,

wherein the playlist batch comprises identifiers of the user-relevant short-form content items, identifiers of short-form content items determined to be popular among a plurality of users, and identifiers of randomly-selected short-form content items.

14 . The system of claim 8 , wherein the generating the playlist batch of short-form content recommendations by the short-form content recommendation engine comprises:

retrieving a short-form content events user history from the real-time data store, the short-form content events user history comprising the user feedback event;

determining long-form content items associated with short-form content items specified in the short-form content events user history, each of the long-form content items having a longer playback duration than respective ones of the short-form content items specified in the short-form content events user history;

retrieving a long-form content events user history from the real-time data store based on the long-form content items;

identifying user-relevant long-form content items by ranking candidate long-form content items using an online sequence model by providing as inferencing inputs to the online sequence model the long-form content events user history and a seed identifying an initially selected short-form content item that is selected by a user from a short-form content selection user interface; and

determining user-relevant short-form content items associated with the user-relevant long-form content items,

wherein the playlist batch comprises identifiers of the user-relevant short-form content items.

15 . One or more non-transitory computer-readable media having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

receiving a user feedback event that includes user feedback on a short-form content item played using a media device;

publishing a message based on the user feedback event to a publish-subscribe (pub/sub) topic via a pub/sub stream, the pub/sub topic comprising queues organized as partitions each held by a broker server of a pub/sub distributed messaging system, each broker server holding one or more partitions;

retrieving the message via a pull operation;

saving the user feedback event from the retrieved message to a real-time data store;

triggering a short-form content recommendation engine to generate a playlist batch of short-form content recommendations based on the user feedback event and on other user feedback events retrieved from the real-time data store by the short-form content recommendation engine; and

transmitting short-form content items specified in the playlist batch to the media device for playback on a short-form content presentation user interface of the media device,

wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within five seconds.

16 . The one or more non-transitory computer-readable media of claim 15 , wherein the receiving, the publishing, the retrieving, the triggering, and the transmitting happen within three seconds.

17 . The one or more non-transitory computer-readable media of claim 15 , wherein the user feedback event is received from the media device as a Hypertext Transfer Protocol (HTTP) request.

18 . The one or more non-transitory computer-readable media of claim 17 , wherein the message is published to the pub/sub topic by an event relay service without any processing of the HTTP request prior to placing the message into the pub/sub stream, and without any persistence of the HTTP request within the event relay service.

19 . The one or more non-transitory computer-readable media of claim 15 , wherein the generating the playlist batch of short-form content recommendations by the short-form content recommendation engine comprises:

retrieving a short-form content events user history and a main-asset events user history from the real-time data store, the short-form content events user history comprising the user feedback event;

detecting, based on the user feedback event, whether the user feedback on the short-form content item indicates like or dislike of the short-form content item; and

identifying user-relevant short-form content items by ranking machine-learning model item embeddings using an approximate nearest neighbor algorithm based on the detected like or dislike of the short-form content item, the main-asset events user history, and a seed identifying an initially selected short-form content item that is selected by a user from a short-form content selection user interface, the machine-learning model item embeddings each representative of a respective content item, the machine-learning model item embeddings generated by inferencing of an offline sequence model,

wherein the playlist batch comprises identifiers of the user-relevant short-form content items, identifiers of short-form content items determined to be popular among a plurality of users, and identifiers of randomly-selected short-form content items.

20 . The one or more non-transitory computer-readable media of claim 15 , wherein the generating the playlist batch of short-form content recommendations by the short-form content recommendation engine comprises:

retrieving a short-form content events user history from the real-time data store, the short-form content events user history comprising the user feedback event;

determining long-form content items associated with short-form content items specified in the short-form content events user history, each of the long-form content items having a longer playback duration than respective ones of the short-form content items specified in the short-form content events user history;

retrieving a long-form content events user history from the real-time data store based on the long-form content items;

identifying user-relevant long-form content items by ranking candidate long-form content items using an online sequence model by providing as inferencing inputs to the online sequence model the long-form content events user history and a seed identifying an initially selected short-form content item that is selected by a user from a short-form content selection user interface; and

determining user-relevant short-form content items associated with the user-relevant long-form content items,

wherein the playlist batch comprises identifiers of the user-relevant short-form content items.