IP Library › Granted Patent US 12684191
Granted Patent B2
US 12684191 · App. 17/963,456 · Granted Jul 14, 2026

Displaying multiple multimedia segments in a display device

Inventors: Dengzhi Zhang (San Jose, CA); Hugo Du Plooy (San Jose, CA)
Assignee: ROKU, INC.
H04N21/4316H04N5/45H04N21/44029H04N21/812
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12684191
App. No.
17/963,456
Granted
Jul 14, 2026
Kind
B2
Abstract

Disclosed herein are system, apparatus, method and/or computer program product embodiments for a multimedia environment that includes a computing device to display multiple multimedia segments within multiple display areas of a display device. The computing device may determine a first multimedia segment including a plurality of frames, and display the first multimedia segment within a first display area of the display device. The computing device may further determine, based on a user profile or content of the first multimedia segment, a second multimedia segment related to the first multimedia segment to be displayed within a second display area of the display device within the predetermined time interval.

Claims (54)

1 . A method, comprising:

determining, by at least one processor, a first video aspect ratio of a first multimedia segment;

generating, based on the first video aspect ratio of the first multimedia segment, a first display area of a wide screen display device in which the first multimedia segment is to be displayed in the first display area within a predetermined time interval;

generating, based on the first video aspect ratio of the first multimedia segment, a second display area of the wide screen display device that is adjacent to a first side of the first display area and a third display area of the wide screen display device that is adjacent to a second side of the first display area that is opposite to the first side, wherein a first size of the first display area, a second size of the second display area, and a third size of the third display area are based on the first video aspect ratio of the first multimedia segment;

determining, based on a user profile or content of the first multimedia segment, a second multimedia segment related to the first multimedia segment to be displayed within the second display area within the predetermined time interval and a third multimedia segment related to the first multimedia segment to be displayed within the third display area within the predetermined time interval;

displaying the second multimedia segment within the second display area within the predetermined time interval when the first multimedia segment is displayed within the first display area;

displaying the third multimedia segment within the third display area within the predetermined time interval when the first multimedia segment is displayed within the first display area;

determining a second video aspect ratio of a fourth multimedia segment, wherein the second video aspect ratio is different from the first video aspect ratio;

re-generating, based on the second video aspect ratio of the fourth multimedia segment, the first display area, the second display area, and the third display area; and

displaying the fourth multimedia segment in the re-generated first display area.

2 . The method of claim 1 , wherein the second display area is disjoint from the first display area.

3 . The method of claim 2 , wherein the first display area and the second display area are of a rectangular shape of a substantially same height.

4 . The method of claim 1 , wherein the second display area overlays with the first display area, and the second multimedia segment is an on-screen display (OSD) of the first multimedia segment.

5 . The method of claim 1 , wherein the second multimedia segment includes an advertisement related to the first multimedia segment or the user profile of the first multimedia segment.

6 . The method of claim 5 , wherein the user profile is identified based on an identification of the wide screen display device.

7 . The method of claim 1 , wherein the content of the first multimedia segment includes audio content, and the second multimedia segment is related to the audio content of the first multimedia segment.

8 . The method of claim 1 , wherein the content of the first multimedia segment includes at least one object included in a frame of the first multimedia segment, and the second multimedia segment is related to the at least one object included in the frame of the first multimedia segment.

9 . The method of claim 8 , wherein the at least one object included in the frame of the first multimedia segment is identified by performing image recognition analysis of the frame.

10 . The method of claim 1 , wherein the content of the first multimedia segment includes automatic content recognition (ACR) content recognized by an ACR server, and the second multimedia segment is related to the ACR content of the first multimedia segment.

11 . A computing device, comprising:

one or more memories; and

at least one processor, each coupled to at least one of the memories and configured to:

determine a first video aspect ratio of a first multimedia segment;

generate, based on the first video aspect ratio of the first multimedia segment, a first display area of a wide screen display device in which the first multimedia segment is to be displayed in the first display area within a predetermined time interval

generate, based on the first video aspect ratio of the first multimedia segment, a second display area of the wide screen display device that is adjacent to a first side of the first display area and a third display area of the wide screen display device that is adjacent to a second side of the first display area that is opposite to the first side, wherein a first size of the first display area, a second size of the second display area, and a third size of the third display area are based on the determined first video aspect ratio of the first multimedia segment;

determine, based on a user profile or content of the first multimedia segment, a second multimedia segment related to the first multimedia segment to be displayed within the second display area within the predetermined time interval and a third multimedia segment related to the first multimedia segment to be displayed within the third display area within the predetermined time interval;

display the second multimedia segment within the second display area within the predetermined time interval when the first multimedia segment is displayed within the first display area;

display the third multimedia segment within the third display area within the predetermined time interval when the first multimedia segment is displayed within the first display area;

determine a second video aspect ratio of a fourth multimedia segment, wherein the second video aspect ratio is different from the first video aspect ratio;

re-generate, based on the second video aspect ratio of the fourth multimedia segment, the first display area, the second display area, and the third display area; and

display the fourth multimedia segment in the re-generated first display area.

12 . The computing device of claim 11 , wherein the second display area is disjoint from the first display area.

13 . The computing device of claim 12 , wherein the first display area and the second display area are of a rectangular shape of a substantially same height.

14 . The computing device of claim 11 , wherein the second multimedia segment includes an advertisement related to the first multimedia segment or the user profile of the first multimedia segment.

15 . The computing device of claim 14 , wherein the user profile is identified based on an identification of the wide screen display device.

16 . The computing device of claim 11 , wherein the content of the first multimedia segment includes at least one object included in a frame of the first multimedia segment, and the second multimedia segment is related to the at least one object included in the frame of the first multimedia segment.

17 . A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

determining a first video aspect ratio of a first multimedia segment;

generating, based on the first video aspect ratio of the first multimedia segment, a first display area of a wide screen display device in which the first multimedia segment is to be displayed in the first display area within a predetermined time interval;

generating, based on the first video aspect ratio of the first multimedia segment, a second display area of the wide screen display device that is adjacent to a first side of the first display area and a third display area of the wide screen display device that is adjacent to a second side of the first display area that is opposite to the first side, wherein a first size of the first display area, a second size of the second display area, and a third size of the third display area are based on the first video aspect ratio of the first multimedia segment;

determining, based on a user profile or content of the first multimedia segment, a second multimedia segment related to the first multimedia segment to be displayed within the second display area within the predetermined time interval and a third multimedia segment related to the first multimedia segment to be displayed within the third display area within the predetermined time interval;

displaying the second multimedia segment within the second display area within the predetermined time interval when the first multimedia segment is displayed within the first display area;

displaying the third multimedia segment within the third display area within the predetermined time interval when the first multimedia segment is displayed within the first display area;

determining a second video aspect ratio of a fourth multimedia segment, wherein the second video aspect ratio is different from the first video aspect ratio;

re-generating, based on the second video aspect ratio of the fourth multimedia segment, the first display area, the second display area, and the third display area; and

displaying the fourth multimedia segment in the re-generated first display area.

18 . The non-transitory computer-readable medium of claim 17 , wherein the second display area is disjoint from the first display area; and

the first display area and the second display area are of a rectangular shape of a substantially same height.

19 . The method of claim 1 , further comprising:

determining an issue exists with audio content associated with the first multimedia segment based on information associated with a plurality of media devices different from the wide screen display device and that played back the first multimedia segment; and

automatically activating closed captioning based on determining the issue exists with the audio content.

20 . The method of claim 1 , further comprising:

determining closed captioning obstructs a portion of video content associated with the first multimedia segment based on information associated with a plurality of media devices different from the wide screen display device and that played back the first multimedia segment; and

automatically deactivating the closed captioning based on determining the closed captioning obstructs a portion of the video content.