IP Library Granted Patent US 12,120,378
Granted Patent B2
US 12,120,378 · App. 18/008,614 · Granted Oct 15, 2024

Video distribution in which provided information is superimposed on video

Inventor: Ko Kikuchi (Tokyo, JP)
Assignee: Rakuten Group, Inc.
H04N21/4316G06T5/50G06T7/50G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,120,378
App. No.
18/008,614
Granted
Oct 15, 2024
Kind
B2
Abstract

In an information processing device to streaming-distribute a video, based on a distance image in which a streamer is captured, an acquirer ( 101 ) acquires a depth distribution in the distance image. A determiner ( 102 ) determines a threshold value, based on the acquired depth distribution. An specifier ( 103 ) specifies, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value. A provider ( 104 ) provides, in conjunction with the video, to a viewer terminal ( 300 ) to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed.

Claims (30)

1. An information processing device to streaming-distribute a video, based on a distance image in which a streamer is captured, comprising one or more processors,

wherein the processor executes processing of:

acquiring a depth distribution in the distance image;

determining a threshold value, based on the acquired depth distribution;

specifying, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value; and

providing, in conjunction with the video, to a viewer terminal to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed.

2. The information processing device according to claim 1 , wherein

the provided information is information provided by a viewer of the viewer terminal, and

the processor executes processing of providing control information to cause the provided information to be displayed in such a way that the provided information, starting from outside of a screen of the viewer terminal, keeps moving toward an inner part of the screen and disappears out of the screen.

3. The information processing device according to claim 1 , wherein the processor executes processing of:

compensating the near region so as to be a region of a predetermined shape encompassing the near region;

compensating the distant region so as to be a region excluding the region of the predetermined shape; and

setting the compensated distant region as the background and the compensated near region as the foreground.

4. The information processing device according to claim 1 , wherein

the acquired depth distribution is a mixed distribution, and

the processor executes processing of:

separating the acquired depth distribution into a first distribution and a second distribution; and

when a first representative value of the first distribution and a second representative value of the second distribution are sufficiently separated from each other, determining a middle value between the first representative value and the second representative value as the threshold value.

5. The information processing device according to claim 4 , wherein the processor executes processing of, when a difference between the first representative value and the second representative value is greater than a value obtained by multiplying a sum of a standard deviation of the first distribution and a standard deviation of the second distribution by a predetermined value, determining that the first representative value of the first distribution and the second representative value of the second distribution are sufficiently separated from each other.

6. The information processing device according to claim 1 , wherein the processor executes processing of providing a streamer terminal with which the streamer captures a video with control information to cause the streamer terminal to display a video or a distance image that is currently being captured and display the distant region and the near region in different display forms in the distance image.

7. A display control method performed by a computer for streaming-distributing a video, based on a distance image in which a streamer is captured, the display control method comprising:

acquiring a depth distribution in the distance image;

determining a threshold value, based on the acquired depth distribution;

specifying, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value; and

providing, in conjunction with the video, to a viewer terminal to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed.

8. A computer-readable recording medium recording a program causing a computer for streaming-distributing a video, based on a distance image in which a streamer is captured to execute processing of:

acquiring a depth distribution in the distance image;

determining a threshold value, based on the acquired depth distribution;

specifying, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value; and

providing, in conjunction with the video, to a viewer terminal to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2024
From: RAKUTEN MOBILE, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 067685/0007 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2022
From: KIKUCHI, KO
To: RAKUTEN MOBILE, INC.
Reel/Frame 061999/0807 →
Continuity (1)
Related Publication 20240244288A1 · Jul 18, 2024