IP Library Granted Patent US 11,102,425
Granted Patent B2
US 11,102,425 · App. 16/482,877 · Granted Aug 24, 2021

Image processing apparatus and image processing method

Inventor: Shogo Takanashi (Tokyo, JP)
Assignee: SONY CORPORATION
H04N5/272G06K9/00228G06T5/50G06T11/001H04N7/0117
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,102,425
App. No.
16/482,877
Filed
Aug 1, 2019
Granted
Aug 24, 2021
Kind
B2
Art Unit
2699
USPC
348/239
Abstract

Visibility of a description on a description field hidden by a presenter is to be ensured while maintaining a positional relationship between the presenter and the description on the description field. The moving image data obtained by imaging a state where the presenter is giving the description onto the description field is processed to determine the description portion. Display data for displaying each of portions determined to be the description portion as a description is generated and superimposed on moving image data. For example, a difference value for each of pixels between a current frame image and a reference frame image is extracted, a group including a series of consecutive pixels having a difference value being a threshold or more is grasped and then, whether or not the group is a description portion is determined for each of the groups.

Claims (53)

1. An image processing apparatus comprising:

circuitry configured to

obtain moving image data of a presenter that presents a description onto a description field,

determine a group of consecutive pixels in a current frame image of the moving image data based on difference values between pixels in the current frame image and pixels in a reference frame image,

determine whether the group of consecutive pixels in the current frame image corresponds to a portion of the description,

generate display data for displaying the portion of the description corresponding to the group of consecutive pixels when the group of consecutive pixels is determined to correspond to the description, and

superimpose the display data over images of the moving image data.

2. The image processing apparatus according to claim 1 , wherein the circuitry is configured to

extract a difference value for each of the pixels between the current frame image and the reference frame image, determine the group of consecutive pixels having the difference values being a threshold or more, and determine whether or not the group of consecutive pixels corresponds to the portion of the description.

3. The image processing apparatus according to claim 2 , wherein the circuitry is configured to

determine whether or not the group corresponds to the portion of the description using a shape evaluation value, an extraction time evaluation value, and a stationary time evaluation value,

obtain the shape evaluation value on a basis of an aspect ratio relationship of a smallest quadrilateral that includes the group and a largest quadrilateral included within the group,

obtain the extraction time evaluation value on a basis of a difference value between the current frame image and the reference frame image among a plurality of past frames, for pixels included in the group, and

obtain the stationary time evaluation value on a basis of a difference value between the current frame image and a preceding frame image among the plurality of past frames, for pixels included in the group.

4. The image processing apparatus according to claim 1 , wherein the circuitry is configured to

perform processing for enhancing visibility of the description when superimposing the display data over images of the moving image data.

5. The image processing apparatus according to claim 4 , wherein the circuitry is configured to

first perform processing of reducing resolution of the images of the moving image data and thereafter superimposes the display data over the images of the moving image data.

6. The image processing apparatus according to claim 4 , wherein the circuitry is configured to

first process the display data so that the portion determined to correspond to the description is bordered and displayed, and thereafter superimpose the display data over the images of the moving image data.

7. The image processing apparatus according to claim 4 , wherein the circuitry is configured to

first process the display data on a basis of color information of the moving image data so that color of the portion determined to correspond to the description is differentiated from color of a background image, and thereafter superimpose the display data over the images of the moving image data.

8. The image processing apparatus according to claim 1 , wherein the circuitry is configured to

not superimpose the display data over the images of the moving image data when a state where a face of the presenter is facing front is detected on a basis of the moving image data.

9. The image processing apparatus according to claim 1 , further comprising:

an imaging sensor configured to image a state where the presenter is presenting a description onto the description field, wherein

the circuitry is configured to process an imaging signal obtained by the imaging sensor and obtain the moving image data.

10. The image processing apparatus according to claim 1 , wherein the circuitry is configured to

transmit the moving image data to an external device.

11. An image processing method comprising:

obtaining moving image data of a presenter that presents a description onto a description field;

determining a group of consecutive pixels in a current frame image of the moving image data based on difference values between pixels in the current frame image and pixels in a reference frame image;

determining, by processing circuitry, whether the group of consecutive pixels in the current frame image corresponds to a portion of the description;

generating display data for displaying the portion of the description corresponding to the group of consecutive pixels when the group of consecutive pixels is determined to correspond to the description; and

superimposing the display data over images of the moving image data.

12. The image processing method according to claim 11 , wherein

the determining the group of consecutive pixels includes

extracting a difference value for each of the pixels between the current frame image and the reference frame image, and

determining the group of consecutive pixels having the difference values being a threshold or more; and

the determining whether the group of consecutive pixels in the current frame image corresponds to the portion of the description includes determining whether or not the group of consecutive pixels corresponds to the portion of the description.

13. The image processing method according to claim 12 , wherein

the determining whether or not the group of consecutive pixels corresponds to the portion of the description includes determining whether or not the group corresponds to the portion of the description using a shape evaluation value, an extraction time evaluation value, and a stationary time evaluation value,

the shape evaluation value is based on an aspect ratio relationship of a smallest quadrilateral that includes the group and a largest quadrilateral included within the group,

the extraction time evaluation value is based on a difference value between the current frame image and the reference frame image among a plurality of past frames, for pixels included in the group, and

the stationary time evaluation value is based on a difference value between the current frame image and a preceding frame image among the plurality of past frames, for pixels included in the group.

14. The image processing method according to claim 11 , wherein

the display data is not superimposed over the images of the moving image data when a state where a face of the presenter is facing front is detected on a basis of the moving image data.

15. A non-transitory computer-readable storage medium storing instructions which when executed by a computer cause the computer to perform:

obtaining moving image data of a presenter that presents a description onto a description field;

determining a group of consecutive pixels in a current frame image of the moving image data based on difference values between pixels in the current frame image and pixels in a reference frame image;

determining whether the group of consecutive pixels in the current frame image corresponds to a portion of the description;

generating display data for displaying the portion of the description corresponding to the group of consecutive pixels when the group of consecutive pixels is determined to correspond to the description; and

superimposing the display data over images of the moving image data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 1, 2019
From: TAKANASHI, SHOGO
To: SONY CORPORATION
Reel/Frame 049934/0791 →
Priority Claims (1)
JP JP2017-043471 · Mar 8, 2017 · national
Continuity (1)
Related Publication 20190364225A1 · Nov 28, 2019
Cited By (1)
US 12,340,710