IP Library Granted Patent US 9,460,337
Granted Patent B2
US 9,460,337 · App. 14/437,605 · Granted Oct 4, 2016

Information processor and information processing method

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,460,337
App. No.
14/437,605
Granted
Oct 4, 2016
Kind
B2
Abstract

An image storage section 48 stores shot image data with a plurality of resolutions transmitted from an imaging device. Depth images 152 with a plurality of resolutions are generated using stereo images with a plurality of resolution levels from the shot image data (S 10 ). Next, template matching is performed using a reference template image 154 that represents a desired shape and size, thus extracting a candidate area for a target picture having the shape and size for each distance range associated with one of the resolutions (S 12 ). A more detailed analysis is performed on the extracted candidate areas using the shot image stored in the image storage section 48 (S 14 ). In some cases, a further image analysis is performed based on the analysis result using a shot image with a higher resolution level (S 16 a and S 16 b ).

Claims (33)

1. An information processor comprising:

an image device arranged to capture a shot image and to transform the shot image into a plurality of depth images having different resolutions;

a candidate area extraction section arranged to extract candidate areas in which a target picture is likely to exist in the shot image by performing template matching on each of the plurality of depth images using a template image, the depth images representing, on an image plane, a distance of a subject from a camera in a depth direction as a pixel value, the template image representing a shape and a size of the target to be analyzed,

wherein, for each of the plurality of depth images, the template image is varied in size during template matching only within a predefined magnification range;

a detailed information acquisition section arranged to analyze the candidate areas extracted by the candidate area extraction section; and

an output information generation section arranged to generate output information based on results of image analysis performed by the detailed information acquisition section and output the output information.

2. The information processor of claim 1 , wherein

the detailed information acquisition section determines, as a target picture area and in each frame of a movie, i.e., the shot image, an area of the associated depth image, the area inside the candidate area extracted by the candidate area extraction section, the area being made up of pixels whose values fall within a given range, so as to estimate the target picture area at the next frame for tracking.

3. The information processor of claim 1 , wherein

the detailed information acquisition section performs tracking of the target in the movie, i.e., the shot image, and corrects tracking results obtained for each frame of the movie based on the candidate area for the target extracted by the candidate area extraction section from the associated depth image.

4. The information processor of claim 1 , wherein

the candidate area extraction section extracts, from each of the plurality of depth images, the candidate area for the picture existing in a distance range in the depth direction associated with one of the resolutions, and

the detailed information acquisition section performs image analysis on the shot image with the resolution associated with that of the depth image from which the candidate area has been extracted.

5. The information processor of claim 1 , wherein

the image analysis is facial recognition analysis, and

the resolution of the depth image from which the candidate area is extracted by the candidate area extraction section is smaller than that of the shot image on which image analysis is performed by the detailed information acquisition section using information relating to the candidate area.

6. The information processor of claim 1 , wherein

the detailed information acquisition section requests the camera to adjust an exposure value in such a manner that if a luminance of the candidate area extracted by the candidate area extraction section in the shot image falls outside a proper range, the luminance falls within the proper range.

7. The information processor of claim 1 , wherein

the candidate area extraction section extracts the candidate area for human head, i.e., the target, and

the detailed information acquisition section detects a facial area by performing a face detection process on the extracted candidate area in the shot image and identifies the person of the picture by performing a face recognition process on the detected facial area using a shot image with a higher resolution than that of the shot image used for the face detection process.

8. An information processing method used by an information processor to process information using a shot image from a camera, the information processing method comprising:

transforming the shot image into a plurality of depth images having differing resolutions,

extracting candidate areas in which a target picture is likely to exist in the shot image by performing template matching on each of the plurality of depth images using a template image, the depth images representing, on an image plane, a distance of a subject from the camera in a depth direction as a pixel value, the template image being read from a memory and representing a shape and a size of the target to be analyzed;

wherein, for each of the plurality of depth images, the template image is varied in size during template matching only within a predefined magnification range;

analyzing the extracted candidate areas; and

generating an output image reflecting the image analysis results and outputting the output image to a display device.

9. A computer program for a computer stored on a non-transitory computer readable medium for causing a computer to execute:

transforming a shot image into a plurality of depth images having differing resolutions;

extracting a candidate area in which a target picture is likely to exist in the shot image by performing template matching on each of the plurality of depth images using a template image, the depth images representing, on an image plane, a distance of a subject from a camera in a depth direction as a pixel value, the template image being read from a memory and representing a shape and a size of the target to be analyzed;

wherein, for each of the plurality of depth images, the template image is varied in size during template matching only within a predefined magnification range;

analyzing extracted candidate areas; and

generating an output image reflecting the image analysis results and outputting the output image to a display device.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2017
From: SONY CORPORATION
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 043761/0975 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 7, 2016
From: SONY INTERACTIVE ENTERTAINMENT INC.
To: SONY CORPORATION
Reel/Frame 039099/0369 →
CHANGE OF NAME Recorded Jun 22, 2016
From: SONY COMPUTER ENTERTAINMENT INC.
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 038983/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2015
From: OHBA, AKIO; SEGAWA, HIROYUKI; INADA, TETSUGO; OKAMOTO, HIROFUMI
To: SONY COMPUTER ENTERTAINMENT INC.
Reel/Frame 035468/0909 →