IP Library › Granted Patent US 11,132,553
Granted Patent B2
US 11,132,553 · App. 16/621,860 · Granted Sep 28, 2021

Information processing apparatus and information processing method

Inventor: Nobuhiro Tsunashima (Kanagawa, JP)
Assignee: SONY CORPORATION
G06K9/00724G06K2009/00738
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,132,553
App. No.
16/621,860
Granted
Sep 28, 2021
Kind
B2
Abstract

An information processing apparatus including a setting processor that sets metadata for objects included in captured images captured in a time series by an imaging device based on the captured image, in which the setting processor processes the captured images in a reverse time series reverse to the time series from a predetermined event detected from the captured images and sets the metadata for objects involved in the predetermined event among the objects included in the captured images.

Claims (74)

1. An information processing apparatus, comprising:

a setting processor configured to:

detect an event from a plurality of captured images, wherein the plurality of captured images is captured in a time series by an imaging device of a plurality of imaging devices;

determine two objects from a plurality of objects as same objects, wherein

the plurality of objects is included in the plurality of captured images,

the determination of the same objects is based on a position of each object of the two objects and a set of images of the plurality of captured images, and

the determined two objects are present in the set of images;

set a plurality of pieces of metadata for the plurality of objects, wherein

two pieces of metadata of the plurality of pieces of metadata associated with the determined two objects are different,

the plurality of pieces of metadata is set based on the plurality of captured images;

process the plurality of captured images in a reverse time series that is reverse to the time series from the detected event; and

integrate the two different pieces of metadata for the same objects.

2. The information processing apparatus according to claim 1 , wherein the setting processor is further configured to:

set a first piece of metadata of the plurality of pieces of metadata for a first object of the plurality of objects directly involved in the detected event, and

set a second piece of metadata of the plurality of pieces of metadata for a second object of the plurality of objects indirectly involved in the detected event.

3. The information processing apparatus according to claim 2 , wherein

the setting processor is further configured to:

identify the first object from the plurality of captured images,

determine a third object of the plurality of objects involved with the identified first object based on a processing operation on the plurality of captured images in the reverse time series, and

set the determined third object involved with the first object as the second object.

4. The information processing apparatus according to claim 3 , wherein

the setting processor is further configured to:

detect the first object for each captured image of the plurality of captured images, wherein each captured image corresponds to a respective time point of the reverse time series,

determine a fourth object of the plurality of objects directly involved with the first object, wherein the fourth object is determined from the plurality of captured images in which the first object is detected, and

set the determined fourth object as the second object.

5. The information processing apparatus according to claim 4 , wherein

the setting processor is further configured to determine a fifth object of the plurality of objects, and

a distance of the determined fifth object from the first object is equal to or less than a threshold.

6. The information processing apparatus according to claim 4 , wherein the setting processor is further configured to:

determine a fifth object of the plurality of objects directly involved with the fourth object, wherein the fifth object is determined from the plurality of captured images that includes the determined fourth object and

set the determined fifth object as the second object.

7. The information processing apparatus according to claim 1 , wherein

the setting processor is further configured to process each captured image of the plurality of captured images,

each captured image of the plurality of captured images corresponds to a respective time point of the reverse time series, and

the plurality of captured images are processed from a first time point at which the event is detected to a second time point at which a condition in the reverse time series is satisfied.

8. The information processing apparatus according to claim 7 , wherein the condition includes at least one of a temporal condition or a condition related to an object of the plurality of objects.

9. The information processing apparatus according to claim 1 , wherein the setting processor is further configured to:

process the plurality of captured images in the time series from detected event; and

set the plurality of pieces of metadata in the plurality of objects.

10. The information processing apparatus according to claim 1 , wherein

the setting processor is further configured to set the plurality of pieces of metadata for the plurality of objects based on the plurality of captured images, and

the plurality of images is captured by the plurality of imaging devices.

11. The information processing apparatus according to claim 10 , wherein the setting processor is further configured to:

process the plurality of captured images for each of the plurality of imaging devices, and

set the plurality of pieces of metadata for the plurality of objects for each of the plurality of imaging devices.

12. The information processing apparatus according to claim 11 , wherein the setting processor is further configured to associate the plurality of objects with the set plurality of pieces of metadata for each of the plurality of imaging devices.

13. The information processing apparatus according to claim 12 , wherein the setting processor is further configured to record relevance of the set plurality of pieces of metadata for the associated plurality of objects for each set piece of metadata of the set plurality of pieces of metadata.

14. The information processing apparatus according to claim 10 , wherein

the setting processor is further configured to:

set the plurality of pieces of metadata for a set of objects of the plurality of objects, wherein the set of objects is involved in the detected event, wherein

the plurality of images is captured by a first set of imaging devices of the plurality of imaging devices, and

the plurality of pieces of metadata is set based on the plurality of captured images captured by the first set of the imaging devices, and

set the plurality of pieces of metadata for the set of objects of the plurality of objects, wherein the set of objects is involved in the detected event, wherein

the plurality of images is captured by a second set of imaging devices other than the first set of the imaging devices,

the plurality of imaging devices includes the second set of imaging devices,

the second plurality of pieces of metadata is set based on a set result of the plurality of pieces of metadata, and

the plurality of pieces of metadata is set based on each of the plurality of captured images captured by the first set of the imaging devices and the plurality of captured images captured by the second set of imaging devices.

15. The information processing apparatus according to claim 1 , further comprising an editing processor configured to edit the plurality of captured images based on a set result of the plurality of pieces of metadata for the plurality of objects.

16. The information processing apparatus according to claim 15 , wherein

the editing processor edits the plurality of captured images based on an image processing operation, and

the image processing operation includes at least one of an electronic zoom operation on the plurality of captured images, or a combination of a plurality of images based on the plurality of captured images.

17. The information processing apparatus according to claim 1 , further comprising a display processor configured to display an image of the plurality of captured images on a display screen based on a set result of the plurality of pieces of metadata, wherein the displayed image displays the plurality of objects involved in the detected event for which the plurality of pieces of metadata is set.

18. The information processing apparatus according to claim 17 , wherein the display processor is further configured to change the display of the plurality of objects based on display setting information.

19. An information processing method, comprising:

detecting an event from a plurality of captured images, wherein the plurality of captured images is captured in a time series by an imaging device of a plurality of imaging devices;

determining two objects from a plurality of objects as same objects, wherein

the plurality of objects is included in the plurality of captured images,

the determination of the same objects is based on a position of each object of the two objects and a set of images of the plurality of captured images, and

the determined two objects are present in the set of images;

setting a plurality of pieces of metadata for the plurality of objects, wherein

two pieces of metadata of the plurality of pieces of metadata associated with the determined two objects are different,

the plurality of pieces of metadata is set based on the plurality of captured images;

processing, in the setting, the plurality of captured images in a reverse time series that is reverse to the time series from the detected event; and

integrating the two different pieces of metadata for the same objects.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2020
From: TSUNASHIMA, NOBUHIRO
To: SONY CORPORATION
Reel/Frame 052051/0196 →
Priority Claims (1)
JP JP2017-141808 · Jul 21, 2017 · national
Continuity (1)
Related Publication 20200167571A1 · May 28, 2020