IP Library Granted Patent US 8,818,097
Granted Patent B2
US 8,818,097 · App. 13/555,443 · Granted Aug 26, 2014

Portable electronic and method of processing a series of frames

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,818,097
App. No.
13/555,443
Granted
Aug 26, 2014
Kind
B2
Abstract

A portable electronic equipment comprises an image sensor configured to capture a series of frames in a time-sequential manner and a processing device coupled to the image sensor and configured to process the series of frames. The processing device is configured to perform an image segmentation for respectively each frame of at least a subset of the series, to respectively identify a section of the frame representing a given object. The image segmentation is respectively performed based on a depth map for the respective frame. The identified sections are superimposed to generate a sequence image.

Claims (46)

1. A portable electronic equipment, comprising:

an image sensor configured to capture a series of frames in a time-sequential manner; and

a processing device coupled to said image sensor and configured to process said series of frames, said processing device being configured to:

perform an image segmentation for respectively each frame of at least a subset of said series, to respectively identify a section of said frame, said section representing a given object which moves in a plane at a distance from the portable electronic equipment, said image segmentation being respectively performed based on a depth map for the respective frame; and

superimpose said identified sections to generate a sequence image which shows the given object at plural states of its movement relative to the portable electronic equipment in the plane which is located at the distance from the portable electronic equipment.

2. The portable electronic equipment of claim 1 ,

wherein said processing device is configured to establish the distance between said given object and said portable electronic equipment using a depth map for a given frame of said series, and to identify said section representing said given object in another frame of said series based on both said established distance and said depth map for said other frame.

3. The portable electronic equipment of claim 2 ,

wherein said processing device is configured to compare pixels of said depth map for said other frame to said established distance to identify said section representing said given object in said other frame.

4. The portable electronic equipment of claim 3 ,

wherein said processing device is configured to determine whether said pixels of said depth map for said other frame correspond to distances which lie within a distance range, said distance range being determined as a function of said established distance.

5. The portable electronic equipment of claim 2 ,

wherein said processing device is configured to identify, for respectively each frame of at least said subset, said section representing said given object such that a distance between said given object and said portable electronic equipment corresponds to said established distance.

6. The portable electronic equipment of claim 5 ,

wherein said distance between said given object and said portable electronic equipment corresponds to a distance of a focal plane of said image sensor from said portable electronic equipment.

7. The portable electronic equipment of claim 2 , further comprising:

an input interface configured to receive a user input indicating said given object in said given frame;

wherein said processing device is coupled to said input interface and is configured to establish said distance as a function of said user input.

8. The portable electronic equipment of claim 1 ,

wherein said processing device is configured to superimpose said identified sections as layers onto one of said frames of said series to generate said sequence image.

9. The portable electronic equipment of claim 1 , further comprising:

an optical output device;

wherein said processing device is coupled to said optical output device and is configured to control said optical output device to output said generated sequence image.

10. A portable electronic equipment, comprising:

an image sensor configured to capture a series of frames in a time-sequential manner,

wherein said image sensor includes a plurality of image sensor sections, each of said image sensor sections being respectively configured to capture an image, wherein fields of views of the plurality of image sensor sections overlap and wherein each one of the image sensor sections has its own dedicated focusing lens; and

a processing device coupled to said image sensor and configured to process said series of frames, said processing device being configured to:

perform an image segmentation for respectively each frame of at least a subset of said series, to respectively identify a section of said frame, said section representing a given object which moves in a plane at a distance from the portable electronic equipment, and

superimpose said identified sections to generate a sequence image which shows the given object at plural states of its movement relative to the portable electronic equipment in the plane which is located at the distance from the portable electronic equipment.

11. The portable electronic equipment of claim 10 ,

wherein said processing device is configured to process, for each frame of at least said subset, images captured by said plurality of image sensor sections to compute the respective frame and the depth map for the frame.

12. A method of processing a series of frames captured by an image sensor of a portable electronic equipment, said method comprising the following acts performed by said portable electronic equipment:

identifying, for respectively each frame of at least a subset of said series, a section of said frame, said section representing a given object which moves in a plane at a distance from the portable electronic equipment, wherein an image segmentation is respectively performed based on a depth map for the respective frame to identify said section; and

superimposing said identified sections to generate a sequence image which shows the given object at plural states of its movement relative to the portable electronic equipment in the plane which is located at the distance from the portable electronic equipment.

13. The method of claim 12 ,

wherein the distance between said given object and said portable electronic equipment is established using a depth map for a given frame of said series; and

wherein said identifying said section representing said given object in another frame of said series respectively comprises:

comparing pixels of said depth map for said other frame to said established distance.

14. The method of claim 13 ,

wherein, for respectively each frame of at least said subset, said section representing said given object is identified such that a distance between said given object and said portable electronic equipment corresponds to said established distance.

15. The method of claim 12 ,

performed by portable electronic equipment, comprising:

an image sensor configured to capture the series of frames in a time-sequential manner; and

a processing device coupled to said image sensor and configured to process said series of frames, said processing device being configured to:

perform the image segmentation for respectively each frame of at least the subset of said series, to respectively identify said section of said frame, said section representing said given object, said image segmentation being respectively performed based on said depth map for the respective frame; and

superimpose said identified sections to generate said sequence image.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 25, 2019
From: SONY MOBILE COMMUNICATIONS, INC.
To: SONY CORPORATION
Reel/Frame 048691/0134 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2017
From: SONY MOBILE COMMUNICATIONS AB
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 043951/0529 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 21, 2017
From: SONY CORPORATION
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 043943/0631 →
ASSIGNMENT OF PARTIAL INTEREST Recorded Jul 11, 2014
From: SONY MOBILE COMMUNICATIONS AB
To: SONY CORPORATION; SONY MOBILE COMMUNICATIONS AB
Reel/Frame 033300/0664 →
CHANGE OF NAME Recorded Jul 10, 2014
From: SONY ERICSSON MOBILE COMMUNICATIONS AB
To: SONY MOBILE COMMUNICATIONS AB
Reel/Frame 033295/0320 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2012
From: WERNERSSON, MATS; HERINGSLACK, HENRIK
To: SONY MOBILE COMMUNICATIONS AB
Reel/Frame 028612/0932 →