IP Library Granted Patent US 8,614,764
Granted Patent B2
US 8,614,764 · App. 13/867,502 · Granted Dec 24, 2013

Acquiring, editing, generating and outputting video data

Inventors: Colvin Pitts (Snohomish, WA); Yi-Ren Ng (Palo Alto, CA)
Assignee: Lytro, Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,614,764
App. No.
13/867,502
Granted
Dec 24, 2013
Kind
B2
Abstract

Systems and methods are directed to acquiring, generating, manipulating and/or editing video data/frames. The video frames may be light field video frames having virtual aperture parameters that may be adjusted after acquisition or recording of such video frames. In one aspect, a method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of light field video frames; selecting a second key frame, wherein the second key frame corresponds to one of the plurality of light field video frames which is temporally spaced apart from the first key frame such that a plurality of light field video frames are temporally disposed between the first and the second key frames; determining virtual aperture parameters for the first key frame and the second key frame; and generating video data.

Claims (55)

1. A method of generating and outputting video data corresponding to a plurality of video data frames, the method comprising:

receiving light field video data, wherein the light field video data comprises a plurality of temporally contiguous light field video frames, wherein each light field video frame is associated with a virtual aperture parameter;

selecting a first key frame, wherein the first key frame corresponds to one of the plurality of light field video frames;

determining a virtual aperture parameter for the first key frame;

selecting a second key frame, wherein the second key frame corresponds to one of the plurality of light field video frames which is temporally spaced apart from the first key frame such that a plurality of light field video frames are temporally disposed between the first and the second key frames;

determining a virtual aperture parameter for the second key frame;

generating video data corresponding to the plurality of light field video frames which are temporally disposed between the first and the second key frames using (i) the virtual aperture parameter for the first key frame, (ii) the virtual aperture parameter for the second key frame, (iii) an inter-frame interpolation, and (iv) the light field video data corresponding to the plurality of light field video frames which are temporally disposed between the first and the second key frames, wherein the generated video data comprises a plurality of video frames, each video frame comprising a virtual aperture parameter which is based on the virtual aperture parameter for the first key frame, the virtual aperture parameter for the second key frame, and the inter-frame interpolation; and

outputting the generated video data.

2. The method of claim 1 , wherein the inter-frame interpolation comprises linear interpolation.

3. The method of claim 1 , wherein the inter-frame interpolation comprises non-linear interpolation.

4. The method of claim 1 , wherein the virtual aperture parameter comprises at least one of a virtual aperture size and a virtual aperture shape.

5. The method of claim 1 , further comprising inputting the inter-frame interpolation.

6. The method of claim 1 , further comprising generating final output video data by processing the generated video data to provide at least one selected from the group consisting of:

compositing;

transitions;

filtering; and

color adjustments

of a plurality of the video frames corresponding to the generated video data.

7. The method of claim 1 , wherein the virtual aperture parameter of the first key frame and the virtual aperture parameter of the second key frame each comprise information which is representative of a virtual aperture.

8. The method of claim 1 , wherein the virtual aperture parameter of the first key frame is different from an optical aperture parameter of the light field video frame corresponding to the first key frame.

9. The method of claim 1 , wherein at least one of determining a virtual aperture parameter for the first key frame and determining a virtual aperture parameter for the second key frame comprises:

identifying at least one subject in the key frame; and

automatically adjusting the virtual aperture parameter to maintain focus on the at least one subject.

10. A non-transitory system for generating and outputting video data corresponding to a plurality of video data frames, comprising:

first memory, configured to store light field video data, wherein the light field video data comprises a plurality of temporally contiguous light field video frames, wherein each light field video frame is associated with a virtual aperture parameter;

a user interface, configured to receive input data representative of a first key frame and a second key frame; and

processing circuitry, coupled to the first memory and the user interface, configured to:

determine a virtual aperture parameter for the first key frame;

determine a virtual aperture parameter for the second key frame; and

generate video data corresponding to the plurality of light field video frames which are temporally disposed between the first and the second key frames using (i) the virtual aperture parameter for the first key frame, (ii) the virtual aperture parameter for the second key frame, (iii) an inter-frame interpolation, and (iv) the light field video data corresponding to the plurality of light field video frames which are temporally disposed between the first and the second key frames, wherein the generated video data comprises a plurality of video frames, each video frame comprising a virtual aperture parameter which is based on the virtual aperture parameter for the first key frame, the virtual aperture parameter for the second key frame, and the inter-frame interpolation.

11. The non-transitory system of claim 10 , further comprising an output device, coupled to the processing circuitry, configured to output at least a portion of the generated video data.

12. The non-transitory system of claim 10 , further comprising a storage device, coupled to the processing circuitry, configured to store at least a portion of the generated video data.

13. The non-transitory system of claim 10 , wherein the processing circuitry is configured to generate final output video by processing the generated video data to provide at least one selected from the group consisting of:

compositing;

transitions;

filtering; and

color adjustments

of a plurality of the video frames corresponding to the generated video data.

14. The non-transitory system of claim 10 , wherein the inter-frame interpolation comprises linear interpolation.

15. The non-transitory system of claim 10 , wherein the inter-frame interpolation comprises non-linear interpolation.

16. The non-transitory system of claim 10 , wherein the virtual aperture parameter comprises at least one of a virtual aperture size and a virtual aperture shape.

17. The non-transitory system of claim 10 , wherein the user interface is further configured to receive input comprising the inter-frame interpolation.

18. The non-transitory system of claim 10 , wherein the virtual aperture parameter of the first key frame and the virtual aperture parameter of the second key frame each comprise information which is representative of a virtual aperture.

19. The non-transitory system of claim 10 , wherein the virtual aperture parameter of the first key frame is different from an optical aperture parameter of the light field video frame corresponding to the first key frame.

20. The non-transitory system of claim 10 , wherein the processing circuitry is configured to perform at least one of determining a virtual aperture parameter for the first key frame and determining a virtual aperture parameter for the second key frame by:

identifying at least one subject in the key frame; and

automatically adjusting the virtual aperture parameter to maintain focus on the at least one subject.

21. A non-transitory system for generating and outputting video data corresponding to a plurality of video data frames, comprising:

means for storing light field video data, wherein the light field video data comprises a plurality of temporally contiguous light field video frames, wherein each light field video frame is associated with a virtual aperture parameter;

means for receiving input data which is representative of a first key frame and a second key frame;

means, coupled to the storing and receiving means, for:

determining a virtual aperture parameter for the first key frame;

determining a virtual focus depth for the second key frame; and

generating video data corresponding to the plurality of light field video frames which are temporally disposed between the first and the second key frames using (i) the virtual aperture parameter for the first key frame, (ii) the virtual aperture parameter for the second key frame, (iii) an inter-frame interpolation, and (iv) the light field video data corresponding to the plurality of light field video frames which are temporally disposed between the first and the second key frames, wherein the generated video data comprises a plurality of video frames, each video frame comprising a virtual aperture parameter which is based on the virtual aperture parameter for the first key frame, the virtual aperture parameter for the second key frame, and the inter-frame interpolation.

22. The non-transitory system of claim 21 , further comprising means for receiving input data which is representative of the inter-frame interpolation.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2019
From: LYTRO, INC.
To: GOOGLE LLC
Reel/Frame 050009/0829 →
SECURITY INTEREST Recorded Mar 14, 2014
From: LYTRO, INC
To: TRIPLEPOINT CAPITAL LLC
Reel/Frame 032445/0362 →
SECURITY AGREEMENT Recorded Jun 17, 2013
From: LYTRO, INC.
To: TRIPLEPOINT CAPITAL LLC
Reel/Frame 030623/0434 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 22, 2013
From: PITTS, COLVIN; NG, YI-REN
To: LYTRO, INC.
Reel/Frame 030272/0053 →
Continuity (6)
Continuation 13563513 · Jul 31, 2012
Division 12622673 · Nov 20, 2009
Provisional Application 61117621 · Nov 25, 2008
Provisional Application 61120530 · Dec 8, 2008
Provisional Application 61170620 · Apr 18, 2009
Related Publication 20130235267A1 · Sep 12, 2013