IP Library Granted Patent US 9,531,965
Granted Patent B2
US 9,531,965 · App. 14/311,334 · Granted Dec 27, 2016

Controller in a camera for creating a registered video image

Inventor: Peter Lablans (Morris Township, NJ)
Assignee: Spatial Cam LLC
H04N5/3415G03B37/00G06T7/0018H04N5/232H04N5/23212H04N5/23238H04N5/23293H04N5/247H04N5/2628H04N5/341H04N5/3456H04N1/00307H04N2101/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,531,965
App. No.
14/311,334
Granted
Dec 27, 2016
Kind
B2
Abstract

Methods and apparatus to create and display panoramic images on a mobile device are disclosed. Such a mobile device can be a mobile phone. Apparatus is provided to control the position of a lens in relation to a reference lens. Methods and apparatus are provided to generate multiple images that are combined into a panoramic image. A panoramic image may be a static image. It may also be a video image. A controller provides correct camera settings for different conditions. An image processor creates a panoramic image from the correct settings provided by the controller. A panoramic camera is applied in a computer gaming system.

Claims (55)

1. A camera system comprising:

a single body, containing;

a processor to execute instructions;

a memory to store data including instructions on and to retrieve data including instructions from;

a first and a second camera;

a first image sensor and a first lens contained in the first camera;

the first image sensor including an array of sensor elements to generate raw image data, a location of a sensor element in the array of sensor elements on the first image sensor is determined by coordinates;

a second image sensor and a second lens contained in the second camera;

the second image sensor including an array of sensor elements to generate raw image data, a location of a sensor element in the array of sensor elements on the second image sensor is determined by coordinates;

a first set of coordinates stored in the memory, the first set of coordinates determining a first plurality of sensor elements in the array of sensor elements on the first image sensor, the first plurality of sensor elements in the array of sensor elements on the first image sensor being smaller than a total number of sensor elements in the array of sensor elements on the first image sensor;

a second set of coordinates stored in the memory, the second set of coordinates determining a first plurality of sensor elements in the array of sensor elements on the second image sensor, the first plurality of sensor elements in the array of sensor elements on the second image sensor being smaller than a total number of sensor elements in the array of sensor elements on the second image sensor;

the processor is configured with instructions to store on the memory a set of raw image data prior to demosaicing generated by the first pluralities of sensor elements of the first and second cameras as defined by the first and second sets of coordinates, the processor is configured with instructions to read the set of stored raw image data as a representation of a registered image of a scene;

the processor is configured with instructions to demosaic the set of raw image data stored on the memory to generate a registered image of the scene in real-time that is part of a series of video images; and

a display screen, the display screen is configured to display the registered image of the scene.

2. The camera system of claim 1 , wherein the registered image of the scene is a panoramic image of the scene.

3. The camera system of claim 1 , wherein the registered image of the scene is a stereographic image of the scene.

4. The camera system of claim 1 , wherein the first lens and the second lens are trained directly on the scene.

5. The camera system of claim 1 , further comprising:

a focus mechanism configured to focus the first lens in a plurality of focus positions including a first focus position, each of the plurality of focus positions of the first lens corresponding to a unique set of coordinates that determines sensor elements in the array of sensor elements on the first image sensor, the first focus position corresponding to the first set of coordinates.

6. The camera system of claim 5 , wherein

the second lens is configured to be placed in a plurality of focus positions including a first focus position by the focus mechanism, each of the plurality of focus positions of the second lens depends on a corresponding focus position of the first lens; and

each of the plurality of focus positions of the second lens corresponds to a unique set of coordinates that determines sensor elements in the array of sensor elements on the second image sensor, the first focus position of the second lens corresponding to the second set of coordinates.

7. The camera system of claim 1 , wherein the first image sensor and the second image sensor have a rotational misalignment angle that is determined during a calibration step.

8. The camera system of claim 7 , wherein the rotational misalignment angle is applied to transform at least one of the first and second sets of coordinates to memory addresses.

9. The camera system of claim 7 , wherein the rotational misalignment angle is applied to determine a scanning angle of at least one of the first and the second image sensors.

10. The camera system of claim 1 , further comprising:

a user interface generated by the processor on the display screen to manually align images generated by the first and second image sensors and to determine and store in the memory the first set and the second set of coordinates.

11. The camera system of claim 1 , further comprising a third camera.

12. The camera system of claim 1 , wherein the camera system is part of a mobile phone.

13. A camera comprising:

a single body;

a memory configured to store and to retrieve data, including instructions;

a processor, configured to retrieve instructions from the memory and to execute the instructions;

a first and a second imaging unit contained in the single body;

a first image sensor and a first lens contained in the first imaging unit;

the first image sensor including an array of sensor elements to generate raw image data, a location of a sensor element in the array of sensor elements on the first image sensor is determined by coordinates;

a second image sensor and a second lens contained in the second imaging unit;

the second image sensor including an array of sensor elements to generate raw image data, a location of a sensor element in the array of sensor elements on the second image sensor is determined by coordinates;

a first set of coordinates stored on the memory, determining a first scan area with a plurality of sensor elements in the array of sensor elements on the first image sensor, the first scan area with the plurality of sensor elements in the array of sensor elements on the first image sensor containing a smaller number of sensor elements than a total number of sensor elements in the array of sensor elements on the first image sensor;

a second set of coordinates stored on the memory, determining a first scan area with a plurality of sensor elements in the array of sensor elements on the second image sensor, the first scan area with the plurality of sensor elements in the array of sensor elements on the second image sensor containing a smaller number of sensor elements than a total number of sensor elements in the array of sensor elements on the second image sensor;

the memory storing raw image data generated by the first scan area on the first image sensor determined by the first set of coordinates and raw image data generated by the first scan area on the second image sensor determined by the second set of coordinates, as raw image data representing a registered image that is generated by the first and a second image sensors prior to demosaicing of the raw image data;

the processor is configured to execute instructions to demosaic the raw image data representing the registered image that is generated by the first and a second image sensors stored on the memory, wherein the registered image is part of a series of video images generated in real-time; and

a display screen, the display screen configured to display the registered image as part of the series of video images.

14. The camera of claim 13 , wherein the registered image is part of a panoramic video of a scene.

15. A method for generating on a display screen a panoramic video of a scene with a single body camera that contains a first image sensor and a first lens and a second image sensor and a second lens, comprising:

determining a first set of coordinates that defines a first plurality of sensor elements in an array of sensor elements on the first image sensor, the first plurality of sensor elements in the array of sensor elements on the first image sensor being smaller than a total number of sensor elements in the array of sensor elements on the first image sensor and storing the coordinates in a memory;

determining a second set of coordinates, defining a first plurality of sensor elements in an array of sensor elements on the second image sensor, the first plurality of sensor elements in the array of sensor elements on the second image sensor being smaller than a total number of sensor elements in the array of sensor elements on the second image sensor and storing the coordinates in the memory;

a processor storing in the memory, prior to demosaicing, a set of raw image data generated by the first and second image sensors by sensor elements in the first plurality of sensor elements in the array of sensor elements on the first and second image sensors in accordance with the stored first and second sets of coordinates, the set of raw image data enabled to be read from the memory by the processor as a representation of a panoramic image;

the processor demosaicing the set of raw image data that is stored in the memory; and

the processor generating in real-time and displaying on the display screen the panoramic image as part of the panoramic video of the scene.

16. The method of claim 15 , wherein the first and second sets of coordinates correspond to a rotational misalignment angle between the first and the second image sensor.

17. The method of claim 15 , the single body camera further comprising applying a focus mechanism to activate sets of coordinates of the first and second image sensors during recording the panoramic video of the scene.

18. The camera system of claim 1 , wherein the raw image data generated by the first pluralities of sensor elements in the array of sensor elements on the first and second image sensors are stored on the memory by the processor, the processor is configured with instructions to perform a memory address transformation from the arrays of sensor elements to the memory in accordance with the first and second sets of coordinates.

19. The camera system of claim 1 , further comprising a focus mechanism to focus at least the first and second lenses and wherein the first and second sets of coordinates correspond to a first setting of the focus mechanism and a different set of coordinates for the first and second image sensors stored on the memory corresponds to a second setting of the focus mechanism.

20. The camera of claim 13 , wherein memory stores the raw image data generated by the first scan area on the first image sensor determined by the first set of coordinates and the raw image data generated by the first scan area on the second image sensor contiguously.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 10, 2025
From: SPATIAL CAM LLC
To: LABLANS, PETER, MR.
Reel/Frame 073178/0036 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 17, 2016
From: LABLANS, PETER
To: SPATIAL CAM LLC
Reel/Frame 040361/0591 →
Continuity (5)
Continuation 13712941 · Dec 12, 2012
Continuation 12538401 · Aug 10, 2009
Provisional Application 61106025 · Oct 16, 2008
Provisional Application 61106768 · Oct 20, 2008
Related Publication 20140300692A1 · Oct 9, 2014