IP Library Granted Patent US 9,787,918
Granted Patent B2
US 9,787,918 · App. 13/984,950 · Granted Oct 10, 2017

Acquiring and displaying images in real-time

Inventor: Arjan Willem De Groot (Groningen, NL)
Assignee: PHOTONIS NETHERLANDS B.V.
H04N5/3532H04N5/23293H04N5/33H04N13/0239H04N13/0296G02B23/125H04N5/262
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,787,918
App. No.
13/984,950
Granted
Oct 10, 2017
Kind
B2
Abstract

An imaging device ( 100 ) for acquiring and displaying images in real-time, the imaging device comprising i) an imaging sensor ( 110 ) comprising a radiation sensitive array ( 120 ) for acquiring an image ( 142 ), ii) a readout circuit ( 140 ) connected to the radiation sensitive array for reading out the image, iii) a signal processor ( 160 ) for processing the image for obtaining a processed image ( 162 ), and iv) a display ( 180 ) for displaying the processed image, the radiation sensitive array being arranged in rows of sensor pixels and the display being arranged in rows of display pixels, and wherein the readout circuit is a rolling shutter circuit for sequentially reading out the rows of sensor pixels for sequentially providing subsets of pixels, the signal processor is configured for, on availability of one of the subsets of pixels, processing the subset of pixels for providing a processed subset of pixels, and the display is configured for, on availability of the processed subset of pixels, displaying the processed subset of pixels on a thereto corresponding subset of display pixels for displaying the processed image sequentially on the rows of display pixels.

Claims (32)

1. A night vision device arranged for acquiring and displaying images in real-time, the night vision device comprising:

i) an imaging sensor comprising a radiation sensitive array for acquiring an image,

ii) a readout circuit connected to the radiation sensitive array for reading out the image,

iii) a signal processor for processing the image for obtaining a processed image, and

iv) a display for displaying the processed image, the radiation sensitive array being arranged in rows of sensor pixels and the display being arranged in rows of display pixels, and wherein:

the readout circuit is a rolling shutter circuit for sequentially reading out the rows of sensor pixels by a) initializing an exposure of a row, and b) reading out its contents after said exposure, for sequentially providing subsets of pixels), each subset of pixels corresponding to one row or a subset of the rows of sensor pixels or a subset of pixels from a single row;

the signal processor is configured for, on availability of one of the subsets of pixels, processing the subset of pixels for providing a processed subset of pixels; and

the display is configured for, on availability of the processed subset of pixels, displaying the processed subset of pixels on a thereto corresponding subset of display pixels for displaying the processed image sequentially on the rows of display pixels, and wherein

the direct view system comprises a further imaging sensor and a further readout circuit, the further imaging sensor comprising a further radiation sensitive array for acquiring a further image, the further readout circuit being connected to the further radiation sensitive array for reading out the further image, the further radiation sensitive array being arranged in rows of further sensor pixels, the further readout circuit being a further rolling shutter circuit for sequentially reading out the rows of further sensor pixels for sequentially providing further subsets of pixels, and wherein the direct view system is configured for synchronously displaying the image and the further image on the display by:

(1) the rolling shutter circuit and the further rolling shutter circuit being configured for synchronously providing the subset of pixels and one of the further subsets of pixels by substantially synchronously reading out corresponding portions of the image and the further image; and

(2) the signal processor being configured for, on availability of the subset of pixels and the further subset of pixels, combining the subset of pixels with the further subset of pixels for obtaining the processed subset of pixels, wherein

the radiation sensitive array has a first spatial resolution, the further radiation sensitive array has a second spatial resolution, the second spatial resolution being lower than the first spatial resolution and the further rolling shutter circuit being configured for reading out the further image with a second readout speed that is lower than a first readout speed of the rolling shutter circuit for enabling said synchronously providing the subset of pixels and the further subset of pixels, and wherein

the rolling shutter circuit is clocked at a first pixel clock for providing the first readout speed and the further rolling shutter circuit is clocked at a second pixel clock for providing the second readout speed.

2. The night vision device according to claim 1 , wherein the imaging sensor is a visible light imaging sensor for sensing visible light and the further imaging sensor is a thermal imaging sensor for sensing infrared radiation for enabling synchronously displaying a visible light image and a thermal image on the display.

3. The night vision device according to claim 1 , wherein the signal processor is configured for combining the subset of pixels with the further subset of pixels by fusing the subset of pixels with the further subset of pixels for obtaining as the processed image an image fusion of the image with the further image.

4. The night vision device according to claim 1 , wherein the rolling shutter circuit is configured for reading out the image with the first readout speed within an imaging frame time (Ti), and the further rolling shutter circuit is configured for reading out the further image with the second readout speed within the imaging frame time.

5. The night vision device according to claim 1 , wherein the night vision device comprises a scaler for spatially scaling the further subset of pixels for providing as the further image a scaled image having the first spatial resolution.

6. The night vision device according to claim 5 , wherein the scaler is configured for performing the spatial scaling using at least one technique out of the group of: pixel repetition, first order linear interpolation, higher order linear interpolation and non-linear interpolation techniques.

7. The night vision device according to claim 1 , wherein the signal processor comprises an image processing pipeline for obtaining a pipelined processing of the subsets of pixels.

8. The night vision device according to claim 1 , wherein the rolling shutter circuit is configured for reading out the image with a first readout speed, and wherein the night vision device is configured for establishing the first readout speed in dependence on an amount of radiation impinging on the radiation sensitive array.

9. The night vision device according to claim 1 , wherein the night vision device comprises an image intensifier for providing intensified visible light to the imaging sensor.

10. A helmet, head mount, rifle sight or handheld device comprising the night vision device according to claim 1 .

11. A method of acquiring and displaying images in real-time with a night vision device, the night vision device comprising i) an imaging sensor comprising a radiation sensitive array for acquiring an image, ii) a readout circuit connected to the radiation sensitive array for reading out the image, iii) a signal processor for processing the image for obtaining a processed image, and iv) a display for displaying the processed image, the radiation sensitive array being arranged in rows of sensor pixels and the display being arranged in rows of display pixels, and wherein the method comprises:

sequentially reading out the rows of sensor pixels with the readout circuit by

(a) initializing an exposure of a row, and (b) reading out its contents after said exposure, for sequentially providing subsets of pixels, each subset of pixels corresponding to one row or a subset of the rows of sensor pixels or a subset of pixels from a single row;

on availability of one of the subsets of pixels, processing the subset of pixels with the signal processor for providing a processed subset of pixels; and

on availability of the processed subset of pixels, displaying the processed subset of pixels with the display on a thereto corresponding subset of display pixels for displaying the processed image sequentially on the rows of display pixels, wherein

the night vision device comprises a further imaging sensor and a further readout circuit, the further imaging sensor comprising a further radiation sensitive array for acquiring a further image, the further readout circuit being connected to the further radiation sensitive array for reading out the further image, the further radiation sensitive array being arranged in rows of further sensor pixels, the further readout circuit being a further rolling shutter circuit for sequentially reading out the rows of further sensor pixels for sequentially providing further subsets of pixels, and wherein the method comprises synchronously displaying the image and the further image on the display by:

synchronously providing the subset of pixels and one of the further subsets of pixels by substantially synchronously reading out corresponding portions of the image and the further image; and

on availability of the subset of pixels and the further subset of pixels, combining the subset of pixels with the further subset of pixels for obtaining the processed subset of pixels, wherein

the radiation sensitive array has a first spatial resolution, the further radiation sensitive array has a second spatial resolution, the second spatial resolution being lower than the first spatial resolution and the method further comprising reading out the further image with a second readout speed that is lower than a first readout speed of the rolling shutter circuit for enabling said synchronously providing the subset of pixels and the further subset of pixels, wherein method comprises clocking the rolling shutter circuit at a first pixel clock for providing the first readout speed and clocking the further rolling shutter circuit at a second pixel clock for providing the second readout speed.

12. A computer program stored on a non-transitory computer-readable medium, the computer program comprising instructions for causing a processor system to perform the method according to claim 11 .

Assignments (2)
SECURITY AGREEMENT Recorded Nov 6, 2013
From: PHOTONIS NETHERLANDS B.V.; IMAGING SENSORS B.V.
To: CREDIT SUISSE AG
Reel/Frame 031589/0396 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2013
From: DE GROOT, ARJAN WILLEM
To: PHOTONIS NETHERLANDS B.V.
Reel/Frame 030990/0066 →
Continuity (1)
Related Publication 20130314546A1 · Nov 28, 2013