IP Library › Granted Patent US 9,811,932
Granted Patent B2
US 9,811,932 · App. 14/689,369 · Granted Nov 7, 2017

Display controller, heads-up image display system and method thereof

Inventors: Michael Andreas Staudenmaier (Munich, DE); Kshitij Bajaj (Sirsa, IN); Chanpreet Singh (S.A.S. Nagar, IN)
Assignee: NXP USA, INC.
G06T11/60B60K35/00G02B27/0101G06T3/0093G06T5/006G09G5/38B60K2350/1096B60K2350/2052G02B2027/013G02B2027/0123G02B2027/0141G09G2340/10G09G2340/12G09G2340/125G09G2358/00G09G2360/02G09G2360/06G09G2380/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,811,932
App. No.
14/689,369
Granted
Nov 7, 2017
Kind
B2
Abstract

The present application relates to a display controller and display system and a method of operating thereof. At a filtering stage display image data are generated on the basis of received pixel-mapped image data. The filtering operation accepts a plurality of pixels out of the received image data as input values to generate a pixel of the display image data as output value. It is further determined whether the plurality of pixels being the input values to the filtering operation are marked. If all pixels thereof are marked, the output pixel being the output value is marked. The marked pixels in the display image data are validated on the basis reference data.

Claims (65)

1. A display controller, comprising:

a filter stage to receive an input image layer comprising a plurality of input image pixels, and to provide a plurality of output image pixels based on the plurality of input image pixels, wherein:

each pixel of the plurality of input image pixels includes marking information indicating safety relevant or non-safety relevant;

each pixel of the plurality of output image pixels includes marking information indicating safety relevant or non-safety relevant;

a particular output image pixel of the plurality of output image pixels is based on a group of more than one of the plurality of input image pixels; and

the marking information of the particular output image pixel is set by the filter stage to indicate safety relevant in response to all of the pixels of the group being marked as safety relevant; and

a verifier stage, coupled to the filter stage, to validate the plurality of output image pixels based on the marking information of an output image pixel indicating that the output image pixel is safety relevant, and based on reference data.

2. The display controller of claim 1 , further comprising:

at least one data input to receive first image data representing a safety relevant image part and to receive second image data representing a non-safety relevant image part;

a merging stage coupled to the at least one data input to receive the first image data and the second image data and arranged to compose the first image data and the second image data to form the plurality of input image pixels to be provided to the filter stage,

wherein pixels belonging to the first image data in the plurality of input image pixels include marking information indicating safety relevant.

3. The display controller of claim 1 ,

wherein the each pixel of the plurality of input image pixels comprises a color information.

4. The display controller of claim 1 , wherein the verifier stage includes:

a comparator portion arranged to compare marked pixels in the plurality of output image pixels with the reference data to validate the plurality of output image pixels with respect to the safety relevant image part included therein.

5. The display controller of claim 1 , wherein the verifier stage includes:

a checksum calculator arranged to determine a checksum from marked pixels in the plurality of output image pixels, wherein the checksum is validated against the reference data.

6. The display controller of claim 5 ,

wherein the checksum calculator is further arranged to determine the checksum on the basis of at least one of pixel values of the marked pixels and locations of the marked pixels in an image space.

7. The display controller of claim 5 , wherein the verifier stage further includes:

at least one image data input to receive the plurality of output image pixels;

at least one parameter input to receive at least one of information about the safety relevant image part and information about the filtering operation applied to generate the plurality of output image pixels; and

a set of reference data, wherein each reference data is associated with a specific safety relevant image part and a specific filtering operation applied to generate the plurality of output image pixels,

wherein the verifier stage is further arranged to select the reference data for validation from the set of reference data on the basis of the received information.

8. The display controller of claim 1 , wherein the verifier stage includes:

a reference table arranged to store a set of reference data records, wherein each record is associated with a specified safety relevant part and a specified filtering operation.

9. The display controller of claim 1 , wherein the filter stage includes:

a warp mapping stage arranged to operate a warp mapping operation to generate the plurality of output image pixels on the basis of the received plurality of input image pixels, wherein a display image composed by the plurality of output image pixels is warped to be displayed on a non-planar surface in such a way that an observer of the displayed image perceives the image as being undistorted.

10. The display controller of claim 9 ,

wherein the warp mapping stage is configurable to adjust the warp mapping operation to generate warped image data for being projected on non-planar surfaces with differing curvatures and/or observed by the observer at differing viewing distances and angles.

11. The display controller of claim 1 , wherein the filter stage includes:

a safety voter portion arranged to generate marking data for the plurality of output image pixels on the basis of marking data of the received plurality of input image pixels.

12. The display controller of claim 1 , further comprising:

a first data input to receive first image data representing a safety relevant image part from at least one image data source; and

a second data input to receive second image data representing a non-safety relevant image part from at least one image data source.

13. The display controller of claim 1 , further comprising:

a display system, including:

a displaying component, coupled to the filter stage, for displaying an image representative of the plurality of output image pixels to an observer;

wherein the filter stage is arranged to supply the display image data to the displaying component.

14. The display controller of claim 13 ,

wherein the display system is a heads-up display system, and

wherein the displaying component is an image projector for projecting the image representative of the plurality of output image pixels on a surface into the line of sight of the observer.

15. The display controller of claim 1 , further comprising:

a merge stage to receive source image data from a plurality of image sources, to generate a merged image based on the source image data and to provide the merged image to the filter stage as the input image layer.

16. A method of operating a display controller, said method comprising:

receiving an input image layer comprising a plurality of input image pixels comprising a safety relevant image part and a non-safety relevant image part, wherein the each pixel of the plurality of input image pixels includes marking information indicating safety relevant or non-safety relevant;

performing a filtering operation on the received plurality of input image pixels, wherein the filtering operation provides a plurality of output image pixels based on the plurality of input image pixels;

each pixel of the plurality of output image pixels includes marking information indicating safety relevant or non-safety relevant;

a particular output image pixel of the plurality of output image pixels is based on a group of more than one of the plurality of input image pixels; and

the marking information of the particular output image pixel is set by the filtering operation to indicate safety relevant in response to all of the pixels of the group being marked as safety relevant;

generating display image data on the basis of the plurality of output image pixels; and

validating the plurality of output image pixels based on the marking information of an output image pixel indicating that the output image pixel is safety relevant, and based on a basis reference data.

17. The method according to claim 16 ,

receiving first image data representing the safety relevant image part;

receiving second image data representing the non-safety relevant image part; and

composing the first image data and the second image data to form the plurality of input image pixels to be provided for filtering.

18. The method according to claim 16 , further comprising:

determining a checksum on the basis of the plurality of output image pixels; and

validating the checksum against the basis reference data.

19. The method according to claim 16 , wherein the filtering operation further comprises:

performing a warp mapping operation on the plurality of input image pixels to generate the plurality of output image pixels, wherein a display image composed by the plurality of output image pixels is warped to be displayed on a non-planar surface in such a way that an observer of the displayed image perceives the image as being undistorted.

20. The method of claim 16 , further comprising:

receiving source image data from a plurality of image sources;

generating a merged image based on the source image data; and

providing the merged image as the input image layer.

Assignments (15)
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE APPLICATION 11759915 AND REPLACE IT WITH APPLICATION 11759935 PREVIOUSLY RECORDED ON REEL 040925 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE RELEASE OF SECURITY INTEREST. Recorded Feb 17, 2020
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP, B.V. F/K/A FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 052917/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE APPLICATION 11759915 AND REPLACE IT WITH APPLICATION 11759935 PREVIOUSLY RECORDED ON REEL 040928 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE RELEASE OF SECURITY INTEREST. Recorded Jan 17, 2020
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP B.V.
Reel/Frame 052915/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 10, 2019
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP B.V.
Reel/Frame 050744/0097 →
CORRECTIVE ASSIGNMENT TO CORRECT THE NATURE OF CONVEYANCE PREVIOUSLY RECORDED AT REEL: 040626 FRAME: 0683. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER AND CHANGE OF NAME EFFECTIVE NOVEMBER 7, 2016. Recorded Jan 12, 2017
From: NXP SEMICONDUCTORS USA, INC. (MERGED INTO); FREESCALE SEMICONDUCTOR, INC. (UNDER)
To: NXP USA, INC.
Reel/Frame 041414/0883 →
CHANGE OF NAME Recorded Nov 16, 2016
From: FREESCALE SEMICONDUCTOR INC.
To: NXP USA, INC.
Reel/Frame 040626/0683 →
RELEASE OF SECURITY INTEREST Recorded Nov 7, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP B.V.
Reel/Frame 040928/0001 →
RELEASE OF SECURITY INTEREST Recorded Sep 21, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: NXP, B.V., F/K/A FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 040925/0001 →
SUPPLEMENT TO THE SECURITY AGREEMENT Recorded Jun 16, 2016
From: FREESCALE SEMICONDUCTOR, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 039138/0001 →
ASSIGNMENT AND ASSUMPTION OF SECURITY INTEREST IN PATENTS Recorded Jan 20, 2016
From: CITIBANK, N.A.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 037565/0510 →
ASSIGNMENT AND ASSUMPTION OF SECURITY INTEREST IN PATENTS Recorded Jan 20, 2016
From: CITIBANK, N.A.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 037565/0527 →
PATENT RELEASE Recorded Dec 21, 2015
From: CITIBANK, N.A., AS COLLATERAL AGENT
To: FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 037357/0859 →
SUPPLEMENT TO IP SECURITY AGREEMENT Recorded Aug 6, 2015
From: FREESCALE SEMICONDUCTOR, INC.
To: CITIBANK, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 036284/0363 →
SUPPLEMENT TO IP SECURITY AGREEMENT Recorded Aug 6, 2015
From: FREESCALE SEMICONDUCTOR, INC.
To: CITIBANK, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 036284/0339 →
SUPPLEMENT TO IP SECURITY AGREEMENT Recorded Aug 6, 2015
From: FREESCALE SEMICONDUCTOR, INC.
To: CITIBANK, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 036284/0105 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2015
From: STAUDENMAIER, MICHAEL ANDREAS; BAJAJ, KSHITIJ; SINGH, CHANPREET
To: FREESCALE SEMICONDUCTOR, INC.
Reel/Frame 035434/0619 →
Continuity (1)
Related Publication 20160307346A1 · Oct 20, 2016