IP Library › Granted Patent US 11,398,050
Granted Patent B2
US 11,398,050 · App. 15/386,001 · Granted Jul 26, 2022

Apparatus, method and system for adjusting predefined calibration data for generating a perspective view

Inventors: Michael Staudenmaier (Munich, DE); Nikhil Jotwani (Pune, IN)
Assignee: NXP USA, Inc.
G06T7/80B60R1/00G06T3/4038G06T7/521G06T7/593G06T7/60B60R2300/105B60R2300/303B60R2300/607G06T2207/10016G06T2207/20021G06T2207/20221G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,398,050
App. No.
15/386,001
Granted
Jul 26, 2022
Kind
B2
Abstract

The present application relates to a system for generating a surround view and a method of operating the system. A synthesizer module synthesizes an output frame from input frames in accordance with predefined calibration data. The input frames have an overlapping region imaging an overlapping field of view captured by two adjacent cameras. An adjustment module receives height level information representative of a height level in the overlapping region; selects a data record out of a set of predefined calibration data records in accordance with the height level information; and updates the predefined a part of the calibration data with the selected data record.

Claims (62)

1. A method of adjusting predefined calibration data for generating a perspective view,

wherein the perspective view is synthesized in an output frame from input frames in accordance with the predefined calibration data, wherein each input frame originates from one of a plurality of cameras comprising two adjacent cameras, the input frames of which have an overlapping region imaging an overlapping field of view captured by the two adjacent cameras,

wherein the method comprises:

in a computer processing system:

providing a set of predefined calibration data records each of which relating to a different height level of a surface at an elevated level in a corresponding reference environment;

receiving height level information representative of a height level of a surface in the overlapping region in images taken by the two adjacent cameras,

selecting a data record out of the set of predefined calibration data records in accordance with the height level of the surface in the overlapping region;

updating the predefined calibration data with the selected data record; and

generating display images using the input frames and the updated predefined calibration data.

2. The method according to claim 1 ,

wherein each data record of the set of predefined calibration data records comprises a predefined number of predefined calibration data sub-records, each of which relates to a sub-region of the overlapping region,

wherein the height level information is representative of height levels of the sub-regions of the overlapping region;

wherein the adjusting of the predefined calibration data further comprises:

selecting a data sub-record out of the predefined number of predefined calibration data sub-records for each sub-region in accordance with the height level information; and

updating the predefined calibration data with the selected data sub-records.

3. The method according to claim 1 ,

wherein the input frames have a plurality of overlapping regions, each of which images an overlapping field of view captured by two adjacent cameras,

wherein the adjusting of the predefined calibration data further comprises:

providing a set of predefined calibration data records for each overlapping region;

receiving height level information representative of several height levels each relating to one of the overlapping regions;

selecting data records out of the respective set of predefined calibration data records in accordance with the height level information, wherein each selected data record relates to a different one of the overlapping regions; and

updating the predefined calibration data with the selected data records.

4. The method according to claim 1 ,

wherein the adjusting of the predefined calibration data is performed independently of the synthesizing of the output frame.

5. The method according to claim 1 ,

wherein each two adjacent cameras of the plurality of cameras capture an overlapping field of view, which is imaged in an overlapping region of the respective two different input frames.

6. The method according to claim 1 ,

wherein the synthesizing further comprises stitching the overlapping region.

7. The method according to claim 1 , further comprising:

providing the predefined calibration data, which maps each pixel location of the output frame to a location in at least one of the input frames.

8. The method according to claim 1 ,

wherein the predefined calibration data includes optical correction data and/or blending parameters.

9. The method according to claim 1 , further comprising:

providing the input frames synchronized in time for synthesizing.

10. The method according to claim 1 , further comprising:

displaying the display images on a display device.

11. A system for adjusting predefined calibration data for generating a perspective view,

wherein the perspective view is synthesized in an output frame by a synthesizer module from input frames in accordance with predefined calibration data,

wherein each input frame originates from a different one of a plurality of cameras comprising two adjacent cameras, the input frames of which have an overlapping region imaging an overlapping field of view captured by the two adjacent cameras,

a set of predefined calibration data records each of which relating to a different height level of a surface in a reference environment,

wherein the system comprises an adjustment module configured to receive height level information representative of a height level of a surface in the overlapping region in images taken by the two adjacent cameras;

to select a data record out of the set of predefined calibration data records in accordance with the height level information; and

to update the predefined calibration data with the selected data record.

12. The system according to claim 11 ,

wherein each data record of the set of predefined calibration data records comprises a predefined number of predefined calibration data sub-records, each of which relates to a sub-region of the overlapping region,

wherein the height level information is representative of height levels of the sub-regions of the overlapping region,

wherein the adjustment module is further configured

to select a data sub-record out of the predefined number of predefined calibration data sub-records for each sub-region in accordance with the height level information; and

to update the predefined calibration data with the selected data sub-records.

13. The system according to claim 11 ,

wherein the input frames have a plurality of overlapping regions, each of which images an overlapping field of view captured by two adjacent cameras,

wherein the system comprises a set of predefined calibration data records for each overlapping region,

wherein the adjustment module is further configured

to receive height level information representative of one or more height levels each relating to one of the overlapping regions;

to select one or more data records out of the one or more respective sets of predefined calibration data records in accordance with the height level information; and

to update the predefined calibration data with the one or more selected data records,

wherein each of the one or more selected data records relates to a different one of the overlapping regions.

14. The system according to claim 11 , further comprising:

wherein the predefined calibration data maps each pixel location of the output frame to a location in at least one of the input frames.

15. The system according to claim 11 , further comprising:

wherein the height level information is determined on the basis of sensor outputs or on the basis of an analysis of the images in the overlapping regions of the input frames.

16. A non-transitory storage medium comprising computer executable instructions, which, when executing on one or more processing devices, cause the one or more processing devices to perform a method of generating a perspective view according to claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 21, 2016
From: STAUDENMAIER, MICHAEL; JOTWANI, NIKHIL
To: NXP USA, INC.
Reel/Frame 041086/0508 →
Priority Claims (1)
EP 16155365 · Feb 11, 2016 · regional
Continuity (1)
Related Publication 20170236305A1 · Aug 17, 2017