IP Library › Granted Patent US 10,412,369
Granted Patent B2
US 10,412,369 · App. 14/815,614 · Granted Sep 10, 2019

Method and apparatus for compensating for camera error in a multi-camera stereo camera system

Inventors: Deeder M. Aurongzeb (Austin, TX); Maxwell Andrews (San Francisco, CA)
Assignee: Dell Products, LP
H04N13/246G06T7/80G06T7/85H04N13/106H04N13/15H04N13/189H04N13/243H04N13/128H04N2013/0081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,412,369
App. No.
14/815,614
Filed
Jul 31, 2015
Granted
Sep 10, 2019
Kind
B2
Art Unit
2483
USPC
348/139
Abstract

A system and method multi-camera error compensation including recording a plurality of raw images via a plurality of digital cameras and an application processor processing with a multi-view stereo imaging system one or more plural image frames from the raw images captured by the plurality of digital cameras. The plural images may be stored in memory and the detection conducted for calibration loss of at least one digital camera via the processor executing instructions for a multi-camera error compensating system to determine loss of calibration in plural images. The multi-camera error compensating system conducts dynamic recalibration of plural image calibration parameters based on at least one plural image frame and in response to detection of calibration loss via the multi-camera error compensating system and a processor reprocesses the plural image frame from a reprocessing queue according to the recalibrated plural image parameters.

Claims (54)

1. A method multi-camera error compensation comprising:

recording a plurality of raw images via a plurality of digital cameras;

consolidating the plurality of raw images into a plural image frame, wherein the plural image frame is a composite image based on blending at least a portion of the plurality of raw images into the composite image;

processing, via a processor executing instructions for a multi-view stereo imaging system, the plural image frame from the raw images captured by the plurality of digital cameras;

storing the plural image frame in a memory; and

detecting calibration loss of at least one digital camera via the processor executing instructions for a multi-camera error compensating system via assessment of the plural image frame,

conducting dynamic recalibration of plural image calibration parameters based on at least one plural image frame and in response to detection of calibration loss via the multi-camera error compensating system; and

the processor reprocessing the plural image frame from a reprocessing queue according to recalibrated plural image parameters.

2. The method of claim 1 further comprising:

identifying the plural image frame as decalibrated.

3. The method of claim 2 wherein the plural image frame identified as decalibrated is added to the reprocessing queue.

4. The method of claim 1 further comprising:

determining whether any user action has been performed to modify or consume the stored plural image.

5. The method of claim 4 further comprising:

adding the stored plural image frame to the reprocessing queue if no user action has been performed to modify or consume the stored plural image.

6. The method of claim 4 wherein the user action performed to modify the stored plural image includes an editing function performed on the stored plural image frame that is more than merely viewing the plural image frame.

7. The method of claim 4 further comprising:

exempting the stored plural image frame from the reprocessing queue if the plural image frame has been modified by user action; and

warning a user of decalibration upon a user accessing the decalibrated plural image frame.

8. An information handling system comprising:

a plurality of digital cameras for recording raw images for consolidation into a plural image frame, wherein the plural image frame is a composite image based on blending at least a portion of a plurality of raw images consolidated to form the plural image frame;

a processor operatively coupled to the plurality of digital cameras for processing plural imaging functions via execution of instructions for a multi-view stereo imaging system;

a memory operatively coupled to the processor, the memory for storing the plural image frame;

the processor executing instructions for a multi-camera error compensating system to conduct detection of calibration loss of at least one digital camera via assessment of the plural image frame;

the multi-camera error compensating system conducting dynamic recalibration of plural image calibration parameters based on at least one plural image frame and in response to detection of calibration loss; and

the processor identifying the plural image frame as decalibrated, determining if any user action has been performed to modify the plural image frame, and storing an indication of image modification for assessment of reprocessing eligibility of the plural image frame.

9. The information handling system of claim 8 wherein the processor operatively coupled to the plurality of digital cameras for processing plural imaging functions includes three-dimensional (3D) imaging by determination of depth values for parts of the plural image frame.

10. The information handling system of claim 8 wherein the plural image calibration parameters dynamically recalibrated includes depth distance for objects recorded in the plural image frame.

11. The information handling system of claim 8 further comprising:

the multi-camera error compensating system reprocessing the plural image frame according to recalibrated plural image parameters.

12. The information handling system of claim 11 further comprising:

the multi-camera error compensating system reprocessing one or more decalibrated plural image frames stored in memory by re-determining depth values in the plural image frame.

13. The information handling system of claim 8 wherein the multi-camera error compensating system flags plural image frames identified as decalibrated and stores the decalibrated flag designations in memory.

14. The information handling system of claim 8 further comprising:

the multi-camera error compensating system warning a user of decalibration upon a user accessing decalibrated plural image frame that has been modified by user action.

15. A method for multi-camera error compensation comprising:

recording a plurality of raw images via a plurality of digital cameras;

consolidating the plurality of raw images into a plural image frame, wherein the plural image frame is a composite image based on at least a portion of the plurality of raw images;

processing, via a processor executing instructions for a multi-view stereo imaging system, the plural image frame from the raw images captured by the plurality of digital cameras;

storing the plural image frame in a memory;

detecting calibration loss of at least one digital camera via the processor executing instructions for a multi-camera error compensating system via assessing the plural image frame;

executing dynamic recalibration of plural image calibration parameters stored in memory based on at least one plural image frame and in response to detection of calibration loss; and

identifying the plural image frame as decalibrated, determining if any user action has been performed to modify the plural image frame, and storing an indication of image modification for assessment of reprocessing eligibility of the plural image frame.

16. The method of claim 15 wherein the processing plural image frame from the raw images captured by the plurality of digital cameras includes determination of a depth values for three-dimensional (3D) image processing of the plural image frame.

17. The method of claim 15 further comprising:

flagging one or more plural image frames as decalibrated and storing the decalibration flag designation in memory.

18. The method of claim 15 further comprising:

identifying one or more plural image frames as decalibrated; and

if the decalibrated plural image frame that has been modified by user action, warning a user of decalibration upon accessing the decalibrated plural image frame.

19. The method of claim 15 further comprising:

reprocessing one or more decalibrated plural image frames in memory by re-determining depth values in the plural image frame according to recalibrated plural image parameters.

20. The method of claim 15 further comprising:

identifying a plurality of stored plural image frames as decalibrated; and

recursively reprocessing each of the plurality of identified plural image frames identified as decalibrated according to recalibrated plural image calibration parameters if no user action has been performed to modify the plural image frame identified as decalibrated.

Assignments (15)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
RELEASE OF REEL 037160 FRAME 0142 (NOTE) Recorded Sep 14, 2016
From: BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
To: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040027/0812 →
RELEASE OF REEL 037160 FRAME 0239 (TL) Recorded Sep 14, 2016
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040028/0115 →
RELEASE OF REEL 037160 FRAME 0171 (ABL) Recorded Sep 13, 2016
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040017/0253 →
SUPPLEMENTAL PATENT SECURITY AGREEMENT - NOTES Recorded Nov 25, 2015
From: DELL PRODUCTS L.P.; DELL SOFTWARE INC.; BOOMI, INC.; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS FIRST LIEN COLLATERAL AGENT
Reel/Frame 037160/0142 →
SUPPLEMENTAL PATENT SECURITY AGREEMENT - TERM LOAN Recorded Nov 25, 2015
From: DELL PRODUCTS L.P.; DELL SOFTWARE INC.; BOOMI, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 037160/0239 →
SUPPLEMENTAL PATENT SECURITY AGREEMENT - ABL Recorded Nov 25, 2015
From: DELL PRODUCTS L.P.; DELL SOFTWARE INC.; BOOMI, INC.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 037160/0171 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 3, 2015
From: AURONGZEB, DEEDER M.; ANDREWS, MAXWELL
To: DELL PRODUCTS, LP
Reel/Frame 036238/0101 →
Continuity (1)
Related Publication 20170034502A1 · Feb 2, 2017
Cited By (1)
US 12,574,488