IP Library Granted Patent US 11,935,183
Granted Patent B2
US 11,935,183 · App. 17/757,354 · Granted Mar 19, 2024

Occlusion solution within a mixed reality design software application

Inventors: Robert William Blodgett (Salt Lake City, UT); Joseph S. Howell (Uintah, UT)
Assignees: DIRTT ENVIRONMENTAL SOLUTIONS LTD.; ARMSTRONG WORLD INDUSTRIES, INC.
G06T15/40G06F30/13G06T7/80G06T19/006G06V10/26G06V20/20G06V20/64G06T2210/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,935,183
App. No.
17/757,354
Granted
Mar 19, 2024
Kind
B2
Abstract

A method for incorporating a real object at varying depths within a rendered three-dimensional architectural design space can include capturing data from a real environment, wherein the real environment comprises at least one real object within a physical architectural space. The method can also comprise extracting the at least one real object from the captured data from the real environment. Further, the method can include providing a rendered three-dimensional architectural design space comprising at least one virtual architectural component. The method can also include projecting the captured data from the real environment on a first plane within the rendered three-dimensional architectural design space and projecting the extracted at least one real object on a at least one additional plane within the rendered three-dimensional architectural design space, such that the rendered at least one real object is properly occluded within the rendered three-dimensional architectural design space.

Claims (43)

1. A computer system for incorporating a real object at varying depths within a rendered three-dimensional architectural design space, comprising:

one or more processors; and

one or more computer-readable media having stored thereon executable instructions that when executed by the one or more processors configure the computer system to perform at least the following:

receive captured data from a real environment, wherein the real environment comprises at least one real object within a physical architectural space;

extract the at least one real object from the captured data from the real environment;

provide a rendered three-dimensional architectural design space comprising one or more virtual architectural components, wherein each virtual architectural component is designed to coordinate with the physical architectural space;

project the captured data from the real environment on a first plane within the rendered three-dimensional architectural design space; and

project the extracted at least one real object on at least one additional plane within the rendered three-dimensional architectural design space, the at least one additional plane being positioned such that the projected at least one real object is properly occluded within the rendered three-dimensional architectural design space.

2. The computer system of claim 1 , wherein extracting the at least one real object from the captured data from the real environment comprises subtraction of an image from a background image comprising the physical architectural space to generate a mask representative of the real object.

3. The computer system of claim 2 , wherein extracting the at least one real object from the captured data from the real environment further comprises separating the mask into a plurality of probability fields.

4. The computer system of claim 1 , wherein a position of the at least one additional plane within the rendered three-dimensional architectural design space is dependent on a relative position of the at least one real object within the real environment.

5. The computer system of claim 4 , wherein the relative position of the at least one real object within the real environment is captured by a depth camera.

6. The computer system of claim 4 , wherein the relative position of the at least one real object within the real environment is captured by one or more motion tracking sensors.

7. The computer system of claim 4 , wherein the position of the at least one additional plane within the rendered three-dimensional architectural design space is determined by the relative position of the at least one real object within the real environment in real time.

8. The computer system of claim 1 , wherein the executable instructions are executable to configure the computer system to calibrate at least one real camera and at least one virtual camera such that they are positioned and oriented as data from the real environment is captured.

9. The computer system of claim 1 , wherein the executable instructions are executable to configure the computer system to:

receive input to adjust at least one dimensional parameter of at least one of the one or more virtual architectural components; and

update the adjusted at least one of the one or more virtual architectural components within the rendered three-dimensional architectural design space while continuing to project the captured data from the real environment on the first plane and the at least one real object on the at least one additional plane.

10. The computer system of claim 1 , wherein the rendered three-dimensional architectural design space corresponds in dimensions to the physical architectural space, and wherein each of the one or more virtual architectural components comprises a CAD design file that is mapped to a particular location within the physical architectural space when rendered within the rendered three-dimensional architectural design space.

11. The computer system of claim 1 , wherein at least one of the one or more virtual architectural components comprises a CAD design file corresponding to the at least one real object, such that the at least one of the one or more virtual architectural objects matches the at least one real object.

12. The computer system of claim 1 , wherein the rendered three-dimensional architectural design space is generated from the captured data from the real environment and one or more CAD design files corresponding to the one or more virtual architectural objects.

13. A computer-implemented method for incorporating a real object at varying depths within a rendered three-dimensional architectural design space, the method executed on one or more computer processors, the method comprising:

receiving captured data from a real environment, wherein the real environment comprises at least one real object within a physical architectural space;

extracting the at least one real object from the captured data from the real environment;

providing a rendered three-dimensional architectural design space comprising one or more virtual architectural components, wherein each virtual architectural component is designed to coordinate with the physical architectural space;

projecting the captured data from the real environment on a first plane within the rendered three-dimensional architectural design space; and

projecting the extracted at least one real object on at least one additional plane within the rendered three-dimensional architectural design space, the at least one additional plane being positioned such that the projected at least one real object is properly occluded within the rendered three-dimensional architectural design space.

14. The computer-implemented method of claim 13 , further comprising:

receiving input to adjust at least one dimensional parameter of at least one of the one or more virtual architectural components; and

updating the adjusted at least one of the one or more virtual architectural components within the rendered three-dimensional architectural design space while continuing to project the captured data from the real environment on the first plane and the at least one real object on the at least one additional plane.

15. The computer-implemented method of claim 13 , wherein the rendered three- dimensional architectural design space corresponds in dimensions to the physical architectural space, and wherein each of the one or more virtual architectural components comprises a CAD design file that is mapped to a particular location within the physical architectural space when rendered within the rendered three-dimensional architectural design space.

16. The computer-implemented method of claim 13 , wherein at least one of the one or more virtual architectural components comprises a CAD design file corresponding to the at least one real object, such that the at least one of the one or more virtual architectural objects matches the at least one real object.

17. A non-transitory computer-readable storage medium comprising one or more physical computer-readable storage media having stored thereon computer-executable instructions that, when executed at a processor, cause a computer system to perform a method for incorporating a real object at varying depths within a rendered three-dimensional architectural design space, the method comprising:

capturing, with one or more sensors, data from a real environment, wherein the real environment comprises at least one real object within a physical architectural space;

extracting the at least one real object from the captured data from the real environment, wherein extraction of the at least one real object comprises separating data related to the at least one real object from data related to the physical architectural space;

providing a rendered three-dimensional architectural design space comprising one or more virtual architectural components, wherein each virtual architectural component is designed to coordinate with the physical architectural space;

projecting the captured data from the real environment on a first plane within the rendered three-dimensional architectural design space; and

projecting the extracted at least one real object on at least one additional plane within the rendered three-dimensional architectural design space, the at least one additional plane being positioned such that the projected at least one real object is properly occluded within the rendered three-dimensional architectural design space.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the executable instructions are executable to configure the computer system to perform the method for incorporating a real object at varying depths within a rendered three-dimensional architectural design space, the method comprising:

receiving input to adjust at least one dimensional parameter of at least one of the one or more virtual architectural components; and

updating the adjusted at least one of the one or more virtual architectural components within the rendered three-dimensional architectural design space while continuing to project the captured data from the real environment on the first plane and the at least one real object on the at least one additional plane.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the rendered three-dimensional architectural design space corresponds in dimensions to the physical architectural space, and wherein each of the one or more virtual architectural components comprises a CAD design file that is mapped to a particular location within the physical architectural space when rendered within the rendered three-dimensional architectural design space.

20. The non-transitory computer-readable storage medium of claim 17 , wherein at least one of the one or more virtual architectural components comprises a CAD design file corresponding to the at least one real object, such that the at least one of the one or more virtual architectural objects matches the at least one real object.

Assignments (7)
DISCLAIMER OF INTEREST Recorded Jun 22, 2023
From: FIRTST AMERICAN COMMERCIAL BANCORP, INC.
To: ARMSTRONG WORD INDUSTRIES, INC.; DIRTT ENVIRONMENTAL SOLUTIONS LTD.
Reel/Frame 064065/0257 →
LETTER OF CONSENT TO ASSIGNMENT Recorded Jun 22, 2023
From: ROYAL BANK OF CANADA
To: ARMSTRONG WORLD INDUSTRIES, INC.; DIRTT ENVIRONMENTAL SOLUTIONS LTD.
Reel/Frame 064065/0386 →
NO INTEREST LETTER Recorded Jun 22, 2023
From: UPPER CANADA FOREST PRODUCTS LTD.
To: ARMSTRONG WORLD INDUSTRIES, INC.; DIRTT ENVIRONMENTAL SOLUTIONS LTD.
Reel/Frame 064065/0271 →
PARTIAL ASSIGNMENT Recorded Jun 20, 2023
From: DIRTT ENVIRONMENTAL SOLUTIONS LTD.
To: ARMSTRONG WORLD INDUSTRIES, INC.
Reel/Frame 064040/0503 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 17, 2022
From: DIRTT ENVIRONMENTAL SOLUTIONS INC.
To: DIRTT ENVIRONMENTAL SOLUTIONS LTD.
Reel/Frame 060234/0047 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 17, 2022
From: BLODGETT, ROBERT WILLIAM; HOWELL, JOSEPH S.
To: DIRTT ENVIRONMENTAL SOLUTIONS INC.
Reel/Frame 060234/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 15, 2022
From: DIRTT ENVIRONMENTAL SOLUTIONS INC.
To: DIRTT ENVIRONMENTAL SOLUTIONS LTD.
Reel/Frame 060216/0125 →
Continuity (2)
Provisional Application 62959321 · Jan 10, 2020
Related Publication 20230021117A1 · Jan 19, 2023