Door and window detection in an AR environment
Various embodiments described herein relate to a method, apparatus, and a non-transitory machine-readable storage medium including one or more of the following: locating a 2D plane correlated with a 3D mesh representing a surface of a room; taking a virtual 2D picture of a 3D mesh along the 2D segment; within the virtual 2D picture, finding a hole; determining the virtual picture floor, and hole width; when the hole intersects the virtual picture floor and is at least as wide as a door width then classifying the hole as a door; else classifying the hole as a window.
1. A method performed by a processor for locating features within a 3D mesh, the method comprising:
locating, within a memory associated with the processor, a 2D segment correlated with a 3D mesh representing a surface of a room;
determining a virtual picture bottom and a virtual picture top of the 3D mesh;
using the virtual picture bottom and virtual picture top, taking a virtual 2D picture of the 3D mesh along the 2D segment;
within the virtual 2D picture:
a) finding a hole,
b) determining a width of the hole, and
c) when the hole intersects the virtual picture bottom and when the hole is at least as wide as a predetermined door width then determine a structural feature for the hole.
2. The method of claim 1 , wherein when the hole does not intersect the virtual picture bottom or when the hole is not at least as wide as the predetermined door width, then classifying the hole as a window.
3. The method of claim 1 , wherein the 3D mesh is created using 3D depth scanner and a 2D camera working in tandem.
4. The method of claim 1 , further comprising determining whether the hole is not an artifact.
5. The method of claim 4 , wherein determining when the hole is not an artifact comprises:
finding a location of the hole within the 3D mesh;
offsetting the location of the hole within the 3D mesh, creating a hole offset;
casting rays from a middle of the hole offset into the 3D mesh; and
when the cast rays have hit a back side of more than one face then determining that the hole is not an artifact.
6. The method of claim 1 , wherein a surface of a room comprises a floor, a ceiling, or a wall.
7. The method of claim 1 , wherein taking a virtual 2D picture of the 3D mesh along the 2D segment comprises taking a virtual picture with a y axis running from the virtual picture bottom to the virtual picture top.
8. The method of claim 7 , further comprising: locating faces in the 3D mesh whose normal vector is opposite to gravity, recording area of the faces in the 3D mesh whose normal vector is opposite to gravity as an up area, locating faces in the 3D mesh whose normal vector is aligned with gravity, and recording area of the faces in the 3D mesh whose normal vector is aligned with gravity as a down area.
9. The method of claim 8 , further comprising creating a histogram of the up area at different heights, and creating a histogram of the down area at different heights.
10. The method of claim 8 , further comprising finding height of a tallest histogram in the down area and declaring it as a determined floor.
11. The method of claim 1 , wherein determining a virtual picture bottom and a virtual picture top of the 3D mesh comprises finding a minimum value of the room in Z as the virtual picture bottom and finding a maximum value of the 3D mesh in Z as the virtual picture top. further comprising finding faces pointing up, recording area of the faces pointing up, finding faces pointing down, and recording area of the faces pointing down.
12. A system for locating features within a 3D mesh, the system comprising:
a processor in communication with a memory storing a 3D mesh, the processor being configured to:
locate, within the memory, a 2D segment correlated with a 3D mesh representing a surface of a room;
determine a virtual picture bottom and a virtual picture top of the 3D mesh;
use the virtual picture bottom and virtual picture top, taking a virtual 2D picture of the 3D mesh along the 2D segment;
take a virtual 2D picture of the 3D mesh along the 2D segment;
within the virtual 2D picture, finding a hole;
determine hole width; and
when the hole intersects the virtual picture bottom and when the hole width is greater or equal to a door width then classifying the hole as a door.
13. The system of claim 12 , further comprising determining when the hole is not an artifact.
14. The system of claim 13 , wherein determining when the hole is not an artifact comprises:
finding a location of the hole within the 3D mesh;
offsetting the location of the hole within the 3D mesh, creating a hole offset;
casting rays from a middle of the hole offset into the 3D mesh; and
when the cast rays have hit a back side of more than one face then determining that the hole is not an artifact.
15. The system of claim 14 , wherein casting rays comprises casting six rays.
16. The system of claim 15 , wherein the six rays are cast in six cardinal directions.
17. A non-transitory machine-readable storage medium encoded with instructions for execution by a processor for locating features within a 3D mesh, the non-transitory machine-readable storage medium comprising:
instructions for locating, by the processor, within a memory associated with the processor, a 2D segment correlated with a 3D mesh representing a surface of a room;
instructions for determining a virtual picture bottom and a virtual picture top of the 3D mesh;
instructions for using the virtual picture bottom and virtual picture top to take a virtual 2D picture of the 3D mesh along the 2D segment;
instructions for, within the virtual 2D picture, finding a hole;
instructions for determining hole width, and
instructions for, when the hole intersects the vertical virtual picture bottom and when the hole is at least as wide as a door width then classifying the hole as a door.
18. The non-transitory machine-readable storage medium of claim 17 further comprising determining when the hole is not an artifact.
19. The non-transitory machine-readable storage medium of claim 18 wherein determining when the hole is not an artifact comprises:
instructions for finding a location of the hole within the 3D mesh;
instructions for offsetting the location of the hole within the 3D mesh, creating a hole offset;
instructions for casting rays from a middle of the hole offset into the 3D mesh;
instructions for, when the cast rays have hit a back side of one or fewer faces then determining that the hole is not an artifact; and
instructions for, when the cat rays have hit the back side of more than one face then determining that the hole is an artifact.
20. The non-transitory machine-readable storage medium of claim 19 further comprising instructions for classifying the hole as a window when the hole does not intersect the virtual picture bottom or when the hole is not at least as wide as the door width.