IP Library › Granted Patent US 12,067,799
Granted Patent B2
US 12,067,799 · App. 17/063,350 · Granted Aug 20, 2024

Camera environment mapping

Inventors: Justin Murray (Redmond, WA); Timothy Clifford (Bellevue, WA)
Assignee: Microsoft Technology Licensing, LLC
G06V40/103G06T7/543G06T7/73
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,067,799
App. No.
17/063,350
Granted
Aug 20, 2024
Kind
B2
Abstract

Systems and methods may be used to map an environment. A method may include receiving frames of video data from a camera. The method may include identifying an object in at least two of the frames to identify an axis of the object. The axis may be used to generate a plane map.

Claims (36)

1. A device for mapping an environment, the device comprising:

a processor; and

memory, communicatively coupled to the processor, the memory including instructions, which when executed, cause the processor to:

receive a set of frames of video data captured by a camera;

identify an object within the set of frames;

identify, based on a determined structure of the object in at least two frames of the set of frames, an axis of the object, the axis including an endpoint in contact with a surface of the environment;

and

generate, without using extrinsic properties of the camera, a plane map including a plane of a non-living surface of the environment that is captured in the at least two frames of the set of frames, the plane map generated based on, a prespecified height of the object, a first identified location of the endpoint in a first of the at least two frames of the set of frames and a second, different identified location of the endpoint in a second of the at least two frames of the set of frames, the generating of the plane map comprising generating a mathematical description of the plane.

2. The device of claim 1 , wherein the camera lacks any hardware-based depth sensing capabilities.

3. The device of claim 1 , wherein the axis is a line segment extending from a point at a first end of the object to a midpoint between two points at an opposite end of the object.

4. The device of claim 1 , wherein the camera is a single camera, and the plane map is generated based solely on data received from the single camera.

5. The device of claim 1 , wherein the plane map is generated based on an assumption that the axis of the object varies based on distance from the camera while a height of the object remains constant.

6. The device of claim 1 , wherein the plane map is generated based on an assumption that the axis is parallel to a gravity vector.

7. The device of claim 1 , wherein the instructions further cause the processor to determine an angle of orientation of the camera with respect to the plane map of the environment.

8. A method for mapping an environment, the method comprising:

receive, at a processor, at least two frames of video data captured by a camera;

identifying, using the processor, an object within the at least two frames;

identifying, based on a determined structure of the object in the at least two frames, an axis of the object, the axis including an endpoint in contact with a surface of the environment;

generate, using the processor and without using extrinsic properties of the camera, a plane map including a plane of a non-living surface of the environment that is captured in the at least two frames, the plane map generated based on, a prespecified height of the object, a first identified location of the endpoint in a first of the at least two frames and a second, different identified location of the endpoint in a second of the at least two frames, the generating of the plane map comprising generating a mathematical description of the plane; and

outputting the plane map.

9. The method of claim 8 , wherein the camera lacks any hardware-based depth sensing capabilities.

10. The method of claim 8 , wherein the axis is a line segment extending from a point at a first end of the object to a midpoint of two points at an opposite end of the object.

11. The method of claim 8 , wherein the camera is a single camera, and the plane map is generated based solely on data received from the single camera.

12. The method of claim 8 , wherein the plane map is generated based on an assumption that the axis of the object varies based on distance from the camera while a height of the object remains constant.

13. The method of claim 8 , wherein the plane map is generated based on an assumption that the axis is parallel to a gravity vector.

14. The method of claim 8 , further comprising determining an angle of orientation of the camera with respect to the plane map of the environment.

15. An apparatus for mapping an environment, the apparatus comprising:

means for receiving a set of frames of video data captured by a camera;

means for identifying an object within the set of frames;

means for identifying, based on a determined structure of the object in at least two frames of the set of frames, an axis of the object, the axis including an endpoint in contact with a surface of the environment;

means for generating, without using extrinsic properties of the camera, a plane map including a plane of a non-living surface of the environment that is captured in the at least two frames of the set of frames, the plane map generated based on, a prespecified height of the object, a first identified location of the endpoint in a first of the at least two frames of the set of frames and a second, different identified location of the endpoint in a second of the at least two frames of the set of frames, the generating of the plane map comprising generating a mathematical description of the plane; and

means for outputting the plane map for display.

16. The apparatus of claim 15 , wherein the camera is a single camera, and wherein the plane map is generated based solely on data received from the single camera.

17. The apparatus of claim 15 , wherein the camera lacks any hardware-based depth sensing capabilities.

18. The apparatus of claim 15 , wherein the axis is a line segment extending from a point at a first end of the object to a midpoint between two points at an opposite end of the object.

19. The apparatus of claim 15 , wherein the plane map is generated based on an assumption that the axis of the object varies based on distance from the camera while a height of the object remains constant.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2020
From: MURRAY, JUSTIN; CLIFFORD, TIMOTHY
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 054596/0718 →
Continuity (2)
Continuation 16244745 · Jan 10, 2019
Related Publication 20210019510A1 · Jan 21, 2021