IP Library Granted Patent US 10,074,182
Granted Patent B2
US 10,074,182 · App. 14/223,614 · Granted Sep 11, 2018

Presenting markup in a scene using depth fading

Inventors: Nikolai Faaland (Sammamish, WA); David Buerer (Woodinville, WA)
Assignee: Microsoft Technology Licensing, LLC
G06T7/0051G01C21/3635G06T15/00G06T17/05G06T19/00G01C21/3638G06K9/00671G06T2210/62G06T2219/004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,074,182
App. No.
14/223,614
Granted
Sep 11, 2018
Kind
B2
Abstract

Architecture that enables the drawing of markup in a scene that neither obscures the scene nor is undesirably obscured by the scene. When drawing markup such as text, lines, and other graphics, into the scene, a determination is made as to the utility to the viewer of drawing the markup with greater prominence than an occluding scene object. The utility of the markup is based on the distance of the scene object and markup from the camera. Thus, if an object that appears small in the scene and is in front of the markup, the markup will be drawn more clearly, whereas if the same object appears large in the scene and is in front of the markup, the markup is rendered faint, if drawn at all.

Claims (36)

1. A system, comprising:

a processor; and

a memory for storing instructions that, when executed by the processor, causes the processor to:

compute, for a given view of a scene, a dominant contribution between a markup of the scene and a scene object of the scene based on a determined markup contribution amount and a scene contribution amount, wherein the markup contribution amount and the scene contribution amount are computed to determine a utility of the markup when compared to the scene object, and wherein the utility is based, at least in part, on:

a view of a virtual camera associated with the view of the scene; and

on a ratio of a distance from an occluded pixel of the markup to the virtual camera and a distance from a corresponding occluding pixel of the scene object to the virtual camera; and

apply a level of opacity to the markup based on the determined utility relative to a location of the scene object of the given view when the markup and the scene object are moved to be more distant from the virtual camera and the markup is located behind the scene object with respect to the virtual camera and wherein the level of opacity causes the markup to be more visible than the scene object due to changes of the ratio.

2. The system of claim 1 , wherein the markup contribution amount and the scene contribution amount are computed on a per pixel basis and the level of opacity is applied on a per pixel basis.

3. The system of claim 1 , wherein the scene is a three-dimensional (3D) scene.

4. The system of claim 1 , wherein the processor is further configured to compute an amount of scene matter, as the scene contribution amount, that blocks an observer from viewing the markup.

5. The system of claim 1 , wherein the markup is drawn with greater prominence than the scene object, which is in front of the markup, when the markup contribution amount indicates greater utility of the markup to the scene than the scene object, the prominence being based on an applied level of opacity.

6. The system of claim 1 , wherein the markup is drawn with less prominence than the scene object, which is in front of the markup, when the scene contribution amount indicates greater utility of the scene object to the scene than the markup, the prominence being based on an applied level of opacity.

7. The system of claim 1 , wherein the scene is a three-dimensional scene.

8. The system of claim 1 , wherein moving the markup and the scene object to be more distant from the virtual camera comprises performing a zoom operation.

9. A method for rendering objects on a display, comprising:

receiving a scene with a markup located in the scene and according to a view of a virtual camera, the view having a scene object that occludes at least a portion of the markup;

computing a utility of the markup and a utility of the scene object to the view based, at least in part, on a markup contribution amount of the markup and a scene object contribution amount of the scene object, wherein the utility of the markup and the utility of the scene object are computed based, at least in part, on a ratio of a distance from an occluded pixel of the markup to the virtual camera and a distance from a corresponding occluding pixel of the scene object to the virtual camera;

comparing the utility of the markup and the utility of the scene object; and

when it is determined that the utility of the markup is greater than the utility of the scene object, applying a level of opacity to the markup when the markup and the scene object are moved to be more distant from the virtual camera and the markup is located behind the scene object with respect to the virtual camera and wherein the level of opacity causes the markup to be more visible than the scene object due to changes of the ratio.

10. The method of claim 9 , further comprising drawing the markup with prominence and readability, if the scene object is in front of the markup and the utility of the markup is greater than the utility of the scene object in the scene.

11. The method of claim 9 , further comprising drawing the markup with less prominence than the scene object, if the scene object is in front of the markup and the utility of the scene object is greater than the utility of the markup in the scene.

12. The method of claim 9 , further comprising computing the markup contribution amount to the scene and the scene object contribution amount to the scene on a per pixel basis.

13. The method of claim 9 , further comprising adjusting the levels of opacity applied to the markup, the scene object, or the markup and the scene object based on a zoom level of the view.

14. The method of claim 9 , wherein the scene is a three-dimensional scene.

15. The method of claim 9 , wherein moving the markup and the scene object to be more distant from the virtual camera comprises performing a zoom operation.

16. A system, comprising:

at least one processor; and

a memory coupled to the at least one processor and storing instructions that, when executed by the at least one processor, perform a method, comprising:

identifying a location of a markup in a three-dimensional scene relative to an occluding scene object and according to a view of the scene from a virtual camera;

computing an amount of the scene object that at least partially blocks the markup based, at least in part, on a distance between the markup and the virtual camera;

determining a utility of the markup with respect to a utility of the scene object based on the distance, wherein the utility of the markup and the utility of the scene object are computed based, at least in part, on a ratio of a distance from an occluded pixel of the markup to the virtual camera and a distance from a corresponding occluding pixel of the scene object to the virtual camera; and

applying a level of opacity to at least one of the markup and the scene object based, at least in part on the determined utility of the markup and the determined utility of the scene markup wherein the level of opacity is applied to the markup when the markup and the scene object are moved to be more distant from the virtual camera and the markup is located behind the scene object with respect to the virtual camera and wherein the level of opacity causes the markup to be more visible than the scene object due to changes of the ratio.

17. The system of claim 16 , further comprising instructions for changing the level of opacity applied based, at least in part, on a change in the view and at least one of the utility of the markup and the utility of the scene markup.

18. The system of claim 16 , further comprising instructions for applying a level of opacity equivalent to full opacity to the markup when the utility of the markup is greater than the utility of the scene object.

19. The system of claim 16 , further comprising instructions for applying a level of opacity to zero opacity to the markup when the utility of the scene object is greater than the utility of the markup.

20. The system of claim 16 , wherein determining the utility of the markup with respect to the utility of the occluding scene object comprises determining the utility of the markup with respect to the utility of the occluding scene object on a per pixel basis.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2015
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 039025/0454 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 25, 2014
From: FAALAND, NIKOLAI; BUERER, DAVID
To: MICROSOFT CORPORATION
Reel/Frame 032512/0852 →
Continuity (2)
Provisional Application 61904438 · Nov 14, 2013
Related Publication 20150130791A1 · May 14, 2015
Cited By (1)
US 12,294,685