Uncertainty display for a multi-dimensional mesh
In various example embodiments, techniques are provided for representing uncertainty when displaying a rendered view of a multi-dimensional mesh (e.g., created by SfM photogrammetry) in a user interface by applying a real-time, obfuscation filter that modifies the rendered view based on uncertainty in screen space. Where the multi-dimensional mesh is within a limit of data accuracy, the rendered view is shown without modification (i.e. as normal), and a user may trust the information displayed. Where the multi-dimensional mesh is beyond the limit of data accuracy, the obfuscation filter obfuscates detail (e.g., by blurring, pixilating, edge enforcing, etc.) in the rendered view so that a user may visually perceive the uncertainty. The amount of obfuscation may be weighted based on uncertainty to allow the user to visually quantify uncertainty.
1. A method for representing uncertainty when displaying a multi-dimensional mesh, comprising:
rendering, by an application executing on a computing device, a view of the multi-dimensional mesh;
determining uncertainty corresponding to the rendered view by
obtaining uncertainty in model space for each point of the multi-dimensional mesh visible in the rendered view, and
based on the uncertainty in model space, computing uncertainty in screen space for each point of the multi-dimensional mesh visible in the rendered view;
applying an obfuscation filter to the rendered view that modifies the rendered view based on the uncertainty in screen space for each point to obfuscate detail in at least a portion of the rendered view to provide a visual indication of uncertainty; and
displaying, in a user interface of the application, the obfuscation-filtered rendered view.
2. The method of claim 1 , wherein the applying the obfuscation filter further comprises:
where uncertainty in screen space is within a limit of data accuracy, showing the rendered view without modification; and
where uncertainty in screen space is beyond the limit of data accuracy, obfuscating detail in the rendered view.
3. The method of claim 2 , wherein the obfuscating detail comprises at least one of blurring, pixilating or edge enforcing the rendered view.
4. The method of claim 2 , wherein an amount of the obfuscating detail is weighted based on uncertainty in screen space to visually quantify uncertainty.
5. The method of claim 1 , wherein the uncertainty in model space for each point forms a sphere about the point, and the computing uncertainty in screen space for each point comprises:
projecting the sphere on a screen to form a region of pixels; and
determining a bounding circle for the region having a diameter that serves as the uncertainty in screen space.
6. The method of claim 1 , wherein the uncertainty in model space for each point is a numeric value in a unit of distance corresponding to a maximal deviation from a point.
7. The method of claim 1 , wherein the rendered view is an original frame having a plurality of pixels and the computing uncertainty in screen space for each point produces an uncertainty image having same dimensions as the original frame that provides an uncertainty in screen space for each pixel in the original frame.
8. The method of claim 7 , wherein the applying the obfuscation filter further comprises:
providing at least the original frame and the uncertainty image as inputs to the obfuscation filter; and
producing an obfuscated frame by the obfuscation filter as an output,
wherein the obfuscated frame is used as the obfuscation-filtered rendered view.
9. The method of claim 1 , further comprising:
generating the multi-dimensional mesh by a structure from motion (SfM) photogrammetry application based on a set of images or point clouds of the physical environment captured by one or more cameras or scanners.
10. A computing device comprising:
a display screen;
a processor; and
a memory coupled to the processor and configured to store a multi-dimensional mesh and an application, the application when executed operable to:
render a view of the multi-dimensional mesh;
determine uncertainty corresponding to the rendered view by
obtaining uncertainty in model space for each point of the multi-dimensional mesh visible in the rendered view, and
based on uncertainty in model space, computing uncertainty in screen space for each point of the multi-dimensional mesh visible in the rendered view;
where uncertainty in screen space is within a limit of data accuracy, display the rendered view on the display screen without modification; and
where uncertainty in screen space is beyond the limit of data accuracy, obfuscate detail in the rendered view to provide a visual indication of uncertainty and display the obfuscated detail.
11. The computing device of claim 10 , wherein the obfuscating detail comprises at least one of blurring, pixilating or edge enforcing the rendered view.
12. The computing device of claim 10 , wherein an amount of the obfuscating detail is weighted based on uncertainty in screen space to visually quantify uncertainty.
13. The computing device of claim 10 , wherein the multi-dimensional mesh is a structure from motion (SfM) photogrammetry-generated mesh based on a set of images or point clouds of the physical environment.
14. A non-transitory electronic-device readable medium having instructions stored thereon, the instructions when executed by one or more electronic devices operable to:
render a view of a scene;
compute uncertainty in screen space for each point of the scene visible in the rendered view;
apply an obfuscation filter to the rendered view that modifies the rendered view based on the uncertainty in screen space for each point, the obfuscation filter to show the rendered view without modification where uncertainty in screen space is within a limit of data accuracy and to obfuscate detail in the rendered view where uncertainty in screen space is beyond the limit of data accuracy to provide a visual indication of uncertainty; and
display the obfuscation-filtered rendered view.
15. The non-transitory electronic-device readable medium of claim 14 , wherein the instructions when executed are further operable to:
obtain uncertainty in model space for each point visible in the rendered view,
wherein the computation of uncertainty in screen space for each point is based on the uncertainty in model space.
16. The non-transitory electronic-device readable medium of claim 15 , wherein the uncertainty in model space for each point is a numeric value in a unit of distance corresponding to a maximal deviation from a point which forms a sphere about the point, and the instructions when executed are further operable to:
for each point, project the sphere on a screen to form a region of pixels; and
determine a bounding circle for the region having a diameter that serves as the uncertainty in screen space.
17. The non-transitory electronic-device readable medium of claim 14 , wherein the rendered view is an original frame having a plurality of pixels and the computation of uncertainty in screen space for each point produces an uncertainty image having same dimensions as the original frame that provides an uncertainty in screen space for each pixel in the original frame.
18. The non-transitory electronic-device readable medium of claim 17 , wherein the instructions when executed are further operable to:
provide at least the original frame and the uncertainty image as inputs to the obfuscation filter; and
produce an obfuscated frame by the obfuscation filter as an output,
wherein the obfuscated frame is used as the obfuscation-filtered rendered view.
19. The non-transitory electronic-device readable medium of claim 14 , wherein the scene is a three-dimensional mesh.
20. The non-transitory electronic-device readable medium of claim 19 , wherein the instructions when executed are further operable to:
generate the three-dimensional mesh by a structure from motion (SfM) photogrammetry application based on a set of images or point clouds of the physical environment captured by one or more cameras or scanners.
21. A method for representing uncertainty when displaying a multi-dimensional mesh, comprising:
rendering, by an application executing on a computing device, a view of the multi-dimensional mesh;
computing uncertainty in screen space for each point of the scene visible in the rendered view;
applying an obfuscation filter to the rendered view that modifies the rendered view based on the uncertainty in screen space for each point, the obfuscation filter to show the rendered view without modification where uncertainty in screen space is within a limit of data accuracy and to at least one of blur, pixilate or edge enforce the rendered view where uncertainty in screen space is beyond the limit of data accuracy; and
displaying, in a user interface of the application, the obfuscation-filtered rendered view.
22. The method of claim 21 , wherein an amount of blurring, pixilating or edge enforcing imposed by the obfuscation filter is weighted based on uncertainty in screen space to visually quantify uncertainty.
23. The method of claim 21 , further comprising:
obtaining uncertainty in model space for each point visible in the rendered view,
wherein the computing uncertainty in screen space for each point is based on the uncertainty in model space.