IP Library Granted Patent US 10,139,985
Granted Patent B2
US 10,139,985 · App. 15/272,337 · Granted Nov 27, 2018

Defining, displaying and interacting with tags in a three-dimensional model

Inventors: James Mildrew (Mountain View, CA); Matthew Tschudy Bell (Palo Alto, CA); Dustin Michael Cook (San Jose, CA); Preston Cowley (San Jose, CA); Lester Lee (Oakland, CA); Peter McColgan (Fremont, CA); Daniel Prochazka (Pacifica, CA); Brian Schulman (San Francisco, CA); James Sundra (San Jose, CA); Alan Tan (San Jose, CA)
Assignee: Matterport, Inc.
G06F3/04815G06F3/0482G06F3/04817G06T15/20G06T17/05G06T19/003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,139,985
App. No.
15/272,337
Granted
Nov 27, 2018
Kind
B2
Abstract

This application generally relates to defining, displaying and interacting with tags in a 3D model. In an embodiment, a method includes generating, by a system including a processor, a three-dimensional model of an environment based on sets of aligned three-dimensional data captured from the environment, and associating tags with defined locations of the three-dimensional model, wherein the tags are respectively represented by tag icons that are spatially aligned with the defined locations of the three-dimensional model as included in different representations of the three-dimensional model rendered via an interface of a device, wherein the different representations correspond to different perspectives of the three-dimensional model, and wherein selection of the tag icons causes the tags respectively associated therewith to be rendered at the device.

Claims (54)

1. A method, comprising:

facilitating, by a device comprising a processor, navigation of a three-dimensional model of an environment as presented via an interface of the device, wherein the three-dimensional model was generated based on sets of aligned three-dimensional data captured from the environment, and wherein the three-dimensional model is associated with tags at defined points of the three-dimensional model;

rendering, by the device, different representations of the three-dimensional model from different perspectives of a virtual camera relative to the three-dimensional model based on received navigational input;

determining, by the device, a first subset of the tags associated with a higher probability of selection relative to a second subset of the tags; and

displaying, by the device, one or more tag icons in the different representations for one or more tags of the first subset of the tags based on the determining, wherein the displaying comprises displaying the one or more tag icons at spatially aligned positions in the different representations based on one or more defined points of the defined points that the one or more tags are respectively associated with, wherein the one or more tag icons are selectable and wherein selection of the one or more tag icons causes the one or more tags respectively associated therewith to be rendered by the device.

2. The method of claim 1 , further comprising:

receiving, by the device, the first subset of the tags from a remote device based on the determining.

3. The method of claim 2 , wherein the receiving comprises receiving respective tags included in the first subset of the tags in an order determined based on respective probabilities of selection associated with the respective tag.

4. The method of claim 1 , wherein the determining comprises determining the first subset of the tags based on machine learning analysis of historical navigation patterns associated with the three-dimensional model and the received navigational input.

5. The method of claim 1 , wherein the determining comprises determining the first subset of the tags based on one or more preferences of a user identity associated with the received navigational input.

6. The method of claim 1 , wherein the tags comprise a social tag that comprises aggregated information from other users regarding historical interaction of the other users with a tag icon or feedback provided by the other users regarding an object associated with the tag icon.

7. The method of claim 1 , further comprising:

determining, by the device, the spatially aligned positions based on the one or more defined points that the one or more tags are respectively associated with and relative positions of objects as included in the different representations such that the objects do not occlude the one or more tag icons.

8. The method of claim 1 , wherein appearances of tag icons respectively associated with the tags vary based on respective types of the tags, and wherein the respective types of the tags vary.

9. The method of claim 1 , wherein the first subset of the tags comprises a group of related tags and wherein interaction with a tag icon associated with one tag included in the group initiates a response associated with other tags included in the group.

10. The method of claim 1 , wherein a tag of the one or more tags comprises information regarding an object that the tag is associated with, and wherein the information identifies one or more other tagged objects that are related to the object and included in the three-dimensional model.

11. The method of claim 10 , wherein the tag further comprises one or more hyperlinks to one or more representations of the three-dimensional model that include the one or more other tagged objects.

12. The method of claim 1 , wherein the tags comprise data selected from the group consisting of: text data, audio data, image data, three-dimensional object data, video data, hyperlink data, and executable application data.

13. The method of claim 1 , wherein the one or more tags comprise a hyperlink tag associated with a first location in the three-dimensional model that provides a hyperlink to a representation associated with a second location in the three-dimensional model or another three-dimensional mode.

14. The method of claim 1 , wherein the one or more tag icons comprise two-dimensional or three-dimensional iconography and wherein the displaying comprises displaying the one or more tag icons such that an orientation of the one or more tag icons is the same in the different representations.

15. A device, comprising,

a memory that stores computer executable components;

a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:

a navigation component configured to receive navigation input indicating desired perspectives for viewing a three-dimensional model presented via an interface of the device, wherein the three-dimensional model is associated with tags at defined points of the three-dimensional model;

a tag component configured to determine a first subset of the tags associated with a higher probability of selection relative to a second subset of the tags; and

a rendering component configured to render, via the interface, different representations of the three-dimensional model from different perspectives of a virtual camera relative to the three-dimensional model determined based on the navigation input,

wherein the rendering component is configured to include one or more tag icons for one or more tags of the first subset of the tags in one or more representations of the different representations at spatially aligned positions in the one or more representations, wherein the spatially aligned positions are based on one or more defined points of the defined points that the one or more tags are respectively associated with, wherein the one or more tag icons are selectable, and wherein selection of the one or more tag icons causes the one or more tags respectively associated therewith to be rendered by the device.

16. The device of claim 15 , wherein the computer executable components further comprise:

a reception component configured to receive the first subset of the tags from a remote device based on determining of the first subset of the tags by the tag component.

17. The device of claim 16 , wherein the reception component is configured to receive respective tags included in the first subset of the tags in an order determined based on respective probabilities of selection associated with the respective tags.

18. The device of claim 15 , wherein the tag component is configured to determine the first subset of the tags based on the navigation input.

19. The device of claim 15 , wherein the tag component is further configured to determine the first subset of the tags based on machine learning analysis of historical navigation patterns associated with the three-dimensional mode.

20. The device of claim 19 , wherein the tag component is further configured to determine the first subset of the tags based on one or more preferences of a user identity associated with the navigation input.

21. The device of claim 15 , wherein the tags comprise a social tag that comprises aggregated information from other users regarding historical interaction of the other users with a tag icon or feedback provided by the other users regarding an object associated with the tag icon.

22. The device of claim 15 , wherein the rendering component is further configured to determine the spatially aligned positions based on the one or more defined points that the one or more tags are respectively associated with and relative positions of objects as included in the one or more representations such that the objects do not occlude the one or more tag icons.

23. The device of claim 22 , wherein appearances of tag icons respectively associated with the tags vary based on respective types of the tags, and wherein the respective types of the tags vary.

24. The device of claim 15 , wherein the first subset of the tags comprises a group of related tags and wherein interaction with a tag icon associated with one tag included in the group initiates a response associated with other tags included in the group.

25. The device of claim 15 , wherein a tag of the one or more tags comprises information regarding an object that the tag is associated with, and wherein the information identifies one or more other tagged objects that are related to the object and included in the three-dimensional model.

26. The device of claim 25 , wherein the tag further comprises one or more hyperlinks to one or more representations of the three-dimensional model that include the one or more other tagged objects.

27. The device of claim 15 , wherein the tags comprise data selected from the group consisting of: text data, audio data, image data, three-dimensional object data, video data, hyperlink data, and executable application data.

28. The device of claim 15 , wherein the one or more tags comprise a hyperlink tag associated with a first location in the three-dimensional model that provides a hyperlink to a representation associated with a second location in the three-dimensional model or another three-dimensional model.

29. The device of claim 15 , wherein the one or more tag icons comprise two-dimensional or three-dimensional iconography, and wherein the rendering component is configured to display the one or more tag icons such that an orientation of the one or more tag icons is the same in the one or more representations.

30. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor of a mobile device, facilitate performance of operations, comprising:

rendering one or more representations of a three-dimensional model of an environment from one or more perspectives of a virtual camera relative to the three-dimensional model based on navigational input received in association with navigation of the three-dimensional model, wherein the three-dimensional model is associated with tags at defined points of the three-dimensional model;

determining a first subset of the tags associated with a higher probability of selection relative to a second subset of the tags; and

based on the determining, including one or more tag icons for one or more tags of the first subset of the tags in the one or more representations, wherein the one or more tag icons are selectable and wherein selection of the one or more tag icons causes the one or more tags respectively associated therewith to be rendered by the device.

31. The non-transitory computer-readable storage medium of claim 30 , wherein the including comprises displaying the one or more tag icons at spatially aligned positions in the one or more representations based on one or more defined points of the defined points that the one or more tags are respectively associated with.

32. The non-transitory computer-readable storage medium of claim 30 , the operations further comprising:

receiving the first subset of the tags from a remote device based on the determining.

33. The non-transitory computer-readable storage medium of claim 32 , wherein the receiving comprises receiving respective tags included in the first subset of the tags in an order determined based on respective probabilities of selection associated with the respective tags.

34. The non-transitory computer-readable storage medium of claim 30 , wherein the determining comprises determining the first subset of the tags based in part on the navigational input.

35. The non-transitory computer-readable storage medium of claim 34 , wherein the determining further comprises determining the first subset of the tags based on machine learning analysis of historical navigation patterns associated with the three-dimensional model.

36. The non-transitory computer-readable storage medium of claim 30 , wherein the determining comprises determining the first subset of the tags based on one or more preferences of a user identity associated with the navigational input.

37. The non-transitory computer-readable storage medium of claim 30 , wherein the tags comprise data selected from the group consisting of: text data, audio data, image data, three-dimensional object data, video data, hyperlink data, and executable application data.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2025
From: MATTERPORT, LLC
To: COSTAR REALTY INFORMATION, INC.
Reel/Frame 072938/0425 →
MERGER AND CHANGE OF NAME Recorded Sep 10, 2025
From: MATTERPORT, INC.; MATRIX MERGER SUB II LLC
To: MATTERPORT, LLC
Reel/Frame 072827/0559 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2016
From: MILDREW, JAMES; BELL, MATTHEW TSCHUDY; COOK, DUSTIN MICHAEL; COWLEY, PRESTON; LEE, LESTER; MCCOLGAN, PETER; PROCHAZKA, DANIEL; SCHULMAN, BRIAN; SUNDRA, JAMES; TAN, ALAN
To: MATTERPORT, INC.
Reel/Frame 040962/0205 →
Continuity (6)
Continuation In Part 15199853 · Jun 30, 2016
Continuation In Part 14219906 · Mar 19, 2014
Continuation In Part 13925772 · Jun 24, 2013
Provisional Application 62187201 · Jun 30, 2015
Provisional Application 61663265 · Jun 22, 2012
Related Publication 20180143756A1 · May 24, 2018
Cited By (32)
US 1,098,175 US 12,190,581 US 12,217,311 US 12,260,156 US 12,266,011 US 12,266,166 US 12,293,470 US 12,333,655 US 12,347,033 US 12,380,645 US 12,400,436 US 12,444,139 US 12,462,435 US 12,462,483 US 12,495,390 US 12,524,977 US 12,541,280 US 12,548,218 US 12,561,924 US 12,602,865 US 12,608,115 US 12,608,877 US 12,608,890 US 12,625,608 US 12,632,170 US 12,633,044 US 12,633,119 US 12,666,163 US 12,675,958 US 12,675,990 US 12,687,922 US 12,688,660