IP Library Granted Patent US 11,422,671
Granted Patent B2
US 11,422,671 · App. 17/021,848 · Granted Aug 23, 2022

Defining, displaying and interacting with tags in a three-dimensional model

Inventors: James Mildrew (Mountain View, CA); Matthew Tschudy Bell (Palo Alto, CA); Dustin Michael Cook (San Jose, CA); Preston Cowley (San Jose, CA); Lester Lee (Oakland, CA); Peter McColgan (Fremont, CA); Daniel Prochazka (Pacifica, CA); Brian Schulman (San Francisco, CA); James Sundra (San Jose, CA); Alan Tan (San Jose, CA)
Assignee: Matterport, Inc.
G06F3/04815G06F3/011G06F3/013G06F3/0482G06F3/04817G06T15/20G06T17/05G06T19/003
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,422,671
App. No.
17/021,848
Granted
Aug 23, 2022
Kind
B2
Abstract

This application generally relates to defining, displaying and interacting with tags in a 3D model. In an embodiment, a method includes generating, by a system including a processor, a three-dimensional model of an environment based on sets of aligned three-dimensional data captured from the environment, and associating tags with defined locations of the three-dimensional model, wherein the tags are respectively represented by tag icons that are spatially aligned with the defined locations of the three-dimensional model as included in different representations of the three-dimensional model rendered via an interface of a device, wherein the different representations correspond to different perspectives of the three-dimensional model, and wherein selection of the tag icons causes the tags respectively associated therewith to be rendered at the device.

Claims (39)

1. A system, comprising:

a memory that stores computer executable components; and

a processor that executes at least the following computer executable components stored in the memory:

a three-dimensional (3D) model generation component that facilitates generation of a three-dimensional model of an environment and determination of positions of one or more objects in the three-dimensional model;

a tag indexing component that facilitates generation of a tag index, the tag index including tags identifying the one or more objects included in the three-dimensional model, at least one tag of the tags is associated with an object or an application related to the object;

a search component that facilitates a search of the tag index for one or more tags which satisfies one or more filter criteria;

a navigation component that facilitates navigation to a view of the three-dimensional model, wherein the view of the three-dimensional model comprises the one or more tags which satisfies the one or more filter criteria and associated with defined positions on or within the three-dimensional model; and

a tag component that facilitates interaction with the tags as included in representations of the three-dimensional model rendered at a device in association with navigation of the three-dimensional model.

2. The system of claim 1 , wherein the tag component further facilitates provision of a hyperlink to the application or activates the application.

3. The system of claim 1 , wherein the navigation component further facilitates to receive navigation input indicating desired perspectives for viewing the three-dimensional model.

4. The system of claim 3 , further comprising a rendering component that facilitates a rendition of different representations of the three-dimensional model from different perspectives of a virtual camera relative to the three-dimensional model determined based on the navigation input.

5. The system of claim 1 , wherein the application comprises an object interaction application that provides for interacting with one or more functions of a tagged object.

6. The system of claim 5 , wherein the tagged object is a virtual device and wherein the virtual device corresponds to an actual device at a remote location, and wherein the application comprises a remote-control application that facilitates remotely controlling or remotely monitoring a function of the actual device.

7. The system of claim 6 , wherein the remote-control application facilitates the remotely controlling or the remotely monitoring of the function of the actual device based on interaction with the virtual device.

8. The system of claim 1 , wherein the tag component further facilitates integrating visual tag icons for a subset of the tags at their defined positions in the representation of the three-dimensional model, and wherein the tag component facilitates interacting with respective tags of the subset of the tags based on interaction with the visual tag icons.

9. The system of claim 8 , wherein the tag component selects the subset of the tags based on a preference or a demographic of a user identity operating the device in association with the navigation.

10. The system of claim 8 , wherein the application is associated with a merchant and facilitates purchasing the tagged object from the merchant online, and wherein the tag component selects the at least one tag for inclusion in the subset of the tags based on a correlation between the tagged object and a preference or a demographic of a user identity operating the device in association with the navigation.

11. The system of claim 1 , wherein the three-dimensional model was generated by aligning two-dimensional images captured from the environment based on three-dimensional data respectively associated with the two-dimensional images.

12. The system of claim 1 , wherein in response to the determining if the at least one tag of the tags is blocked or partially blocked, the tag component further changes one or more properties of the at least one tag, the one or more properties of the at least one tag including a size of the at least one tag, a shape of the at least one tag, and a length of a stem of the at least one tag.

13. A method comprising:

generating a three-dimensional model of an environment and determining positions of one or more objects in the three-dimensional model;

generating a tag index, the tag index including tags identifying the one or more objects included in the three-dimensional model, at least one tag of the tags is associated with an object or an application related to the object;

searching of the tag index for one or more tags which satisfies one or more filter criteria;

navigating to a view of the three-dimensional model, wherein the view of the three-dimensional model comprises the one or more tags which satisfies the one or more filter criteria and associated with defined positions on or within the three-dimensional model; and

interacting with the tags as included in representations of the three-dimensional model rendered at a device in association with navigation of the three-dimensional model.

14. The method of claim 13 , further comprising providing a hyperlink to the application or activates the application.

15. The method of claim 13 , further comprising receiving navigation input indicating desired perspectives for viewing the three-dimensional model.

16. The method of claim 15 , further comprising rendering different representations of the three-dimensional model from different perspectives of a virtual camera relative to the three-dimensional model determined based on the navigation input.

17. The method of claim 13 , wherein the tagged object is a virtual device and wherein the virtual device corresponds to an actual device at a remote location, and wherein the application comprises a remote-control application that facilitates remotely controlling or remotely monitoring a function of the actual device.

18. The method of claim 13 , wherein in response to the determining if the at least one tag of the tags is blocked or partially blocked, the device further changes one or more properties of the at least one tag, the one or more properties of the at least one tag including a size of the at least one tag, a shape of the at least one tag, and a length of a stem of the at least one tag.

19. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor of a device, facilitate performance of operations, comprising:

generating a three-dimensional model of an environment and determining positions of one or more objects in the three-dimensional model;

generating a tag index, the tag index including tags identifying the one or more objects included in the three-dimensional model, at least one tag of the tags is associated with an object or an application related to the object;

searching of the tag index for one or more tags which satisfies one or more filter criteria;

navigating to a view of the three-dimensional model, wherein the view of the three-dimensional model comprises the one or more tags which satisfies the one or more filter criteria and associated with defined positions on or within the three-dimensional model; and

interacting with the tags as included in representations of the three-dimensional model rendered at a device in association with navigation of the three-dimensional model.

20. The non-transitory machine-readable storage medium of claim 19 , further comprising providing a hyperlink to the application or activates the application.

21. The non-transitory machine-readable storage medium of claim 19 , further comprising receiving navigation input indicating desired perspectives for viewing the three-dimensional model.

22. The non-transitory machine-readable storage medium of claim 21 , further comprising rendering different representations of the three-dimensional model from different perspectives of a virtual camera relative to the three-dimensional model determined based on the navigation input.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 25, 2025
From: MATTERPORT, LLC
To: COSTAR REALTY INFORMATION, INC.
Reel/Frame 072938/0425 →
MERGER AND CHANGE OF NAME Recorded Sep 10, 2025
From: MATTERPORT, INC.; MATRIX MERGER SUB II LLC
To: MATTERPORT, LLC
Reel/Frame 072827/0559 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2020
From: MILDREW, JAMES; BELL, MATTHEW TSCHUDY; COOK, DUSTIN MICHAEL; COWLEY, PRESTON; LEE, LESTER; MCCOLGAN, PETER; PROCHAZKA, DANIEL; SCHULMAN, BRIAN; SUNDRA, JAMES; TAN, ALAN
To: MATTERPORT, INC.
Reel/Frame 053848/0878 →
Continuity (8)
Continuation 16162683 · Oct 17, 2018
Continuation 15272337 · Sep 21, 2016
Continuation In Part 15199853 · Jun 30, 2016
Continuation In Part 14219906 · Mar 19, 2014
Continuation In Part 13925772 · Jun 24, 2013
Provisional Application 62187201 · Jun 30, 2015
Provisional Application 61663265 · Jun 22, 2012
Related Publication 20210064217A1 · Mar 4, 2021
Cited By (4)
US 12,217,311 US 12,390,721 US 12,456,262 US 12,633,047