IP Library › Granted Patent US 12,211,157
Granted Patent B2
US 12,211,157 · App. 18/355,209 · Granted Jan 28, 2025

Crowd sourced mapping system

Inventors: Piers Cowburn (London, GB); Isac Andreas Müller Sandvik (London, GB); Qi Pan (London, GB); David Li (London, GB)
Assignee: Snap Inc.
G06T19/006G06F16/51G06F16/54G06T7/97G06T19/20H04L51/224H04L67/51G06T2219/024H04L67/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,211,157
App. No.
18/355,209
Granted
Jan 28, 2025
Kind
B2
Abstract

A crowd-sourced modeling system to perform operations that include: receiving image data that comprises image attributes; accessing a 3D model based on at least the image attributes of the image data, wherein the 3D model comprises a plurality of parts that collectively depict an object or environment; identifying a change in the object or environment based on a comparison of the image data with the plurality of parts of the 3D model, the change corresponding to a part of the 3D model from among the plurality of parts; and generating an update to the part of the 3D model based on the image attributes of the image data.

Claims (53)

1. A method comprising:

receiving image data that comprises image attributes and location data that identifies a location from a client device;

detecting a change in an environment associated with the location based on the image attributes of the image data;

accessing a repository that comprises a collection of image data associated with the location identified by the location data, the collection of image data comprising metadata that includes temporal data;

selecting image data from among the collection of image data based on the temporal data; and

generating an update to a mesh model that comprises a representation of a surface of the location based on the image data.

2. The method of claim 1 , wherein the generating the update to the mesh model includes:

accessing a three-dimensional (3D) model that corresponds with an object associated with the location, the 3D model comprising the mesh model that comprises the representation of the surface of the location, wherein a portion of the mesh model corresponds with a position of the object within the location;

generating the update to the portion of the mesh model that corresponds with the position of the object within the location based on image attributes of the image data.

3. The method of claim 1 , wherein the selecting the image data from among the collection of image data is based on the temporal data, and wherein the temporal data includes a timestamp.

4. The method of claim 1 , wherein the receiving the request that includes the location data from the client device includes:

detecting the client device within a threshold distance of the location; and

causing the client device to generate the request in response to the detecting the client device within the threshold distance of the location.

5. The method of claim 1 , wherein the selecting the image data from among the collection of image data based on the metadata includes:

selecting the image data based on the metadata and the location data.

6. The method of claim 1 , wherein the request further comprises image data, and wherein the generating the update to the mesh model that comprises the representation of the surface of the location includes:

detecting a change to the surface of the location based on the image data; and

generating the update to the mesh model responsive to the change.

7. A system comprising:

a memory; and

at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising:

receiving image data that comprises image attributes and location data that identifies a location from a client device;

detecting a change in an environment associated with the location based on the image attributes of the image data;

accessing a repository that comprises a collection of image data associated with the location identified by the location data, the collection of image data comprising metadata that includes temporal data;

selecting image data from among the collection of image data based on the temporal data; and

generating an update to a mesh model that comprises a representation of a surface of the location based on the image data.

8. The system of claim 7 , wherein the generating the update to the mesh model includes:

accessing a three-dimensional (3D) model that corresponds with an object associated with the location, the 3D model comprising the mesh model that comprises the representation of the surface of the location, wherein a portion of the mesh model corresponds with a position of the object within the location;

generating the update to the portion of the mesh model that corresponds with the position of the object within the location based on image attributes of the image data.

9. The system of claim 7 , wherein the selecting the image data from among the collection of image data is based on the temporal data, and wherein the temporal data includes a timestamp.

10. The system of claim 7 , wherein the receiving the request that includes the location data from the client device includes:

detecting the client device within a threshold distance of the location; and

causing the client device to generate the request in response to the detecting the client device within the threshold distance of the location.

11. The system of claim 7 , wherein the selecting the image data from among the collection of image data based on the metadata includes:

selecting the image data based on the metadata and the location data.

12. The system of claim 7 , wherein the request further comprises image data, and wherein the generating the update to the mesh model that comprises the representation of the surface of the location includes:

detecting a change to the surface of the location based on the image data; and

generating the update to the mesh model responsive to the change.

13. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving image data that comprises image attributes and location data that identifies a location from a client device;

detecting a change in an environment associated with the location based on the image attributes of the image data;

accessing a repository that comprises a collection of image data associated with the location identified by the location data, the collection of image data comprising metadata that includes temporal data;

selecting image data from among the collection of image data based on the temporal data; and

generating an update to a mesh model that comprises a representation of a surface of the location based on the image data.

14. The non-transitory machine-readable storage medium of claim 13 , wherein the generating the update to the mesh model includes:

accessing a three-dimensional (3D) model that corresponds with an object associated with the location, the 3D model comprising the mesh model that comprises the representation of the surface of the location, wherein a portion of the mesh model corresponds with a position of the object within the location;

generating the update to the portion of the mesh model that corresponds with the position of the object within the location based on image attributes of the image data.

15. The non-transitory machine-readable storage medium of claim 13 , wherein the selecting the image data from among the collection of image data is based on the temporal data, and wherein the temporal data includes a timestamp.

16. The non-transitory machine-readable storage medium of claim 13 , wherein the receiving the request that includes the location data from the client device includes:

detecting the client device within a threshold distance of the location; and

causing the client device to generate the request in response to the detecting the client device within the threshold distance of the location.

17. The non-transitory machine-readable storage medium of claim 13 , wherein the selecting the image data from among the collection of image data based on the metadata includes:

selecting the image data based on the metadata and the location data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 24, 2023
From: COWBURN, PIERS; SANDVIK, ISAC ANDREAS MULLER; PAN, QI; LI, DAVID
To: SNAP INC.
Reel/Frame 064361/0018 →
Continuity (4)
Continuation 17948813 · Sep 20, 2022
Continuation 17329435 · May 25, 2021
Continuation 16447591 · Jun 20, 2019
Related Publication 20230360344A1 · Nov 9, 2023
References Cited (47)
US 9036898B1 · Beeler · 2015 [cited by applicant]
US 10495476B1 · Yu et al. · 2019 [cited by applicant]
US 10586238B2 · Morey · 2020 [cited by examiner]
US 10769437B2 · Mukherjee et al. · 2020 [cited by applicant]
US 11087543B1 · Cowburn · 2021 [cited by examiner]
US 11488368B2 · Cowburn et al. · 2022 [cited by applicant]
US 20120226437A1 · Li et al. · 2012 [cited by applicant]
US 20130194110A1 · Kim et al. · 2013 [cited by applicant]
US 20130321257A1 · Moore · 2013 [cited by applicant]
US 20130321398A1 · Howard · 2013 [cited by examiner]
US 20140201126A1 · Zadeh · 2014 [cited by applicant]
US 20140369595A1 · Pavlidis · 2014 [cited by examiner]
US 20150204687A1 · Yoon et al. · 2015 [cited by applicant]
US 20170372322A1 · Morey · 2017 [cited by examiner]
US 20180110398A1 · Schwartz · 2018 [cited by examiner]
US 20190035143A1 · Pavlidis et al. · 2019 [cited by applicant]
US 20190094554A1 · Benesh · 2019 [cited by examiner]
US 20190311199A1 · Mukherjee et al. · 2019 [cited by applicant]
US 20190324705A1 · Bolaños · 2019 [cited by examiner]
US 20190392635A1 · Ma et al. · 2019 [cited by applicant]
US 20200025935A1 · Liang et al. · 2020 [cited by applicant]
US 20200027265A1 · Levy et al. · 2020 [cited by applicant]
US 20200074705A1 · Berger · 2020 [cited by examiner]
US 20200149916A1 · Phiar · 2020 [cited by applicant]
US 20200186731A1 · Chen · 2020 [cited by examiner]
US 20200192352A1 · Rastoll · 2020 [cited by examiner]
US 20210279969A1 · Cowburn et al. · 2021 [cited by applicant]
US 20230011686A1 · Cowburn et al. · 2023 [cited by applicant]
US 20230360344A1 · Cowburn · 2023 [cited by examiner]
“U.S. Appl. No. 16/447,591, Final Office Action mailed Sep. 21, 2020”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 16/447,591, Non Final Office Action mailed Apr. 1, 2020”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 16/447,591, Non Final Office Action mailed Dec. 23, 2020”, 15 pgs. [cited by applicant]
“U.S. Appl. No. 16/447,591, Notice of Allowance mailed Mar. 31, 2021”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 16/447,591, Response filed Mar. 23, 2021 to Non Final Office Action mailed Dec. 23, 2020”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 16/447,591, Response filed Jun. 30, 2020 to Non Final Office Action mailed Apr. 1, 2020”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 16/447,591, Response filed Nov. 23, 2020 to Final Office Action mailed Sep. 21, 2020”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 17/329,435, Final Office Action mailed Jan. 25, 2022”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 17/329,435, Non Final Office Action mailed Apr. 13, 2022”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/329,435, Non Final Office Action mailed Nov. 9, 2021”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 17/329,435, Notice of Allowance mailed Jul. 1, 2022”, 7 pgs. [cited by applicant]
“U.S. Appl. No. 17/329,435, Response filed Mar. 25, 2022 to Final Office Action mailed Jan. 25, 2022”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/329,435, Response filed May 13, 2022 to Non Final Office Action mailed Apr. 13, 2022”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/329,435, Response filed Dec. 7, 2021 to Non Final Office Action mailed Nov. 9, 2021”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/948,813, Corrected Notice of Allowability mailed Jun. 14, 2023”, 2 pgs. [cited by applicant]
“U.S. Appl. No. 17/948,813, Non Final Office Action mailed Feb. 28, 2023”, 12 pgs. [cited by applicant]
“U.S. Appl. No. 17/948,813, Notice of Allowance mailed Jun. 7, 2023”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 17/948,813, Response filed Mar. 30, 2023 to Non Final Office Action mailed Feb. 28, 2023”, 11 pgs. [cited by applicant]