Method and apparatus for accurate localization of points of interest
Geo-referenced and oriented media items may be used to determine a location of one or more points of interest depicted by the media items. A difference between an actual capture location and orientation and a reported location and orientation may be modeled according to one or more distributions, which distribution(s) may be used to assign one or more weights to each location in the world where such weight(s) may be considered to be a likelihood that a point of interest might have been seen by a capturing device. A density map may be acquired by superimposing the derived likelihoods, and a maximum, e.g., local maximum, may be determined to represent a location of a point of interest.
1. A method comprising:
selecting, using at least one computing system, a plurality of digital media items, each media item of the plurality having associated capture information identifying a location and orientation of a digital media device capturing the media item;
generating, using the at least one computing system, a plurality of weights for each media item in the plurality, each weight of the media item's plurality of weights corresponding to a geographic location of a plurality of geographic locations, the weight reflects an estimated inaccuracy in the identified location and orientation of the digital media device and is generated using information including the media item's identified location and orientation, the generated weight for use in determining whether a point of interest depicted in the media item is located at the geographic location; and
for each geographic location of the plurality, aggregating, using the at least one computing system, the plurality of weights generated for the geographic location in connection with each media item of the plurality of media items, an aggregated weight for a given geographic location identifying a probability that at least one point of interest is located at the given geographic location.
2. The method of claim 1 , the generating further comprising:
generating each weight of the plurality of weights for a media item of the plurality of media items as a combined weight for a geographic location of the plurality of geographic locations, comprising, for each geographic location of the plurality of geographic locations:
generating, by the at least one computing system, a capture orientation inaccuracy weight for the media item using at least one statistical orientation inaccuracy model;
generating, by the at least one computing system, a capture location inaccuracy weight for the media item using at least one statistical location inaccuracy model; and
generating, by the at least one computing system, the combined weight for the media item by combining the capture orientation and location inaccuracies determined for the geographic location.
3. The method of claim 2 , wherein one of the capture orientation inaccuracy weight and the capture location inaccuracy weight is optional and the other is non-optional, and the combining resets the optional one of them so that the combined weight reflects only the non-optional one of the weights.
4. The method of claim 2 , the at least one statistical orientation inaccuracy model comprising at least one one-dimensional distribution centered on a mean orientation and having a standard deviation reflecting an orientation spread.
5. The method of claim 4 ,
for each geographic location u w of the plurality of geographic locations, the generating a capture orientation inaccuracy weight for the media item using at least one statistical orientation inaccuracy model further comprising the at least one computing system:
selecting a location along a line of sight indicated by the capture information's orientation, the line of sight location is selected so that a first distance between the geographic location u w and the capture location u p is the same as a second distance between the capture location u p and the line of sight location;
determining an orientation angle comprising an angle α(u w , u p ,θ p ) representing an orientation angle between the capture location's orientation and the geographic location's orientation, the mean orientation of the statistical orientation inaccuracy model is represented as θ p and the standard deviation is represented as σ θ ; and
determining the capture orientation inaccuracy weight for the media item and the geographic location u w using the angle α(u w , u p ,θ p ) and the standard deviation σ θ in accordance with the at least one statistical orientation inaccuracy model.
6. The method of claim 5 , the capture orientation inaccuracy weight for the media item and the geographic location u w is represented as G θ p (u w ;u p ,θ p ,σ θ ), which is determined using an expression:
G
θ
p
(
u
w
;
u
p
,
θ
p
,
σ
θ
)
=
1
2
π
σ
θ
ⅇ
-
α
(
u
w
,
u
p
,
θ
p
)
2
2
σ
θ
2
.
7. The method of claim 5 , wherein the capture orientation inaccuracy weight for the media item and the geographic location u w is determined using a statistical distribution for the at least one statistical orientation inaccuracy model.
8. The method of claim 2 , the at least one statistical location inaccuracy model comprising at least one two-dimensional model.
9. The method of claim 8 ,
for each geographic location u w of the plurality of geographic locations, the generating a capture location inaccuracy weight for the media item using at least one statistical location inaccuracy model further comprising the at least one computing system:
determining a distance represented as δ(u w ,u p ) between the capture location and the geographic location, the mean location of the statistical location inaccuracy model is the capture location represented as u p ; and
determining the capture location inaccuracy weight for the media item and the geographic location u w using the distance δ(u w ,u p ) and the standard deviation represented as σ u in accordance with the at least one statistical location inaccuracy model.
10. The method of claim 9 , the capture location inaccuracy weight for the media item and the geographic location u w is represented as G u p (u w ;u p ;σ u ) and is determined using an expression:
G
u
p
(
u
w
;
u
p
,
σ
u
)
=
1
2
πσ
u
2
ⅇ
-
δ
(
u
w
,
u
p
)
2
2
σ
u
2
.
11. The method of claim 9 , wherein the capture location inaccuracy weight for the media item and the geographic location u w is determined using a statistical distribution for the at least one statistical location inaccuracy model.
12. A system comprising:
processor;
a storage medium for tangibly storing thereon program logic for execution by the processor, the stored program logic comprising:
selecting logic executed by the processor for selecting a plurality of digital media items, each media item of the plurality having associated capture information identifying a location and orientation of a digital media device capturing the media item;
generating logic executed by the processor for generating a plurality of weights for each media item in the plurality, each weight of the media item's plurality of weights corresponding to a geographic location of a plurality of geographic locations, the weight reflects an estimated inaccuracy in the identified location and orientation of the digital media device and is generated using information including the media item's identified location and orientation, the generated weight for use in determining whether a point of interest depicted in the media item is located at the geographic location; and
aggregating logic executed by the processor for aggregating, for each geographic location of the plurality, use the at least one computing system, the plurality of weights generated for the geographic location in connection with each media item of the plurality of media items, an aggregated weight for a given geographic location identifying a probability that at least one point of interest is located at the given geographic location.
13. The system of claim 12 , the generating logic executed by the processor for generating a plurality of weights further comprising:
generating logic executed by the processor for generating each weight of the plurality of weights for the media item of the plurality of media items as a combined weight for the geographic location of the plurality of geographic locations, the logic further comprising, for each geographic location of the plurality of geographic locations:
generating logic executed by the processor for generating a capture orientation inaccuracy weight for the media item using at least one statistical orientation inaccuracy model;
generating logic executed by the processor for generating a capture location inaccuracy weight for the media item using at least one statistical location inaccuracy model; and
generating logic executed by the processor for generating the combined weight for the media item by combining the capture orientation and location inaccuracies determined for the geographic location.
14. The system of claim 13 , wherein one of the capture orientation inaccuracy weight and the capture location inaccuracy weight is optional and the other is non-optional, and the combining resets the optional one of them so that the combined weight reflects only the non-optional one of the weights.
15. The system of claim 13 , the at least one statistical orientation inaccuracy model comprising at least one one-dimensional distribution centered on a mean orientation and having a standard deviation reflecting an orientation spread.
16. The system of claim 15 ,
for each geographic location u w of the plurality of geographic locations, the generating logic executed by the processor for generating a capture orientation inaccuracy weight for the media item using at least one statistical orientation inaccuracy model further comprising:
selecting logic executed by the processor for selecting a location along a line of sight indicated by the capture information's orientation, the line of sight location is selected so that a first distance between the geographic location u w and the capture location u p is the same as a second distance between the capture location u p and the line of sight location;
determining logic executed by the processor for determining an orientation angle comprising an angle α(u w ,u p ,θ p ) representing an orientation angle between the capture location's orientation and the geographic location's orientation, the mean orientation of the statistical orientation inaccuracy model is represented as θ p and the standard deviation is represented as σ θ ; and
determining logic executed by the processor for determining the capture orientation inaccuracy weight for the media item and the geographic location u w using the angle α(u w ,u p ,θ p ) and the standard deviation σ θ in accordance with the at least one statistical orientation inaccuracy model.
17. The system of claim 16 , the capture orientation inaccuracy weight for the media item and the geographic location u w is represented as G θ p (u w ;u p ,θ p ,σ θ ), which is determined using an expression:
G
θ
p
(
u
w
;
u
p
,
θ
p
,
σ
θ
)
=
1
2
π
σ
θ
ⅇ
-
α
(
u
w
,
u
p
,
θ
p
)
2
2
σ
θ
2
.
18. The system of claim 16 , wherein the capture orientation inaccuracy weight for the media item and the geographic location u w is determined using a statistical distribution for the at least one statistical orientation inaccuracy model.
19. The system of claim 13 , the at least one statistical location inaccuracy model comprising at least one two-dimensional model.
20. The system of claim 19 ,
for each geographic location u w of the plurality of geographic locations, the instructions to generate capture location inaccuracy weight for the media item using at least one statistical location inaccuracy model further comprising instructions to:
determine a distance represented as δ(u w ,u p ) between the capture location and the geographic location, the mean location of the statistical location inaccuracy model is the capture location represented as u p ; and
determine the capture location inaccuracy weight for the media item and the geographic location u w using the distance δ(u w ,u p ) and the standard deviation represented as σ u in accordance with the at least one statistical location inaccuracy model.
21. The system of claim 20 , the capture location inaccuracy weight for the media item and the geographic location u w is represented as G u p (u w ;u p ;σ u ) and is determined using an expression:
G
u
p
(
u
w
;
u
p
;
σ
u
)
=
1
2
πσ
u
2
ⅇ
-
δ
(
u
w
,
u
p
)
2
2
σ
u
2
.
22. The system of claim 20 , wherein the capture location inaccuracy weight for the media item and the geographic location u w is determined using a statistical distribution for the at least one statistical location inaccuracy model.
23. A computer readable non-transitory storage medium for tangibly storing thereon computer readable instructions that when executed cause at least one processor to:
select a plurality of digital media items, each media item of the plurality having associated capture information identifying a location and orientation of a digital media device capturing the media item;
generate a plurality of weights for each media item in the plurality, each weight of the media item's plurality of weights corresponding to a geographic location of a plurality of geographic locations, the weight reflects an estimated inaccuracy in the identified location and orientation of the digital media device and is generated using information including the media item's identified location and orientation, the generated weight for use in determining whether a point of interest depicted in the media item is located at the geographic location; and
for each geographic location of the plurality, aggregating, use the at least one computing system, the plurality of weights generated for the geographic location in connection with each media item of the plurality of media items, an aggregated weight for a given geographic location identifying a probability that at least one point of interest is located at the given geographic location.
24. The computer readable non-transitory storage medium of claim 23 , the instructions to generate further comprising instructions to:
generate each weight of the plurality of weights for a media item of the plurality of media items as a combined weight for a geographic location of the plurality of geographic locations, the instructions further comprising, for each geographic location of the plurality of geographic locations, instructions to:
generate a capture orientation inaccuracy weight for the media item using at least one statistical orientation inaccuracy model;
generate a capture location inaccuracy weight for the media item using at least one statistical location inaccuracy model; and
generate the combined weight for the media item by combining the capture orientation and location inaccuracies determined for the geographic location.
25. The computer readable non-transitory storage medium of claim 24 , wherein one of the capture orientation inaccuracy weight and the capture location inaccuracy weight is optional and the other is non-optional, and the combining resets the optional one of them so that the combined weight reflects only the non-optional one of the weights.
26. The computer readable non-transitory storage medium of claim 24 , the at least one statistical orientation inaccuracy model comprising at least one one-dimensional distribution centered on a mean orientation and having a standard deviation reflecting an orientation spread.
27. The computer readable non-transitory storage medium of claim 26 , for each geographic location u w of the plurality of geographic locations, the instructions to generate a capture orientation inaccuracy weight for the media item using at least one statistical orientation inaccuracy model father comprising instructions to:
select a location along a line of sight indicated by the capture information's orientation, the line of sight location is selected so that a first distance between the geographic location u w and the capture location u p is the same as a second distance between the capture location u p and the line of sight location;
determine an orientation angle comprising an angle α(u w , u p ,θ p ) representing an orientation angle between the capture location's orientation and the geographic location's orientation, the mean orientation of the statistical orientation inaccuracy model is represented as θ p and the standard deviation is represented as σ θ ; and
determine the capture orientation inaccuracy weight for the media item and the geographic location u w using the angle α(u w , u p ,θ p ) and the standard deviation σ θ in accordance with the at least one statistical orientation inaccuracy model.
28. The computer readable non-transitory storage medium of claim 27 , the capture orientation inaccuracy weight for the media item and the geographic location u w is represented as G θ p (u w ;u p ,θ p ,σ θ ), which is determined using an expression:
G
θ
p
(
u
w
;
u
p
,
θ
p
,
σ
θ
)
=
1
2
π
σ
θ
ⅇ
-
α
(
u
w
,
u
p
,
θ
p
)
2
2
σ
θ
2
.
29. The computer readable non-transitory storage medium of claim 27 , wherein the capture orientation inaccuracy weight for the media item and the geographic location u w is determined using a statistical distribution for the at least one statistical orientation inaccuracy model.
30. The computer readable non-transitory storage medium of claim 24 , the at least one statistical location inaccuracy model comprising at least one two-dimensional model.
31. The computer readable non-transitory storage medium of claim 30 , for each geographic location u w of the plurality of geographic locations, the instructions to generate capture location inaccuracy weight for the media item using at least one statistical location inaccuracy model further comprising instructions to:
determine a distance represented as δ(u w ,u p ) between the capture location and the geographic location, the mean location of the statistical location inaccuracy model is the capture location represented as u p and the standard deviation is represented as σ u ;
and determine the capture location inaccuracy weight for the media item and the geographic location u w using the distance δ(u w ,u p ) and the standard deviation represented as σ u in accordance with the at least one statistical location inaccuracy model.
32. The computer readable non-transitory storage medium of claim 31 , the capture location inaccuracy weight for the media item and the geographic location u w is represented as G u p (u w ;u p ;σ u ) and is determined using an expression:
G
u
p
(
u
w
;
u
p
;
σ
u
)
=
1
2
πσ
u
2
ⅇ
-
δ
(
u
w
,
u
p
)
2
2
σ
u
2
.
33. The computer readable non-transitory storage medium of claim 31 , wherein the capture location inaccuracy weight for the media item and the geographic location u w is determined using a statistical distribution for the at least one statistical location inaccuracy model.