IP Library Granted Patent US 12,494,027
Granted Patent B2
US 12,494,027 · App. 18/514,985 · Granted Dec 9, 2025

Systems and methods for generating augmented reality content

Inventors: Peter Wilczynski (San Francisco, CA); Daniel Cervelli (Mountain View, CA); Andrew Elder (New York, NY); Anand Gupta (Washington, DC); Praveen Kumar Ramalingam (Washington, DC); Robert Imig (Austin, TX)
Assignee: Palantir Technologies Inc.
G06T19/006H04N21/2187H04N21/2353
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,494,027
App. No.
18/514,985
Granted
Dec 9, 2025
Kind
B2
Abstract

In some embodiments, a method comprises obtaining a video stream of a portion of a geographic area, the video stream comprising a plurality of video frames, each of the plurality of video frames captured at a respective first time. Contextual metadata is obtained, the contextual metadata associated with one or more objects located in the portion of the geographic area at a second time, the second time being before each of the respective first times. The contextual metadata is inserted into one or more of the plurality of video frames, thereby causing the contextual metadata associated with the one or more objects to be overlaid on one or more corresponding portions of the one or more of the plurality of video frames.

Claims (43)

1 . A system comprising:

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the system to perform:

obtaining a selection of a geographical region;

based on the selection of the geographical region, generating an executable query to search for vehicles within a threshold vicinity of the geographical region and to search for pre-recorded video content streams within the threshold vicinity;

executing the executable query to search for the vehicles within the threshold vicinity of the geographical region;

presenting one or more live video content streams generated by the vehicles and one or more pre-recorded content streams within the threshold vicinity;

obtaining a selection of a video content stream generated by a vehicle, the video content stream comprising video frames, each of the video frames captured at respective first times and having respective fields of view (FOVs);

obtaining one or more annotations based on the pre-recorded video content streams associated with one or more objects within perimeter borders of the respective FOVs;

generating contextual metadata based on the obtained annotations; and

overlaying the contextual metadata onto one or more particular video frames of the video content stream.

2 . The system of claim 1 , wherein the one or more objects comprise one or more building structures, and the annotations indicate information associated with the one or more building structures.

3 . The system of claim 1 , wherein the one or more objects comprise cellular signals, radio signals, and Wireless Fidelity (Wi-Fi) signals.

4 . The system of claim 1 , wherein the contextual metadata further comprises a first layer comprising a physical structure, a second layer comprising electronic signal activity, and a third layer comprising message activity.

5 . The system of claim 1 , wherein the instructions that, when executed by the one or more processors, cause the system to perform:

in response to obtaining the video content stream, processing the video content stream by performing two-dimensional georectification and post-processing to add elevation data to produce a three-dimensional model, wherein, following the post-processing, the three-dimensional model has a higher degree of complexity and accuracy compared to the obtained video content stream.

6 . A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:

obtaining a selection of a geographical region;

based on the selection of the geographical region, generating an executable query to search for vehicles within a threshold vicinity of the geographical region and to search for pre-recorded video content streams within the threshold vicinity;

executing the executable query to search for the vehicles within the threshold vicinity of the geographical region;

presenting live video content streams generated by the vehicles and one or more pre-recorded content streams within the threshold vicinity;

obtaining a selection of a video content stream generated by a vehicle, the video content stream comprising video frames, each of the video frames captured at respective first times and having respective fields of view (FOVs);

obtaining one or more annotations associated with one or more objects within perimeter borders of the respective FOVs based on the pre-recorded video content streams;

generating contextual metadata based on the obtained annotations; and

overlaying the contextual metadata onto one or more particular video frames of the video content stream.

7 . The method of claim 6 , wherein the one or more objects comprise cellular signals, radio signals, and Wireless Fidelity (Wi-Fi) signals.

8 . The method of claim 6 , wherein the contextual metadata further comprises a first layer comprising a physical structure, a second layer comprising electronic signal activity, and a third layer comprising message activity.

9 . The method of claim 6 , wherein the video content stream is obtained by:

acquiring a selection of a portion of a geographic area;

obtaining geographic coordinates corresponding to the portion;

searching for a video content stream associated with the portion from the vehicle within a threshold vicinity of the selected portion; and

receiving a selection of the video content stream.

10 . A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:

obtaining a selection of a geographical region;

based on the selection of the geographical region, generating an executable query to search for vehicles within a threshold vicinity of the geographical region and to search for pre-recorded video content streams within the threshold vicinity;

executing the executable query to search for the vehicles within the threshold vicinity of the geographical region;

presenting live video content streams generated by the vehicles and one or more pre-recorded content streams within the threshold vicinity;

obtaining a selection of a video content stream generated by a vehicle, the video content stream comprising video frames, each of the video frames captured at respective first times and having respective fields of view (FOVs);

obtaining one or more annotations associated with one or more objects within perimeter borders of the respective FOVs based on the pre-recorded video content streams;

generating contextual metadata based on the obtained annotations; and

overlaying the contextual metadata onto one or more particular video frames of the video content stream.

11 . The non-transitory computer readable medium of claim 10 , wherein the one or more objects comprise one or more building structures, and the annotations indicate information associated with the one or more building structures.

12 . The non-transitory computer readable medium of claim 10 , wherein the one or more objects comprise cellular signals, radio signals, and Wireless Fidelity (Wi-Fi) signals.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 20, 2023
From: WILCZYNSKI, PETER; CERVELLI, DANIEL; ELDER, ANDREW; GUPTA, ANAND; RAMALINGAM, PRAVEEN KUMAR; IMIG, ROBERT
To: PALANTIR TECHNOLOGIES INC.
Reel/Frame 065624/0942 →
Continuity (4)
Continuation 17572995 · Jan 11, 2022
Continuation 16129513 · Sep 12, 2018
Provisional Application 62720733 · Aug 21, 2018
Related Publication 20240087263A1 · Mar 14, 2024
References Cited (97)
US 6642925B2 · Roy et al. · 2003 [cited by applicant]
US 7133771B1 · Nesbitt · 2006 [cited by applicant]
US 7881861B2 · Kravets et al. · 2011 [cited by applicant]
US 8108141B2 · Ehrlacher · 2012 [cited by applicant]
US 8463537B2 · Mueller et al. · 2013 [cited by applicant]
US 9360335B1 · Powelson et al. · 2016 [cited by applicant]
US 9547986B1 · Curlander et al. · 2017 [cited by applicant]
US 9600146B2 · Cervelli et al. · 2017 [cited by applicant]
US 9784589B1 · Gyenes et al. · 2017 [cited by applicant]
US 9996229B2 · Erenrich et al. · 2018 [cited by applicant]
US 10133995B1 · Reiss et al. · 2018 [cited by applicant]
US 10241588B1 · Misra · 2019 [cited by examiner]
US 10484114B1 · Nijim et al. · 2019 [cited by applicant]
US 11222470B1 · Wilczynski et al. · 2022 [cited by applicant]
US 20060015499A1 · Clissold et al. · 2006 [cited by applicant]
US 20060184314A1 · Couckuyt · 2006 [cited by applicant]
US 20070168118A1 · Lappe · 2007 [cited by applicant]
US 20070293958A1 · Stehle · 2007 [cited by applicant]
US 20090248300A1 · Dunko et al. · 2009 [cited by applicant]
US 20090303251A1 · Balogh et al. · 2009 [cited by applicant]
US 20100036599A1 · Froeberg · 2010 [cited by applicant]
US 20100217784A1 · Carter · 2010 [cited by applicant]
US 20100253541A1 · Seder · 2010 [cited by applicant]
US 20110106429A1 · Poppen · 2011 [cited by applicant]
US 20120123678A1 · Poppen · 2012 [cited by applicant]
US 20120123806A1 · Schumann, Jr. · 2012 [cited by applicant]
US 20120136689A1 · Ickman · 2012 [cited by applicant]
US 20120253661A1 · Tuukkanen · 2012 [cited by applicant]
US 20130132389A1 · Majidian · 2013 [cited by applicant]
US 20130182072A1 · Seo et al. · 2013 [cited by applicant]
US 20130298083A1 · Bertoldo · 2013 [cited by applicant]
US 20140005941A1 · Paek · 2014 [cited by applicant]
US 20140036090A1 · Black · 2014 [cited by applicant]
US 20140108419A1 · Udeshi et al. · 2014 [cited by applicant]
US 20140123067A1 · Sokolsky · 2014 [cited by examiner]
US 20140210839A1 · Castelli · 2014 [cited by applicant]
US 20140267723A1 · Davidson, Jr. · 2014 [cited by applicant]
US 20140282093A1 · Burke · 2014 [cited by applicant]
US 20140333651A1 · Cervelli et al. · 2014 [cited by applicant]
US 20140372498A1 · Mian · 2014 [cited by applicant]
US 20150032366A1 · Man · 2015 [cited by applicant]
US 20150177964A1 · Spirer · 2015 [cited by applicant]
US 20150281250A1 · Miller et al. · 2015 [cited by applicant]
US 20150338852A1 · Ramanujam · 2015 [cited by applicant]
US 20150339928A1 · Ramanujam · 2015 [cited by applicant]
US 20160069694A1 · Tao · 2016 [cited by applicant]
US 20160109251A1 · Thakur · 2016 [cited by applicant]
US 20160109252A1 · Caine · 2016 [cited by applicant]
US 20160142964A1 · Todasco · 2016 [cited by applicant]
US 20160203422A1 · Demarchi · 2016 [cited by applicant]
US 20160229404A1 · Byun · 2016 [cited by applicant]
US 20160273930A1 · Wada · 2016 [cited by applicant]
US 20160298974A1 · Newlin · 2016 [cited by applicant]
US 20160298977A1 · Newlin · 2016 [cited by applicant]
US 20160334233A1 · Baverstock · 2016 [cited by applicant]
US 20160379486A1 · Taylor · 2016 [cited by applicant]
US 20170052654A1 · Cervelli et al. · 2017 [cited by applicant]
US 20170052655A1 · Cervelli et al. · 2017 [cited by applicant]
US 20170084037A1 · Barajas Hernandez · 2017 [cited by applicant]
US 20170167882A1 · Ulloa Paredes · 2017 [cited by applicant]
US 20170169373A1 · Roulland · 2017 [cited by applicant]
US 20170192437A1 · Bier · 2017 [cited by applicant]
US 20170262790A1 · Khasis · 2017 [cited by applicant]
US 20170263092A1 · Rankin · 2017 [cited by applicant]
US 20170268891A1 · Dyrnaes · 2017 [cited by applicant]
US 20170276507A1 · Zacharenko · 2017 [cited by applicant]
US 20170300049A1 · Seally · 2017 [cited by applicant]
US 20170314948A1 · Racah et al. · 2017 [cited by applicant]
US 20170314949A1 · Rovik · 2017 [cited by applicant]
US 20170329569A1 · Wilczynski et al. · 2017 [cited by applicant]
US 20170337813A1 · Taylor · 2017 [cited by applicant]
US 20180143027A1 · Schlesinger · 2018 [cited by applicant]
US 20180143649A1 · Miao · 2018 [cited by applicant]
US 20180211541A1 · Rakah · 2018 [cited by applicant]
US 20180211546A1 · Smartt · 2018 [cited by applicant]
US 20180259976A1 · Williams · 2018 [cited by applicant]
US 20190146996A1 · Jang · 2019 [cited by examiner]
US 20190197787A1 · Easter · 2019 [cited by applicant]
US 20190206130A1 · Ericson · 2019 [cited by applicant]
Rau et al. (Geometrical Building Modeling and Its Application to the Ortho-Rectification for Aerial Images, Journal of Photogrammetry and Remote Sensing vol. 9, No. 1, Mar. 2004) (Year: 2004). [cited by examiner]
Koeda et al. (Annotation-Based Assistance System for Unmanned Helicopter with Wearable Augmented Reality Environment, IEEE 2014) (Year: 2004). [cited by applicant]
Official Communication for U.S. Appl. No. 15/348,860 dated Dec. 15, 2017. [cited by applicant]
Official Communication for U.S. Appl. No. 15/666,368 dated Nov. 16, 2017. [cited by applicant]
Official Communication for U.S. Appl. No. 15/676,509 dated Nov. 16, 2017. [cited by applicant]
Official Communication for U.S. Appl. No. 15/666,368 dated Feb. 22, 2018. [cited by applicant]
Official Communication for U.S. Appl. No. 15/676,509 dated Sep. 7, 2018. [cited by applicant]
Official Communication for U.S. Appl. No. 15/676,509 dated Jan. 11, 2019. [cited by applicant]
Official Communication for U.S. Appl. No. 15/676,509 dated Feb. 26, 2018. [cited by applicant]
Official Communication for U.S. Appl. No. 15/826,446 dated Dec. 10, 2018. [cited by applicant]
Official Communication for U.S. Appl. No. 15/826,446 dated Jan. 22, 2018. [cited by applicant]
Official Communication for U.S. Appl. No. 15/826,446 dated Aug. 10, 2018. [cited by applicant]
Non-Final Office Action dated Feb. 16, 2023, issued in related U.S. Appl. No. 17/572,995 (15 pages). [cited by applicant]
FAI Pre-Interview Communication dated May 3, 2019, issued in related U.S. Appl. No. 16/129,513 (4 pages). [cited by applicant]
Final Office Action dated Dec. 2, 2019, issued in related U.S. Appl. No. 16/129,513 (11 pages). [cited by applicant]
Non-Final Office Action dated Jun. 1, 2020, issued in related U.S. Appl. No. 16/129,513 (16 pages). [cited by applicant]
Final Office Action dated Oct. 27, 2020, issued in related U.S. Appl. No. 16/129,513 (18 pages). [cited by applicant]
Non-Final Office Action dated May 24, 2021, issued in related U.S. Appl. No. 16/129,513 (16 pages). [cited by applicant]
Cited By (1)
US 12,573,156