IP Library › Granted Patent US 12,488,571
Granted Patent B2
US 12,488,571 · App. 18/191,845 · Granted Dec 2, 2025

Generating images for neural network training

Inventors: Rui Wang (Zurich, CH); Le Chen (Zurich, CH); Marc André Léon Pollefeys (Zurich, CH)
Assignee: Microsoft Technology Licensing, LLC.
G06V10/774G06T7/74G06V10/82G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,488,571
App. No.
18/191,845
Granted
Dec 2, 2025
Kind
B2
Abstract

A plurality of training examples is accessed, each training example comprising an image of a scene and a pose of a viewpoint from which the image was captured. A neural radiance field is trained using the training examples. A plurality of generated images is computed, by, for each of a plurality of randomly selected viewpoints, generating a color image and a depth image of the scene from the neural radiance field. A neural network is trained using the generated images.

Claims (20)

1 . A method comprising: accessing a plurality of training examples, each training example comprising a color image of a scene, a depth image of the scene, and a pose of a viewpoint from which the color image and depth image were captured; training a neural radiance field using the training examples; computing a plurality of generated images comprising, for a plurality of randomly selected viewpoints, generating a color image and a depth image of the scene from the neural radiance field and for the plurality of generated images, inspecting depth values of the generated image; and training a neural network using the generated images, the training of the neural network including omitting at least one of the generated images according to the depth values being below a threshold.

2 . The method of claim 1 wherein training the neural network comprises training the neural network with the generated images such that the neural network is able to predict correspondences between two dimensional (2D) image elements of an image of the scene and three-dimensional (3D) locations in a map of the scene comprising a 3D point cloud.

3 . The method of claim 2 comprising: receiving an image captured by a mobile image capture device in the scene; computing a plurality of correspondences by inputting the received image to the neural network; and computing a 3D position and orientation of the mobile image capture device with respect to the map, by inputting the correspondences to a perspective n point solver.

4 . The method of claim 1 comprising, pre training the neural network using the plurality of training examples.

5 . The method of claim 1 further comprising: for the plurality of generated images, computing a color uncertainty map from the neural radiance field; and wherein training the neural network comprises omitting at least one of the generated images according to the color uncertainty map of the omitted image indicating uncertainty over a threshold; and wherein the neural radiance field is trained using a negative log-likelihood loss with a variance-weighting term on color output of the neural radiance field.

6 . The method of claim 5 wherein the variance-weighting term is an adaptive learning rate.

7 . The method of claim 6 wherein the variance-weighting term comprises a parameter allowing for interpolation between the negative log-likelihood loss and a mean squared error loss.

8 . The method of claim 1 comprising, for the plurality of generated images, computing a depth uncertainty map from the neural radiance field; and wherein training the neural network comprises omitting one of the generated images according to the depth uncertainty map of the omitted image indicating uncertainty over a threshold.

9 . The method of claim 8 comprising training the neural radiance field using a Gaussian negative log-likelihood loss on depth output of the neural radiance field.

10 . The method of claim 8 comprising training the neural radiance field using a loss computed as a sum over rays projected into the neural radiance field to generate an image of: a logarithm of a square of a standard deviation of a predicted depth for the ray plus the square of an L2 difference between the predicted depth of the ray minus an actual depth of the ray, divided by the square of the standard deviation of the predicted depth for the ray.

11 . The method of claim 1 comprising training the neural radiance field using a loss function having a color term and a depth term.

12 . The method of claim 1 wherein the neural network outputs, for a predicted correspondence, an uncertainty value.

13 . The method of claim 12 wherein the uncertainty value for the predicted correspondence is computed by predicting hyperparameters of a normal inverse-gamma distribution.

14 . The method of claim 1 comprising, pre training the neural network using the plurality of training examples; and prior to training the neural network using the generated images, predicting correspondences having associated uncertainty values from the generated images using the neural network; and wherein training the neural network comprises omitting one of the generated images according to an uncertainty value of a predicted correspondence for the omitted generated image being above a threshold.

15 . The method of claim 1 comprising pre training the neural network using the plurality of training examples; and prior to training the neural network using the generated images, predicting correspondences having associated uncertainty values from the generated images using the neural network; and wherein training the neural network comprises using a loss function having terms which weigh pixels according to color uncertainty and depth uncertainty from the neural radiance field.

16 . An apparatus comprising: a processor; a memory storing instructions that, when executed by the processor, perform a method comprising: accessing a plurality of training examples, each training example comprising a color image of a scene, a depth image of the scene and a pose of a viewpoint from which the color image and depth image were captured; training a neural radiance field using the training examples; computing a plurality of generated images comprising, for a plurality of randomly selected viewpoints, generating a color image and a depth image of the scene from the neural radiance field and computing a color uncertainty map from the neural radiance field; and training a scene coordinate regression neural network using the generated images, the training of the scene coordinate regression neural network including omitting at least one of the generated images according to the color uncertainty map of the omitted image indicating an uncertainty over a threshold.

17 . A computer storage medium having computer-executable instructions that, when executed by a computing system, direct the computing system to perform operations comprising: accessing a plurality of training examples, each training example comprising a color image of a scene, a depth image of the scene, and a pose of a viewpoint from which the color image and depth image were captured; training a neural radiance field using the training examples; computing a plurality of generated images comprising, for a plurality of randomly selected viewpoints, generating a color image and a depth image of the scene from the neural radiance field; for the plurality of generated images, computing a color uncertainty map from the neural radiance field; and training a neural network using the generated images, the training of the neural network including omitting at least one of the generated images according to the color uncertainty map of the omitted image indicating an uncertainty over a threshold.

18 . The computer storage medium of claim 17 , wherein the computing system further performs operations comprising: training the neural radiance field using a loss function having a color term and a depth term.

19 . The computer storage medium of claim 17 , wherein the computing system further performs operations comprising: receiving an image captured by a mobile image capture device in the scene; computing a plurality of correspondences by inputting the received image to the neural network; and computing a three-dimensional (3D) position and orientation of the mobile image capture device, by inputting the correspondences to a perspective n point solver.

20 . The computer storage medium of claim 17 , wherein training the neural network comprises training the neural network with the generated images such that the neural network is able to predict correspondences between two-dimensional (2D) image elements of an image of the scene and three-dimensional (3D) locations in a map of the scene comprising a 3D point cloud.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2023
From: WANG, RUI; CHEN, LE; POLLEFEYS, MARC ANDRÉ LÉON
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 063139/0964 →
Continuity (1)
Related Publication 20240331356A1 · Oct 3, 2024
References Cited (123)
US 20240265504A1 · Wynn · 2024 [cited by examiner]
Adamkiewicz, et al., “Vision-Only Robot Navigation in a Neural Radiance World”, In Journal of IEEE Robotics and Automation Letters, vol. 7, Issue: 2, Feb. 11, 2022, pp. 4606-4613. [cited by applicant]
Amini, et al., “Deep Evidential Regression”, In Journal of Advances in Neural Information Processing Systems, vol. 33, Dec. 6, 2020, pp. 1-11. [cited by applicant]
Arandjelovic, et al., “All About VLAD”, In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Jun. 23, 2013, pp. 1578-1585. [cited by applicant]
Arandjelovic, et al., “NetVLAD: CNN Architecture for Weakly Supervised Place Recognition”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 27, 2016, pp. 5297-5307. [cited by applicant]
Bao, et al., “Evidential Deep Learning for Open Set Action Recognition”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 13349-13358. [cited by applicant]
Barron, et al., “Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 5470-5479. [cited by applicant]
Bay, et al., “SURF: Speeded Up Robust Features”, In Proceedings of European conference on computer vision, May 7, 2006, pp. 404-417. [cited by applicant]
Bi, et al., “Neural Reflectance Fields for Appearance Acquisition”, In Repository of arXiv:2008.03824v1, Aug. 9, 2020, 11 Pages. [cited by applicant]
Brachmann, et al., “DSAC—Differentiable RANSAC for Camera Localization”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jul. 21, 2017, pp. 6684-6692. [cited by applicant]
Brachmann, et al., “Learning Less Is More—6D Camera Localization via 3D Surface Regression”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 18, 2018, pp. 4654-4662. [cited by applicant]
Brachmann, et al., “Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 27, 2019, pp. 4322-4331. [cited by applicant]
Brachmann, et al., “Visual Camera Re-Localization From RGB and RGB-D Images Using DSAC”, In Journal of Transactions on Pattern Analysis and Machine Intelligence, vol. 44, Issue: 9., Apr. 2, 2021, pp. 5847-5865. [cited by applicant]
Brahmbhatt, et al., “Geometry-Aware Learning of Maps for Camera Localization”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 18, 2018, pp. 2616-2625. [cited by applicant]
Brualla, et al., “NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 19, 2021, pp. 7210-7219. [cited by applicant]
Cavallari, et al., “On-The-Fly Adaptation of Regression Forests for Online Camera Relocalisation”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jul. 21, 2017, pp. 4457-4466. [cited by applicant]
Chen, et al., “DFNet: Enhance Absolute Pose Regression with Direct Feature Matching”, In Proceedings of 17th ECCV, Nov. 3, 2022, 17 Pages. [cited by applicant]
Chen, et al., “Direct-PoseNet: Absolute Pose Regression with Photometric Consistency”, In Proceedings of the International Conference on 3D Vision, Dec. 1, 2021, pp. 1175-1185. [cited by applicant]
Chen, et al., “Exploring Simple Siamese Representation Learning”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 19, 2021, pp. 15750-15758. [cited by applicant]
Chen, et al., “iNeRF: Inverting Neural Radiance Fields for Pose Estimation”, In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 27, 2021, pp. 1323-1330. [cited by applicant]
Chen, et al., “MVSNeRF: Fast Generalizable Radiance Field Reconstruction From Multi-View Stereo”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 14124-14133. [cited by applicant]
Chen, et al., “NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields”, In Proceedings of International Conference on Robotics and Automation, May 23, 2022, pp. 6496-6503. [cited by applicant]
Chen, et al., “Representation Subspace Distance for Domain Adaptation Regression”, In Proceedings of the 38th International Conference on Machine Learning, Jul. 18, 2021, 11 Pages. [cited by applicant]
Chen, et al., “TensoRF: Tensorial Radiance Fields”, In Proceedings of European Conference on Computer Vision, Nov. 11, 2022, pp. 333-350. [cited by applicant]
Deng, et al., “Depth-Supervised NeRF: Fewer Views and Faster Training for Free”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 12882-12891. [cited by applicant]
Detone, et al., “SuperPoint: Self-Supervised Interest Point Detection and Description”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 18, 2018, pp. 337-349. [cited by applicant]
Dong, et al., “Visual Localization via Few-Shot Scene Region Classification”, In Repository of arXiv:2208.06933v1, Aug. 14, 2022, 12 Pages. [cited by applicant]
Dusmanu, et al., “D2-Net: A Trainable CNN for Joint Description and Detection of Local Features”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 16, 2019, pp. 8092-8101. [cited by applicant]
Fischler, et al., “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography”, In Journal of Communications of the ACM vol. 24, Issue 6, Jun. 1, 1981, pp. 381-39… [cited by applicant]
Gal, et al., “Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning”, In Proceedings of The 33rd International Conference on Machine Learning, Jun. 11, 2016, pp. 1050-1059. [cited by applicant]
Gao, et al., “Complete solution classification for the perspective-three-point problem”, In Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 25, Issue: 8, Aug. 4, 2003, pp. 930-943. [cited by applicant]
Garbin, et al., “FastNeRF: High-Fidelity Neural Rendering at 200FPS”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 14346-14355. [cited by applicant]
Georgakis, et al., “Learning to Map for Active Semantic Goal Navigation”, In Repository of arXiv:2106.15648v1, Jun. 29, 2021, 17 Pages. [cited by applicant]
Georgakis, et al., “Uncertainty-driven Planner for Exploration and Navigation”, In Repository of arXiv:2202.11907v1, Feb. 24, 2022, 8 Pages. [cited by applicant]
Guo, et al., “Neural 3D Scene Reconstruction With the Manhattan-World Assumption”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 5511-5520. [cited by applicant]
Harris, et al., “A Combined Corner And Edge Detector”, In Proceedings of Alvey Vision Conference, vol. 15, No. 50, Aug. 31, 1998, pp. 147-152. [cited by applicant]
Hedman, et al., “Deep blending for free-viewpoint image-based rendering”, In Journal of ACM Transactions on Graphics, vol. 37, Issue 6, Dec. 4, 2018, 15 Pages. [cited by applicant]
Huang, et al., “VS-Net: Voting With Segmentation for Visual Localization”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 19, 2021, pp. 6101-6111. [cited by applicant]
Ichnowski, et al., “Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects”, In Repository of arXiv:2110.14217v1, Oct. 27, 2021, 11 Pages. [cited by applicant]
Jain, et al., “Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles”, In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, No. 4, Apr. 3, 2020, pp. 4264-4271. [cited by applicant]
Kendall, et al., “Geometric Loss Functions for Camera Pose Regression With Deep Learning”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jul. 21, 2017, pp. 5974-5983. [cited by applicant]
Kendall, et al., “PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization”, In Proceedings of the IEEE International Conference on Computer Vision, Dec. 7, 2015, pp. 2938-2946. [cited by applicant]
Kendall, et al., “What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?”, In Journal of Advances in Neural Information Processing Systems, vol. 30, Dec. 4, 2017, 11 Pages. [cited by applicant]
Kingma, et al., “Adam: A Method for Stochastic Optimization”, In Repository of arXiv:1412.6980v6, Jun. 23, 2015, 15 Pages. [cited by applicant]
Kingma, et al., “Variational dropout and the local reparameterization trick”, In Journal of Advances in neural Information processing systems, vol. 28, Dec. 7, 2015, 9 Pages. [cited by applicant]
Kononenko, Igor, “Bayesian Neural Networks”, In Journal of Biological Cybernetics, vol. 61, Issue 5, Sep. 1989, pp. 361-370. [cited by applicant]
Lakshminarayanan, et al., “Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles”, In Journal of Advances in Neural Information Processing Systems, vol. 30, Dec. 4, 2017, 12 Pages. [cited by applicant]
Lee, et al., “Uncertainty Guided Policy for Active Robotic 3D Reconstruction Using Neural Radiance Fields”, In Journal of IEEE Robotics and Automation Letters, vol. 7, Issue: 4, Oct. 10, 2022, pp. 12070-12077. [cited by applicant]
Lepetit, et al., “EPnP: An Accurate O(n) Solution to the PnP Problem”, In International Journal of Computer Vision, vol. 81, No. 2, Jul. 19, 2008, pp. 155-166. [cited by applicant]
Levoy, et al., “Light Field Rendering”, In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, Aug. 1, 1996, pp. 31-42. [cited by applicant]
Li, et al., “Full-Frame Scene Coordinate Regression for Image-Based Localization”, In Repository of arXiv:1802.03237v1, Feb. 9, 2018, 9 Pages. [cited by applicant]
Wang, et al., “Image Quality Assessment: From Error Visibility to Structural Similarity”, In Journal of IEEE Transactions on Image Processing, vol. 13, Issue: 4, Apr. 13, 2004, pp. 600-612. [cited by applicant]
Wang, et al., “NeRF-: Neural Radiance Fields Without Known Camera Parameters”, In Repository of arXiv:2102.07064v2, Feb. 16, 2021, 10 Pages. [cited by applicant]
Wang, et al., “NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction”, In Journal of Advances in Neural Information Processing Systems, vol. 34, Dec. 6, 2021, pp. 1-13. [cited by applicant]
Wei, et al., “NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-View Stereo”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 5610-5619. [cited by applicant]
Xu, et al., “SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image”, In Repository of arXiv:2204.00928v1, Apr. 2, 2022, 18 Pages. [cited by applicant]
Yang, et al., “Camera Pose Estimation and Localization with Active Audio Sensing”, In Proceedings of European Conference on Computer Vision, Oct. 22, 2022, pp. 271-291. [cited by applicant]
Yariv, et al., “Volume Rendering of Neural Implicit Surfaces”, In Journal of Advances in Neural Information Processing Systems, vol. 34, Dec. 6, 2021, pp. 1-11. [cited by applicant]
Yu, et al., “PixelNeRF: Neural Radiance Fields From One or Few Images”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 19, 2021, pp. 4578-4587. [cited by applicant]
Yu, et al., “PlenOctrees for Real-Time Rendering of Neural Radiance Fields”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5752-5761. [cited by applicant]
Zhang, et al., “Learning to Detect Features in Texture Images”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 18, 2018, pp. 6325-6333. [cited by applicant]
Zhang, et al., “NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination”, In Journal of ACM Transactions on Graphics, vol. 40, Issue 6, Dec. 10, 2021, pp. 1-18. [cited by applicant]
Zhang, et al., “Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis”, In International Journal of Computer Vision, vol. 129, No. 4, Dec. 23, 2020, pp. 821-844. [cited by applicant]
Zhang, et al., “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric”, In Proceedings of the EEE Conference on Computer Vision and Pattern Recognition, Jun. 18, 2018, pp. 586-595. [cited by applicant]
Zhi, et al., “In-Place Scene Labelling and Understanding With Implicit Scene Representation”, In Proceedings of the EEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 15838-15847. [cited by applicant]
Zhou, et al., “Evaluating Local Features for Day-Night Matching”, In Proceedings of European Conference on Computer Vision, Nov. 24, 2016, pp. 724-736. [cited by applicant]
Zhou, et al., “KFNet: Learning Temporal Camera Relocalization Using Kalman Filtering”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 13, 2020, pp. 4919-4928. [cited by applicant]
International Search Report and Written Opinion received for PCT Application No. PCT/US2024/019329, Jun. 26, 2024, 16 pages. [cited by applicant]
Li, et al.,“Full-Frame Scene Coordinate Regression for Image-Based Localization,” arXiv preprint, Jun. 25, 2018, 9 Pages. [cited by applicant]
Moreau, et al., “LENS: Localization enhanced by NeRF synthesis,” Conference on Robot Learning, Oct. 13, 2021, pp. 1347-1356. [cited by applicant]
Roessle, et al., “Dense Depth Priors for Neural Radiance Fields from Sparse Input Views,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 12892-12901. [cited by applicant]
Shen, et al.,“Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification,” arXiv preprint, Feb. 9, 2018, pp. 540-557. [cited by applicant]
Li, et al., “Hierarchical Scene Coordinate Classification and Regression for Visual Localization”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 13, 2020, pp. 11983-11992. [cited by applicant]
Lin, et al., “BARF: Bundle-Adjusting Neural Radiance Fields”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 5741-5751. [cited by applicant]
Lindenberger, et al., “Pixel-Perfect Structure-From-Motion With Featuremetric Refinement”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 5987-5997. [cited by applicant]
Lowe, Davidg. , “Distinctive Image Features from Scale-Invariant Keypoints”, In International Journal of Computer Vision, vol. 60, Issue 2, Jan. 22, 2004, pp. 91-110. [cited by applicant]
Mackay, et al., “Bayesian Neural Networks and Density Networks”, In Journal of Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, vol. 354, No… [cited by applicant]
Massiceti, et al., “Random forests versus Neural Networks—What's best for camera localization?”, In Proceedings of International Conference on Robotics and Automation, May 29, 2017, pp. 5118-5125. [cited by applicant]
Max, et al., “Optical Models for Direct Volume Rendering”, In Journal of IEEE Transactions on Visualization and Computer Graphics, vol. 1, Issue 2, Jun. 1995, pp. 99-108. [cited by applicant]
Meng, et al., “Backtracking regression forests for accurate camera relocalization”, In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 24, 2017, pp. 6886-6893. [cited by applicant]
Meng, et al., “Exploiting Points and Lines in Regression Forests for RGB-D Camera Relocalization”, In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 1, 2018, pp. 6827-6834. [cited by applicant]
Mildenhall, et al., “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis”, In Journal of Communications of the ACM, vol. 65, Issue 1, Dec. 17, 2021, pp. 99-106. [cited by applicant]
Moreau, et al., “LENS: Localization Enhanced by NeRF Synthesis”, In Proceedings of the 5th Conference on Robot earning, Jan. 11, 2022, pp. 1347-1356. [cited by applicant]
Müller, et al., “Instant Neural Graphics Primitives with a Multiresolution Hash Encoding”, In Journal of ACM Transactions on Graphics, vol. 41, No. 4, Jul. 2022, 15 Pages. [cited by applicant]
Ng, et al., “Reassessing the Limitations of CNN Methods for Camera Pose Regression”, In Repository of arXiv:2108.07260v1, Aug. 16, 2021, pp. 1-15. [cited by applicant]
Pan, et al., “ActiveNeRF: Learning Where to See with Uncertainty Estimation”, In Proceedings of 17th European Conference on Computer Vision, Oct. 23, 2022, pp. 230-246. [cited by applicant]
Pathak, et al., “Self-Supervised Exploration via Disagreement”, In Proceedings of the 36th International Conference on Machine Learning, Jun. 9, 2019, 10 Pages. [cited by applicant]
Reiser, et al., “KiloNeRF: Speeding Up Neural Radiance Fields With Thousands of Tiny MLPs”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 14335-14345. [cited by applicant]
Rematas, et al., “Urban Radiance Fields”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 12932-12942. [cited by applicant]
Riegler, et al., “Free View Synthesis”, In Proceedings of European Conference on Computer Vision, Nov. 13, 2020, pp. 623-640. [cited by applicant]
Roessle, et al., “Dense Depth Priors for Neural Radiance Fields From Sparse Input Views”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 12892-12901. [cited by applicant]
Rosinol, et al., “NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields”, In Repository of arXiv:2210.13641v1, Oct. 24, 2022, 10 Pages. [cited by applicant]
Sarlin, et al., “Back to the Feature: Learning Robust Camera Localization From Pixels To Pose”, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Jun. 19, 2021, pp. 3247-3257. [cited by applicant]
Sarlin, et al., “From Coarse to Fine: Robust Hierarchical Localization at Large Scale”, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 16, 2019, pp. 12716-12725. [cited by applicant]
Sarlin, et al., “SuperGlue: Learning Feature Matching With Graph Neural Networks”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 13, 2020, pp. 4938-4947. [cited by applicant]
Sattler, et al., “Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization”, In Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 39, Issue 9, Sep. 20, 2016, pp. 1… [cited by applicant]
Sattler, et al., “Image Retrieval for Image-Based Localization Revisited”, In Journal of BMVC, vol. 1, No. 2, Sep. 3, 2012, pp. 1-12. [cited by applicant]
Sattler, et al., “Improving Image-Based Localization by Active Correspondence Search”, In Proceedings of European Conference on Computer Vision, Oct. 7, 2012, pp. 752-765. [cited by applicant]
Sattler, et al., “Understanding the Limitations of CNN-Based Absolute Camera Pose Regression”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 16, 2019, pp. 3302-3312. [cited by applicant]
Seitzer, et al., “On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks”, In Repository of arXiv:2203.09168v1, Mar. 17, 2022, pp. 1-24. [cited by applicant]
Sensoy, et al., “Evidential Deep Learning to Quantify Classification Uncertainty”, In Journal of Advances in Neural Information Processing Systems, vol. 31, Dec. 3, 2018, pp. 1-11. [cited by applicant]
Seung, et al., “Query by Committee”, In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, Jul. 1, 1992, pp. 287-294. [cited by applicant]
Shavit, et al., “Learning Multi-Scene Absolute Pose Regression With Transformers”, In Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct. 10, 2021, pp. 2733-2742. [cited by applicant]
Shen, et al., “Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification”, In Proceedings of Computer Vision-ECCV, 17th European Conference, Oct. 23, 2022, pp. 540-557. [cited by applicant]
Shen, et al., “Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations”, In Proceedings of International Conference on 3D Vision, Dec. 1, 2021, pp. 972-981. [cited by applicant]
Shotton, et al., “Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 23, 2013, pp. 2930-2937. [cited by applicant]
Simonyan, et al., “Very Deep Convolutional Networks for Large-Scale Image Recognition”, In Repository of arXiv:1409.1556v6, Apr. 10, 2015, 14 Pages. [cited by applicant]
Simonyan, et al., “Very Deep Convolutional Networks for Large-Scale Image Recognition”, In Repository of arXiv:1409.1556v1, Sep. 4, 2014, pp. 1-10. [cited by applicant]
Srinivasan, et al., “NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 19, 2021, pp. 7495-7504. [cited by applicant]
Straub, et al., “The Replica Dataset: A Digital Replica of Indoor Spaces”, In Repository of arXiv:1906.05797v1, Jun. 13, 2019, 10 Pages. [cited by applicant]
Sucar, et al., “iMAP: Implicit Mapping and Positioning in Real-Time”, In Proceedings of Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 6229-6238. [cited by applicant]
Sun, “Direct Voxel Grid Optimization: Super-Fast Convergence for Radiance Fields Reconstruction”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 5459-5469. [cited by applicant]
Sun, et al., “LoFTR: Detector-Free Local Feature Matching With Transformers”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 19, 2021, pp. 8922-8931. [cited by applicant]
Taira, et al., “InLoc: Indoor Visual Localization With Dense Matching and View Synthesis”, In Proceedings of the EEE Conference on Computer Vision and Pattern Recognition, Jun. 18, 2018, pp. 7199-7209. [cited by applicant]
Tancik, et al., “Block-NeRF: Scalable Large Scene Neural View Synthesis”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 8248-8258. [cited by applicant]
Tang, et al., “Learning Camera Localization via Dense Scene Matching”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 19, 2021, pp. 1831-1841. [cited by applicant]
Torii, et al., “24/7 Place Recognition by View Synthesis”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 7, 2015, pp. 1808-1817. [cited by applicant]
Turki, et al., “Mega-NERF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs”, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 18, 2022, pp. 12922-12931. [cited by applicant]
Valentin, et al., “Exploiting Uncertainty in Regression Forests for Accurate Camera Relocalization”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 7, 2015, pp. 4400-4408. [cited by applicant]
Valentin, et al., “Learning to Navigate the Energy Landscape”, In Proceedings of Fourth International Conference on BD Vision (3DV), Oct. 25, 2016, pp. 323-332. [cited by applicant]
Walch, et al., “Image-Based Localization Using LSTMs for Structured Feature Correlation”, In Proceedings of the EEE International Conference on Computer Vision, Oct. 22, 2017, pp. 627-637. [cited by applicant]
Wang, et al., “AtLoc: Attention Guided Camera Localization”, In Proceedings of the AAAI Conference on Artificial Intelligence, Apr. 3, 2020, pp. 10393-10401. [cited by applicant]
International Preliminary Report on Patentability (Chapter I) received for PCT Application No. PCT/US2024/019329, mailed on Oct. 9, 2025, 11 pages. [cited by applicant]