Anomaly detection using neural radiance fields
Detecting data anomalies by receiving a query image, determining a query image viewpoint according to a trained neural radiance field model, generating a 2D reference image according to the neural radiance field model, determining a difference between the query image and the reference image, and highlighting the difference in a presentation of the query image.
1 . A computer implemented method for detecting data anomalies, the method comprising:
receiving a query image;
determining a query image viewpoint according to a trained model by minimizing rendering loss between weights of the trained model and the query image;
generating a two-dimensional reference image according to the query image viewpoint determined by the model;
determining a difference between the query image and the reference image; and
highlighting the difference in a presentation of the query image.
2 . The computer implemented method according to claim 1 , further comprising:
receiving a of set images;
training a neural radiance field model according to the set of images; and
providing the trained neural radiance field model.
3 . The computer implemented method according to claim 2 , further comprising freezing the weights of the neural radiance field model.
4 . The computer implemented method according to claim 1 , wherein determining the query image viewpoint comprises optimizing the neural radiance field model for the query image viewpoint.
5 . The computer implemented method according to claim 1 , wherein determining the difference between the query image and the reference image comprising comparing query image and reference image pixels.
6 . The computer implemented method according to claim 1 , further comprising
defining a query image depth map;
defining a reference image depth map;
determining a difference between the query image depth map and the reference image depth map; and
presenting details about the difference to a user.
7 . The computer implemented method according to claim 6 , further comprising determining the difference using a red-green-blue-depth camera.
8 . A computer program product for detecting anomalies, the computer program product comprising one or more computer readable storage media and collectively stored program instructions on the one or more computer readable storage media, the stored program instructions which, when executed, cause one or more computer systems to:
receive a query image;
determine a query image viewpoint according to a trained neural radiance field model by minimizing rendering loss between weights of the trained neural radiance field model and the query image;
generate a two-dimensional reference image according to the query image viewpoint determined by the model neural radiance field model;
determine a difference between the query image and the reference image; and
highlight the difference in a presentation of the query image.
9 . The computer program product according to claim 8 , the computer program instructions further causing the one or more computer systems to:
receive a set of images;
train a neural radiance field model according to the set of images; and
provide the trained neural radiance field model.
10 . The computer program product according to claim 9 , the computer program instructions further causing the one or more computer systems to freeze the weights of the neural radiance field model.
11 . The computer program product according to claim 8 , wherein determining the query image viewpoint comprises optimizing the neural radiance field model for the query image viewpoint.
12 . The computer program product according to claim 8 , wherein determining the difference between the query image and the reference image comprising comparing query image and reference image pixels.
13 . The computer program product according to claim 8 , the computer program instructions further causing the one or more computer systems to:
define a query image depth map;
define a reference image depth map;
determine a difference between the query image depth map and the reference image depth map; and
present details about the difference to a user.
14 . The computer program product according to claim 13 , further comprising determining the difference using a red-green-blue-depth camera.
15 . A computer system for detecting data anomalies, the computer system comprising:
one or more computer processors;
one or more computer readable storage devices; and
stored program instructions on the one or more computer readable storage devices for execution by the one or more computer processors, the stored program instructions which, when executed, cause the one or more computer processors to:
receive a query image;
determine a query image viewpoint according to a trained neural radiance field model by minimizing rendering loss between weights of the trained neural radiance field model and the query image;
generate a two-dimensional reference image according to the query image viewpoint determined by the model neural radiance field model;
determine a difference between the query image and the reference image; and
highlight the difference in a presentation of the query image.
16 . The computer system according to claim 15 , the computer program instructions further causing the one or more processors to:
receive a set of images;
train a neural radiance field model according to the set of images; and
provide the trained neural radiance field model.
17 . The computer system according to claim 16 , the computer program instructions further causing the one or more processors to freeze the weights of the neural radiance field model.
18 . The computer system according to claim 15 , wherein determining the query image viewpoint comprises optimizing the neural radiance field model for the query image viewpoint.
19 . The computer system according to claim 15 , wherein determining the difference between the query image and the reference image comprising comparing query image and reference image pixels.
20 . The computer system according to claim 15 , the computer program instructions further causing the one or more processors to:
define a query image depth map;
define a reference image depth map;
determine a difference between the query image depth map and the reference image depth map; and
present details about the difference to a user.