Adversarial masks for scene-customized false detection removal
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving multiple images from a camera, each image of the multiple images representative of a detection of an object within the image. For each image of the multiple images the methods include: determining a set of detected objects within the image, each object defined by a respective bounding box, and determining, from the set of detected objects within the image and ground truth labels, a false detection of a first object. The methods further include determining that a target object threshold is met based on a number of false detections of the first object in the multiple images, generating, based on the number of false detections for the first object meeting the target object threshold, an adversarial mask for the first object, and providing, to the camera, the adversarial mask.
1. A computer-implemented method comprising:
accessing a set of target regions for a camera;
accessing a set of images that were captured by the camera;
initializing, for a first image of the set of images, an adversarial mask including a first value for an adversarial mask parameter for the set of target regions;
generating, for a second image of the set of images, an updated adversarial mask, the generating including:
applying the adversarial mask to the second image, and
updating the first value to a second value for the adversarial mask parameter using a result of applying the adversarial mask to the second image; and
providing, to the camera, the updated adversarial mask for use by an object detection model.
2. The method of claim 1 , comprising accessing data representing the set of target regions that indicate regions within a field of view of the camera in which there was a false positive detection.
3. The method of claim 1 , comprising detecting, from a plurality of regions within a field of view of the camera, the set of target regions that are associated with false positive detections.
4. The method of claim 1 , comprising:
detecting, in a first image, a first bounding box for a first object that was a false positive detection;
detecting, in a second image, a second bounding box for a second object that was a true positive detection;
determining that the first bounding box and the second bounding box overlap in an image region represented by the same coordinates in the respective images; and
assigning the image region as a target region in the set of target regions that are associated with false positive detections.
5. The method of claim 1 , wherein initializing the adversarial mask comprises determining an altered version of the first image.
6. The method of claim 1 , wherein initializing the adversarial mask including the first value for the adversarial mask parameter comprises:
performing a forward pass on the adversarial mask using a fast gradient sign method;
performing a backward pass on the adversarial mask using the fast gradient sign method;
determining a sign of a gradient of a loss function associated with an object detection model; and
using the sign of the gradient as an initialization value of a noise mask learnable parameter for the adversarial mask.
7. The method of claim 1 , wherein applying the adversarial mask to the second image comprises:
performing a forward pass on the second image using a fast gradient sign method;
performing a backward pass on the second image using the fast gradient sign method;
determining, using the forward and backward passes, an object detection loss; and
determining, using the object detection loss, a noise mask learnable parameter of the updated adversarial mask.
8. A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
accessing by the one or more computers, a set of target regions for a camera;
accessing, by the one or more computers, a set of images from the camera;
initializing, for a first image of the set of images, an adversarial mask including a first value for an adversarial mask parameter for the set of target regions;
generating, for a second image of the set of images, an updated adversarial mask, the generating including:
applying the adversarial mask to the second image, and
updating the first value to a second value for the adversarial mask parameter using a result of applying the adversarial mask to the second image; and
providing, to the camera, the updated adversarial mask for use by an object detection model.
9. The system of claim 8 , wherein the operations comprise accessing data representing the set of target regions that indicate regions within a field of view of the camera that are associated with false positive detections.
10. The system of claim 8 , wherein the operations comprise detecting, from a plurality of regions within a field of view of the camera, the set of target regions that are associated with false positive detections.
11. The system of claim 8 , wherein:
the set of images comprise at least a first image that includes a first representation of a first object and a second representation of a second object,
a first bounding box for the first representation of the first object overlaps a second bounding box for the second representation of the second object,
and the second object is an object of interest.
12. The system of claim 8 , wherein initializing the adversarial mask comprises determining an altered version of the first image.
13. The system of claim 8 , wherein initializing the adversarial mask including a first adversarial mask parameter comprises:
performing a forward pass on the adversarial mask using a fast gradient sign method;
performing a backward pass on the adversarial mask using the fast gradient sign method;
determining a sign of a gradient of a loss function associated with an object detection model; and
using the sign of the gradient as an initialization value of a noise mask learnable parameter for the adversarial mask.
14. The system of claim 8 , wherein applying the adversarial mask to the second image comprises:
performing a forward pass on the second image using a fast gradient sign method;
performing a backward pass on the second image using the fast gradient sign method;
determining, using the forward and backward passes, an object detection loss; and
determining, using the object detection loss, a noise mask learnable parameter of the updated adversarial mask.
15. The system of claim 8 , wherein the operations comprise:
detecting, in a first image, a first bounding box for a first object that was a false positive detection;
detecting, in a second image, a second bounding box for a second object that was a true positive detection;
determining that the first bounding box and the second bounding box overlap in an image region represented by the same coordinates in the respective images; and
assigning the image region as a target region in the set of target regions that are associated with false positive detections.
16. One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform the following operations:
accessing by the one or more computers, a set of target regions for a camera;
accessing, by the one or more computers, a set of images from the camera;
initializing, for a first image of the set of images, an adversarial mask including a first value for an adversarial mask parameter for the set of target regions;
generating, for a second image of the set of images, an updated adversarial mask, the generating including:
applying the adversarial mask to the second image, and
updating the first value to a second value for the adversarial mask parameter using a result of applying the adversarial mask to the second image; and
providing, to the camera, the updated adversarial mask for use by an object detection model.
17. The non-transitory computer storage media of claim 16 , wherein the operations comprise accessing data representing the set of target regions that indicate regions within a field of view of the camera that are associated with false positive detections.
18. The non-transitory computer storage media of claim 16 , wherein the operations comprise detecting, from a plurality of regions within a field of view of the camera, the set of target regions that are associated with false positive detections.
19. The non-transitory computer storage media of claim 16 , wherein:
the set of images comprise at least a first image that includes a first representation of a first object and a second representation of a second object,
a first bounding box for the first representation of the first object overlaps a second bounding box for the second representation of the second object,
and the second object is an object of interest.
20. The non-transitory computer storage media of claim 16 , wherein initializing the adversarial mask comprises determining an altered version of the first image.