IP Library › Granted Patent US 12711758
Granted Patent B2
US 12711758 · App. 18/431,913 · Granted Aug 18, 2026

Runtime ranking of object detection

Inventors: Dongdong Chen (Redmond, WA); Yunsheng Li (Sammamish, WA); Lu Yuan (Redmond, WA); Xuelu Feng (Buffalo, NY)
Assignee: Microsoft Technology Licensing, LLC
G06V10/993G06V10/771
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12711758
App. No.
18/431,913
Granted
Aug 18, 2026
Kind
B2
Abstract

Example solutions for ranking object detection results generate or receive a plurality of segmentation masks each corresponding to one or more images. Each segmentation mask of each plurality of segmentation masks is generated using a different object detector or setting options. A quality predictor assigns a quality score to each segmentation mask, without using ground truth for the image(s). A set (one or more, but less than all) of the highest quality scores is identified for each image. In some examples, an image processing task is performed using the segmentation masks having an assigned quality score that is within the set of highest quality scores. In some examples, only the segmentation mask having the highest quality score for an image is used in the image processing task. In some examples, a quality threshold is provided, and the segmentation masks meeting the quality threshold are used in the image processing task.

Claims (52)

1 . A system comprising:

a processor; and

a computer-readable medium storing instructions that are operative upon execution by the processor to:

receive an image, the image including an object;

generate, using a first object detector having a first set of operational settings, a first segmentation mask corresponding to the object;

generate, using a second object detector, the second object detector different than the first object detector, the second object detector having a second set of operational settings different from the first set of operational settings, a second segmentation mask corresponding to the object;

cause a quality predictor to assign, a first quality score to the first segmentation mask, wherein the quality predictor comprises a multi-stage vision transformer model having multiple sequential transformer stages and a query head;

cause the quality predictor to assign, a second quality score to the second segmentation mask based on paired input of the image and the second segmentation mask, wherein the query head outputs a learned Q-vector representing predicted segmentation-mask quality, the quality predictor having been trained using human-provided segmentation-mask quality scores independent of object identity or object count;

based on at least the first quality score exceeding the second quality score and the first quality score exceeding a user-specified quality threshold, select the first segmentation mask for an image processing task; and

perform the image processing task using the first segmentation mask.

2 . The system of claim 1 , wherein selecting the first segmentation mask for the image processing task comprises determining that the first quality score meets a quality threshold and wherein not selecting the second segmentation mask for the image processing task comprises determining that the second quality score does not meet the quality threshold.

3 . The system of claim 1 , wherein causing the quality predictor to assign the first quality score and causing the quality predictor to assign the second quality score are performed contemporaneously with performing the image processing task.

4 . The system of claim 1 , wherein the instructions are further operative to:

generate the first segmentation mask using the first object detector; and

generate the second segmentation mask using the second object detector.

5 . The system of claim 4 , wherein generating the first segmentation mask and generating the second segmentation mask is-are performed contemporaneously with causing the quality predictor to assign the first quality score and causing the quality predictor to assign the second quality score.

6 . The system of claim 1 , wherein the instructions are further operative to:

receive a plurality of training images, a plurality of segmentation masks corresponding to the plurality of training images, and a plurality of quality scores associated with each segmentation mask and training image; and

using the plurality of training images, the plurality of segmentation masks, and the plurality of quality scores, cause the quality predictor to assign quality scores to segmentation masks based on an input image, without needing ground truth for the input image.

7 . A computer-implemented method comprising:

receiving an image, the image including an object;

generating, using a first object detector having a first set of operational settings, a first segmentation mask corresponding to the object;

generating, using a second object detector, the second object detector different than the first object detector, the second object detector having a second set of operational settings different from the first set of operational settings, a second segmentation mask corresponding to the object;

causing a quality predictor to assign a first quality score to the first segmentation mask wherein the quality predictor comprises a multi-stage vision transformer model having multiple sequential transformer stages and a query head;

causing the quality predictor to assign a second quality score to the second segmentation mask based on paired input of the image and the second segmentation mask, wherein the query head outputs a learned Q-vector representing predicted segmentation-mask quality, the quality predictor having been trained using human-provided segmentation-mask quality scores independent of object identity or object count;

based on at least the first quality score exceeding the second quality score and the first quality score exceeding a user-specified quality threshold, selecting the first segmentation mask for an image processing task; and

performing the image processing task using the first segmentation mask.

8 . The computer-implemented method of claim 7 , wherein selecting the first segmentation mask for the image processing task comprises determining that the first quality score meets a quality threshold and wherein not selecting the second segmentation mask for the image processing task comprises determining that the second quality score does not meet the quality threshold.

9 . The computer-implemented method of claim 7 , wherein assigning the first quality score and the second quality score is performed contemporaneously with performing the image processing task.

10 . The computer-implemented method of claim 7 , further comprising:

generating the first segmentation mask using the first object detector; and

generating the second segmentation mask using the second object detector.

11 . The computer-implemented method of claim 10 , wherein generating the first segmentation mask and the second segmentation mask is performed contemporaneously with assigning the first quality score and the second quality score.

12 . The computer-implemented method of claim 7 , further comprising:

receiving a plurality of training images, a plurality of segmentation masks corresponding to the plurality of training images, and a plurality of quality scores associated with each segmentation mask and training image; and

using the plurality of training images, the plurality of segmentation masks, and the plurality of quality scores, causing the quality predictor to assign quality scores to segmentation masks based on an input image, without needing ground truth for the input image.

13 . A computer storage device having computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising:

receiving an image, the image including an object;

generating, using a first object detector having a first set of operational settings, a first segmentation mask corresponding to the object;

generating, using a second object detector, the second object detector different than the first object detector, the second object detector having a second set of operational settings different from the first set of operational settings, a second segmentation mask corresponding to the object;

causing a quality predictor to assign a first quality score to the first segmentation mask, wherein the quality predictor comprises a multi-stage vision transformer model having multiple sequential transformer stages and a query head;

causing the quality predictor to assign a second quality score to the second segmentation mask based on paired input of the image and the second segmentation mask, wherein the query head outputs a learned Q-vector representing predicted segmentation-mask quality, the quality predictor having been trained using human-provided segmentation-mask quality scores independent of object identity or object count;

based on at least the first quality score exceeding the second quality score and the first quality score exceeding a user-specified quality threshold, selecting the first segmentation mask for an image processing task; and

performing the image processing task using the first segmentation mask.

14 . The computer storage device of claim 13 , wherein is causing the quality predictor to assign the first quality score and causing the quality predictor to assign the second quality score are performed contemporaneously with performing the image processing task.

15 . The computer storage device of claim 13 , wherein the operations further comprise:

generating the first segmentation mask and generating the second segmentation mask contemporaneously with causing the quality predictor to assign the first quality score and the second quality score.

16 . The computer storage device of claim 15 , wherein the operations further comprise:

receiving a plurality of training images, a plurality of segmentation masks corresponding to the plurality of training images, and a plurality of quality scores associated with each segmentation mask and training image; and

using the plurality of training images, the plurality of segmentation masks, and the plurality of quality scores, causing the quality predictor to assign quality scores to segmentation masks based on an input image, without needing ground truth for the input image.

17 . The computer storage device of claim 13 , wherein to select the first segmentation mask for the image processing task comprises determining that the first quality score meets a quality threshold and wherein to not select the second segmentation mask for the image processing task comprises determining that the second quality score does not meet the quality threshold.

18 . The computer storage device of claim 13 , wherein causing the quality predictor to assign the first quality score and causing the quality predictor to assign the second quality score are performed contemporaneously with performing the image processing task.