Granted Patent B2

US 12,354,331 · App. 17/647,313 · Granted Jul 8, 2025

Inventors: Christian Heinzemann (Ludwigsburg, DE); Christoph Gladisch (Renningen, DE); Matthias Woehrle (Bietigheim-Bissingen, DE); Ulrich Seger (Leonberg-Warmbronn, DE)

Assignee: ROBERT BOSCH GMBH

G06V10/774G06F3/0484G06F18/214G06N20/00G06T7/35G06V10/82G06V20/56

Abstract

Claims (78)

1. A computer-implemented method for analysing relevance of visual parameters for training a computer vision model, the method comprising the following steps:

obtaining the computer vision model in an initial training state configured to perform a computer vision function of characterizing elements of observed scenes;

obtaining a visual data set and selecting from the visual data set a first subset of items of visual data, and providing a first subset of groundtruth data that corresponds to the first subset of visual data;

obtaining a first visual parameter set, with at least one visual parameter therein defining at least one visual state of at least one item in the first subset of visual data, wherein the at least one visual state is capable of affecting a regression result of the computer vision model;

applying the first subset of items of visual data to the computer vision model to obtain a plurality of predictions of elements of observed scenes in the first subset of items of visual data, wherein the predictions include at least one regression result of the at least one item in the first subset of visual data;

computing a corresponding plurality of performance scores of the first visual parameter set characterizing accuracy of the computer vision model when providing the predictions of the at least one regression result, using the first subset of groundtruth data;

performing a sensitivity analysis of the plurality of performance scores over a domain of the first visual parameter set;

generating a second subset of items of visual data and a second subset of groundtruth data that corresponds to the second subset of visual data according to the sensitivity analysis of the plurality of performance scores over the domain of the first visual parameter set.

2. The computer-implemented method according to claim 1 , wherein the sensitivity analysis of the plurality of performance scores over the domain of the first visual parameter set includes computing a plurality of variances of performance scores over the domain of the first visual parameter set.

3. The computer-implemented method according to claim 2 , wherein from the first visual parameter set, one or more visual parameters of the first visual data set having corresponding variances of performance scores that are below a corresponding significance threshold are removed.

4. The computer-implemented method according to claim 2 , further comprising:

displaying, via an output interface, the plurality of variances of performance scores over the domain of the first visual parameter set to a user, via a user interface.

5. The computer-implemented method according to claim 1 , further comprising:

modifying the first visual parameter set by adding, or removing, or redefining at least one visual parameter so as to increase an overall measure of variance of performance scores, thus yielding a second visual parameter set.

6. The computer-implemented method according to claim 1 ,

wherein the generating the second subset of items of visual data and the second subset of groundtruth data includes sampling further items of visual data from the visual data set and corresponding groundtruth and/or synthesizing further items of visual data and corresponding groundtruth, each according to the second visual parameter set.

7. The computer-implemented method according to claim 1 , further comprising:

generating a ranked list of visual parameters of the first visual parameter set according to the sensitivity analysis of the plurality of performance scores, thus yielding a first ordered visual parameter set;

wherein the ranked list of visual parameters of the first visual parameter set is provided according to the plurality of computed variances of respective performance scores of the plurality of performance scores with respect to a plurality of visual parameters.

8. The computer-implemented method according to claim 7 , further comprising:

displaying the ranked list of visual parameters of the first visual parameter set;

receiving, via an input interface, a command to modify the first visual parameter set by removing at least one visual parameter in the first visual parameter set, and/or by changing the ranked list of at least one visual parameter in the first visual parameter set, to thus yield a user modified visual parameter set.

9. The computer-implemented method according to claim 8 , wherein the second subset of items of visual data and the second subset of groundtruth data are assembled according to the user modified visual parameter set.

10. The computer-implemented method according to claim 1 , wherein the domain of the first visual parameter set includes a subset, in a finite-dimensional vector space, of numerical representations in which visual parameters are allowed.

11. The computer-implemented method according to claim 10 , wherein the finite-dimensional space is a multi-dimensional interval of continuous or discrete visual parameters.

12. The computer-implemented method according to claim 1 , wherein the domain includes a set of numerical representations of visual parameters in a finite-dimensional vector space.

13. The computer-implemented method according to claim 1 , wherein the performance scores include, or are based on, any one or combination, of: a confusion matrix, or a precision score, or a recall score, or an F1 score, or a union intersection score, or a mean average score.

14. A computer-implemented method for training a computer vision model, comprising the following steps:

obtaining the computer vision model in an initial training state configured to perform a computer vision function of characterizing elements of observed scenes;

obtaining a visual data set and selecting from the visual data set a first subset of items of visual data, and providing a first subset of groundtruth data that corresponds to the first subset of visual data;

obtaining a first visual parameter set, with at least one visual parameter therein defining at least one visual state of at least one item in the first subset of visual data, wherein the at least one visual state is capable of affecting a regression result of the computer vision model;

applying the first subset of items of visual data to the computer vision model to obtain a plurality of predictions of elements of observed scenes in the first subset of items of visual data, wherein the predictions include at least one regression result of the at least one item in the first subset of visual data;

computing a corresponding plurality of performance scores of the first visual parameter set characterizing accuracy of the computer vision model when providing the predictions of the at least one regression result, using the first subset of groundtruth data;

performing a sensitivity analysis of the plurality of performance scores over a domain of the first visual parameter set;

generating, as a training data set, a second subset of items of visual data and a second subset of groundtruth data that corresponds to the second subset of visual data according to the sensitivity analysis of the plurality of performance scores over the domain of the first visual parameter set;

iteratively training a further computer vision model using the training data set;

outputting the further computer vision model.

15. An apparatus for performing computer vision, comprising:

an input interface;

a processor;

a memory; and

an output interface;

wherein the input interface is configured to receive a further visual data set and a further computer vision model trained by:

obtaining a computer vision model in an initial training state configured to perform a computer vision function of characterizing elements of observed scenes,

obtaining a visual data set and selecting from the visual data set a first subset of items of visual data, and providing a first subset of groundtruth data that corresponds to the first subset of visual data,

obtaining a first visual parameter set, with at least one visual parameter therein defining at least one visual state of at least one item in the first subset of visual data, wherein the at least one visual state is capable of affecting a regression result of the computer vision model,

applying the first subset of items of visual data to the computer vision model to obtain a plurality of predictions of elements of observed scenes in the first subset of items of visual data, wherein the predictions include at least one regression result of the at least one item in the first subset of visual data,

computing a corresponding plurality of performance scores of the first visual parameter set characterizing accuracy of the computer vision model when providing the predictions of at least one regression result, using the first subset of groundtruth data,

performing a sensitivity analysis of the plurality of performance scores over a domain of the first visual parameter set,

generating, as a training data set, a second subset of items of visual data and a second subset of groundtruth data that corresponds to the second subset of visual data according to the sensitivity analysis of the plurality of performance scores over the domain of the first visual parameter set,

iteratively training the further computer vision model using the training data set,

outputting the further computer vision model;

wherein the processor is configured to read the further visual data set and the further computer vision model from the memory, and to apply the further visual data set to the further computer vision model, to thus generate the at least one regression result; and

wherein the output interface is configured to output the at least one regression result.

16. A non-transitory computer readable medium on which is stored a computer program including machine-readable instructions for training a computer vision model, the machine readable instructions, when executed by a processor, causing the processor to perform the following steps:

obtaining the computer vision model in an initial training state configured to perform a computer vision function of characterizing elements of observed scenes;

obtaining a visual data set and selecting from the visual data set a first subset of items of visual data, and providing a first subset of groundtruth data that corresponds to the first subset of visual data;

obtaining a first visual parameter set, with at least one visual parameter therein defining at least one visual state of at least one item in the first subset of visual data, wherein the at least one visual state is capable of affecting a regression result of the computer vision model;

applying the first subset of items of visual data to the computer vision model to obtain a plurality of predictions of elements of observed scenes in the first subset of items of visual data, wherein the predictions include at least one regression result of the at least one item in the first subset of visual data;

computing a corresponding plurality of performance scores of the first visual parameter set characterizing accuracy of the computer vision model when providing the predictions of the at least one regression result, using the first subset of groundtruth data;

performing a sensitivity analysis of the plurality of performance scores over a domain of the first visual parameter set;

generating, as a training data set, a second subset of items of visual data and a second subset of groundtruth data that corresponds to the second subset of visual data according to the sensitivity analysis of the plurality of performance scores over the domain of the first visual parameter set;

iteratively training a further computer vision model using the training data set;

outputting the further computer vision model.

17. A distributed data communications system, comprising:

a data processing agent;

a communications network; and

a terminal device;

wherein the data processing agent is configured to transmit a further computer vision model to the terminal device via the communications network, the further computer vision model being trained by:

obtaining a computer vision model in an initial training state configured to perform a computer vision function of characterizing elements of observed scenes,

obtaining a visual data set and selecting from the visual data set a first subset of items of visual data, and providing a first subset of groundtruth data that corresponds to the first subset of visual data,

obtaining a first visual parameter set, with at least one visual parameter therein defining at least one visual state of at least one item in the first subset of visual data, wherein the at least one visual state is capable of affecting a regression result of the computer vision model,

applying the first subset of items of visual data to the computer vision model to obtain a plurality of predictions of elements of observed scenes in the first subset of items of visual data, wherein the predictions include at least one regression result of the at least one item in the first subset of visual data,

computing a corresponding plurality of performance scores of the first visual parameter set characterizing accuracy of the computer vision model when providing the predictions of the-at least one regression result, using the first subset of groundtruth data,

performing a sensitivity analysis of the plurality of performance scores over a domain of the first visual parameter set,

generating, as a training data set, a second subset of items of visual data and a second subset of groundtruth data that corresponds to the second subset of visual data according to the sensitivity analysis of the plurality of performance scores over the domain of the first visual parameter set, and

iteratively training the further computer vision model using the training data set.

18. The distributed data communications system according to claim 17 , wherein the terminal is within an automobile or a robot.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (11)