Method for validation of obstacle candidate

1. A method for validation of an obstacle candidate identified within a sequence of image frames, the method comprising the following steps:

A) for a current image frame of the sequence of image frames:

determining within the current image frame a region of interest representing the obstacle candidate;

dividing the region of interest into sub-regions; and

for each sub-region, determining a Time-To-Contact (TTC) based on at least the current image frame and a preceding or succeeding image frame of the sequence of image frames;

B) determining one or more classification features based on the TTCs of the sub-regions determined for the current image frame; and

C) classifying the obstacle candidate based on the determined one or more classification features, as to whether the obstacle candidate is valid, not valid, or validity of the obstacle candidate is unclear.

2. The method in accordance with claim 1 , wherein:

the step A) is iterated, with, in a first iteration, the current image frame being a first image frame of the sequence of image frames and with, in each subsequent iteration, the current image frame being an image frame succeeding the current image frame of a respective previous iteration; and

the step B) is based on averaged TTCs of the sub-regions, the averaged TTCs being obtained for each sub-region by averaging the TTCs of the sub-region over a predetermined number of iterations.

3. The method in accordance with claim 1 , wherein determining of the region of interest comprises tracking the region of interest representing the obstacle candidate through the sequence of image frames up to at least the current frame.

4. The method in accordance with claim 1 , wherein dividing of the region of interest comprises dividing the region of interest along a vertical axis into a predetermined number of sub-regions of equal width which are regularly offset from each other with mutual overlap.

5. The method in accordance with claim 1 , wherein determining the TTC of a sub-region comprises:

determining a change of size of the sub-region between the current image frame and the preceding or succeeding image frame;

wherein step of the determining the TTC of a sub-region comprises

determining interest points of the sub-region;

for each interest point, determining a change of position of the interest point between the current image frame and the preceding or succeeding image frame; and

deriving the change of size of the sub-region between the current image frame and the preceding or succeeding image frame from at least the determined changes of positions of the interest points.

6. The method in accordance with claim 5 ,

wherein determining of interest points of a sub-region comprises selecting, in the current image frame, from a set of interest points those interest points that are contained in the sub-region; and

wherein the set of interest points is obtained by:

in a first image frame of the sequence of image frames, defining an initial set of interest points based on the points of an equidistant grid applied to the region of interest;

for each subsequent image frame of the sequence of image frames, modifying the set of interest points based on tracking the interest points through the sequence of image frames; and

when the tracking an interest point leads to an invalid result, deleting the interest point from the set of interest points and replacing the deleted interest point with a new interest point.

7. The method in accordance with claim 6 , wherein the new interest point is defined by:

selecting one of the sub-regions, preferring those containing fewer interest points over those containing more interest points;

randomly selecting a point of the sub-region;

if the point does not comply with one or more predetermined suitability conditions, repeating the steps A), B), or C) until the point does comply with the one or more predetermined suitability conditions; and

defining the last selected point as the new interest point.

8. The method in accordance with claim 5 , wherein the step of determining of the change of position an interest point comprises tracking the interest point through the sequence of image frames up to at least the current or succeeding image frame.

9. The method in accordance with claim 5 , wherein the step of deriving of the change of size a sub-region comprises:

determining for the current image frame a first center position of the sub-region and a first set of relative coordinates of the interest points of the sub-region, the relative coordinates being defined relative to the first center position;

determining for the preceding or succeeding image frame a second center position of the sub-region and a second set of relative coordinates of the interest points of the sub-region, the relative coordinates being defined relative to the second center position; and

deriving the change of size of the sub-region from at least a relation between the first set of relative coordinates and the second set of relative coordinates.

10. The method in accordance with claim 6 , wherein the step of tracking a region or a point comprises:

in a first image frame of the sequence of image frames, defining at least one template based on the tracked region or on a region surrounding the tracked point, respectively; and

in at least one of the subsequent image frames of the sequence of image frames, determining a region which corresponds best to the at least one template.

11. The method in accordance with claim 1 , wherein the step of determining of one or more classification features in step B) comprises:

determining, for each sub-region, at least one classification feature that based on, a difference between the determined TTC of the sub-region and a TTC of an entire region of interest determined for the current image frame; and

the step of determining the TTC of the entire region of interest comprises determining a change of size of the region of interest between the current image frame and the preceding or succeeding image frame.

12. The method in accordance with claim 1 , wherein the said determining a Time-To-Contract (TTC) in step A, comprises evaluating whether a TTC can be validly determined for the respective sub-region, and wherein the step of determining one or more classification features in step B) comprises determining, for each sub-region, a classification feature that is based on a ratio of valid TTCs within the TTCs of the sub-region determined for the current image frame and for a predetermined number of image frames preceding the current image frame.

13. The method in accordance with claim 1 , wherein:

step B) further comprises:

determining at least one further classification feature based on one or more of:

a TTC of an entire region of interest determined for the current image frame;

a size or a position of the region of interest; and

the position of the current image frame within the sequence of image frames; and

the classifying of the obstacle candidate in step C) is additionally based on the determined at least one further classification feature.

14. The method in accordance with claim 1 ,

wherein the step of classifying of the obstacle candidate in step C) comprises:

after pre-classifying by:

determining that at least one of the classification features meets a respective abort condition if at least one of the classification features exceeds a respective threshold;

in response to meeting the respective abort condition, classifying the obstacle candidate based on the abort condition; and

refraining from carrying out the steps of:

determining a characteristic of the sequence of image frames corresponding to a size of the region of interest or a TTC of the entire region of interest;

in dependence on the determined characteristic, selecting a Support Vector Machine (SVM) from a predetermined set of different SVMs obtained by training the SVMs for different respective ranges of the characteristic; and

applying the selected SVM to the determined one or more classification features.

15. A non-transitory computer-readable storage media comprising instructions that when executed by one or more processors in a computing device configures the one or more processors to validate an obstacle candidate identified within a sequence of image frames by performing the following steps:

A) for a current image frame of the sequence of image frames:

determining within the current image frame a region of interest representing the obstacle candidate;

dividing the region of interest into sub-regions; and

for each sub-region, determining a Time-To-Contact (TTC) based on at least the current image frame and a preceding or succeeding image frame of the sequence of image frames;

B) determining one or more classification features based on the TTCs of the sub-regions determined for the current image frame; and

C) classifying the obstacle candidate based on the determined one or more classification features, as to whether the obstacle candidate is valid, not valid, or validity of the obstacle candidate is unclear.

16. The non-transitory computer-readable storage media in accordance with claim 15 , wherein:

the step A) is iterated, with, in the first iteration, the current image frame being a first image frame of the sequence of image frames and with, in each subsequent iteration, the current image frame being an image frame succeeding the current image frame of the respective previous iteration; and

the step B) is based on averaged TTCs of the sub-regions, the averaged TTCs being obtained for each sub-region by averaging the TTCs of the sub-region over a predetermined number of iterations.

17. The non-transitory computer-readable storage media in accordance with claim 15 , wherein determining of the region of interest comprises tracking the region of interest representing the obstacle candidate through the sequence of image frames up to at least the current frame.

18. The non-transitory computer-readable storage media in accordance with claim 15 , wherein dividing of the region of interest comprises dividing the region of interest along a vertical axis into a predetermined number of sub-regions of equal width which are regularly offset from each other with mutual overlap.

19. The non-transitory computer-readable storage media in accordance with claim 15 , wherein determining the TTC of a sub-region comprises:

determining a change of size of the sub-region between the current image frame and the preceding or succeeding image frame;

wherein step of the determining the TTC of a sub-region comprises determining interest points of the sub-region;

for each interest point, determining a change of position of the interest point between the current image frame and the preceding or succeeding image frame; and

deriving the change of size of the sub-region between the current image frame and the preceding or succeeding image frame from at least the determined changes of positions of the interest points.

20. The non-transitory computer-readable storage media in accordance with claim 19 , wherein determining of interest points of a sub-region comprises:

selecting, in the current image frame, from a set of interest points those interest points that are contained in the sub-region;

wherein the set of interest points is obtained by:

in a first image frame of the sequence of image frames, defining an initial set of interest points based on the points of an equidistant grid applied to the region of interest; and

for each subsequent image frame of the sequence of image frames, modifying the set of interest points based on tracking the interest points through the sequence of image frames; and

when the tracking an interest point leads to an invalid result, deleting the interest point from the set of interest points and replacing the deleted interest point with a new interest point.