Apparatus for obstacle detection based on relative image motion
A vehicular vision system includes a camera that captures image data. The vehicular vision system, responsive to processing by an image processor of image data captured by the camera, detects a plurality of objects present within the field of view of the camera. The vehicular vision system, responsive to detecting the objects, generates a plurality of intra-hypotheses. The vehicular vision system, responsive to generating the plurality of intra-hypotheses, generates a plurality of inter-hypothesis, with each inter-hypothesis (i) based on at least one of the intra-hypotheses and (ii) representing a respective detected object different from each other inter-hypothesis. The vehicular vision system, responsive to generating the plurality of inter-hypothesis tracks the detected object. The vehicular vision system, responsive to tracking the object, controls a driver assistance system of the vehicle.
1 . A vehicular vision system, the vehicular vision system comprising:
a camera disposed at a vehicle equipped with the vehicular vision system, the camera viewing exterior of the vehicle and capturing image data;
an electronic control unit (ECU) comprising electronic circuitry and associated software;
wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;
wherein the vehicular vision system, responsive to processing at the ECU of image data captured by the camera, detects a plurality of objects present within the field of view of the camera;
wherein the vehicular vision system, responsive to detecting the plurality of objects, generates a plurality of intra-hypotheses by (i) merging detected objects in a current frame of captured image data with tracked objects from previous frames of captured image data to produce filtered hypotheses, (ii) merging the filtered hypotheses with predicted hypotheses derived from the previous frames of captured image data to produce assigned hypotheses, and (iii) applying a Kalman filter to the assigned hypotheses to produce predicted hypotheses and evaluating the predicted hypotheses to generate the plurality of intra-hypotheses, and wherein each detected object of the plurality of detected objects is represented by at least one intra-hypothesis of the plurality of intra-hypotheses;
wherein the vehicular vision system, responsive to generating the plurality of intra-hypotheses, generates a plurality of inter-hypotheses by merging (i) one or more intra-hypotheses from the current frame of captured image data that represent a single detected object with (ii) a corresponding inter-hypothesis tracked from one or more previous frames of captured image data, to generate a new inter-hypothesis for the single detected object;
wherein the vehicular vision system tracks each detected object of the plurality of detected objects based on the generated plurality of inter-hypotheses; and
wherein the vehicular vision system, responsive to tracking each detected object, controls a driver assistance system of the vehicle.
2 . The vehicular vision system of claim 1 , wherein the vehicular vision system determines an associated distance of each detected object using a structure from motion (SfM) method or a back-to-ground projection.
3 . The vehicular vision system of claim 1 , wherein a quantity of the plurality of inter-hypotheses is less than a quantity of the plurality of intra-hypotheses.
4 . The vehicular vision system of claim 1 , wherein the vehicular vision system tracks each detected object of the plurality of detected objects using a tracking-by-detection algorithm.
5 . The vehicular vision system of claim 4 , wherein the tracking-by-detection algorithm comprises at least one kernelized correlation filter.
6 . The vehicular vision system of claim 1 , wherein the vehicular vision system merges the tracked object with newly generated objects from the plurality of inter-hypotheses.
7 . The vehicular vision system of claim 1 , wherein the driver assistance system comprises an automatic emergency braking system.
8 . The vehicular vision system of claim 1 , wherein the camera comprises a fisheye lens.
9 . A vehicular vision system, the vehicular vision system comprising:
a camera disposed at a vehicle equipped with the vehicular vision system, the camera viewing exterior of the vehicle and capturing image data;
an electronic control unit (ECU) comprising electronic circuitry and associated software;
wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;
wherein the vehicular vision system, responsive to processing at the ECU of image data captured by the camera, detects a plurality of objects present within the field of view of the camera;
wherein the vehicular vision system, responsive to detecting the plurality of objects, generates a plurality of intra-hypotheses by (i) merging detected objects in a current frame of captured image data with tracked objects from previous frames of captured image data to produce filtered hypotheses, (ii) merging the filtered hypotheses with predicted hypotheses derived from the previous frames of captured image data to produce assigned hypotheses, and (iii) applying a Kalman filter to the assigned hypotheses to produce predicted hypotheses and evaluating the predicted hypotheses to generate the plurality of intra-hypotheses, and wherein each detected object of the plurality of detected objects is represented by at least one intra-hypothesis of the plurality of intra-hypotheses;
wherein the vehicular vision system, responsive to generating the plurality of intra-hypotheses, generates a plurality of inter-hypotheses by merging (i) one or more intra-hypotheses from the current frame of captured image data that represent a single detected object with (ii) a corresponding inter-hypothesis tracked from one or more previous frames of captured image data, to generate a new inter-hypothesis for the single detected object;
wherein the vehicular vision system tracks each detected object of the plurality of detected objects based on the generated plurality of inter-hypotheses, and wherein the vehicular vision system tracks each detected object of the plurality of detected objects using a tracking-by-detection algorithm; and
wherein the vehicular vision system, responsive to tracking each detected object, controls a driver assistance system of the vehicle.
10 . The vehicular vision system of claim 9 , wherein the vehicular vision system determines an associated distance of each detected object using a structure from motion (SfM) method or a back-to-ground projection.
11 . The vehicular vision system of claim 9 , wherein the tracking-by-detection algorithm comprises at least one kernelized correlation filter.
12 . The vehicular vision system of claim 9 , wherein the driver assistance system comprises an automatic emergency braking system.
13 . The vehicular vision system of claim 9 , wherein the camera comprises a fisheye lens.
14 . A vehicular vision system, the vehicular vision system comprising:
a camera disposed at a vehicle equipped with the vehicular vision system, the camera viewing exterior of the vehicle and capturing image data, and wherein the camera comprises a fisheye lens;
an electronic control unit (ECU) comprising electronic circuitry and associated software;
wherein the electronic circuitry of the ECU comprises an image processor for processing image data captured by the camera;
wherein the vehicular vision system, responsive to processing at the ECU of image data captured by the camera, detects a plurality of objects present within the field of view of the camera;
wherein the vehicular vision system, responsive to detecting the plurality of objects, generates a plurality of intra-hypotheses by (i) merging detected objects in a current frame of captured image data with tracked objects from previous frames of captured image data to produce filtered hypotheses, (ii) merging the filtered hypotheses with predicted hypotheses derived from the previous frames of captured image data to produce assigned hypotheses, and (iii) applying a Kalman filter to the assigned hypotheses to produce predicted hypotheses and evaluating the predicted hypotheses to generate the plurality of intra-hypotheses, and wherein each detected object of the plurality of detected objects is represented by at least one intra-hypothesis of the plurality of intra-hypotheses;
wherein the vehicular vision system, responsive to generating the plurality of intra-hypotheses, generates a plurality of inter-hypotheses by merging (i) one or more intra-hypotheses from the current frame of captured image data that represent a single detected object with (ii) a corresponding inter-hypothesis tracked from one or more previous frames of captured image data, to generate a new inter-hypothesis for the single detected object, and wherein a quantity of the plurality of inter-hypotheses is less than a quantity of the plurality of intra-hypotheses;
wherein the vehicular vision system tracks each detected object of the plurality of detected objects based on the generated plurality of inter-hypotheses; and
wherein the vehicular vision system, responsive to tracking each detected object, controls a driver assistance system of the vehicle.
15 . The vehicular vision system of claim 14 , wherein the vehicular vision system determines an associated distance of each detected object using a structure from motion (SfM) method or a back-to-ground projection.