Virtual overlays in camera mirror systems
A computer system comprising a processor device configured to receive initial image data from at least one camera of a vehicle; detect, in the initial image data, at least one predetermined target mounted to a trailer of the vehicle, the at least one predetermined target at an initial position and initial angular orientation relative to the at least one camera; position a virtual assist feature in a first position and a first angular orientation in a display view based on the initial position and the initial angular orientation of the at least one predetermined target; detect, in subsequent image data, the at least one predetermined target at a subsequent position and a subsequent angular orientation relative to the at least one camera; and adjust the virtual assist feature in the display view based on the subsequent position and the subsequent angular orientation relative to the at least one camera.
1 . A computer system comprising a processor device configured to:
receive initial image data from at least one camera of a vehicle;
detect, in the initial image data, at least two predetermined targets mounted to a trailer of the vehicle, the at least two predetermined targets at initial positions and initial angular orientations relative to the at least one camera;
position a virtual assist feature in first positions and first angular orientations in a display view based on the initial positions and the initial angular orientations of the at least two predetermined targets, the virtual assist feature in the display view reaching between the first positions of each of the at least two predetermined targets;
detect, in subsequent image data, the at least two predetermined targets at respective subsequent positions and respective subsequent angular orientations relative to the at least one camera;
adjust the virtual assist feature in the display view according to the subsequent positions and the subsequent angular orientations relative to the at least one camera.
2 . A computer-implemented method, comprising:
receiving, by a processor device of a computer system, initial image data from at least one camera of a vehicle;
detecting, by the processor device, in the initial image data, at least two predetermined target-targets mounted to a trailer of the vehicle, the at least two predetermined target-targets at initial positions and initial angular orientations relative to the at least one camera;
positioning, by a processor device, a virtual assist feature in first positions and first angular orientations in a display view based on the initial positions and the initial angular orientations of the at least two predetermined targets, the virtual assist feature in the display view reaching between the first positions of each of the at least two predetermined targets;
detecting, by the processor device, in subsequent image data, the at least two predetermined targets at respective subsequent positions and respective subsequent angular orientations relative to the at least one camera;
adjusting, by the processor device, the virtual assist feature in the display view based on according to the subsequent positions and the subsequent angular orientations relative to the at least one camera.
3 . The method of claim 2 , where the initial position and the initial angular orientation, and the subsequent position and the subsequent angular orientation of the at least two predetermined targets, are determined based on a predetermined transformation function from position and angular orientation relative to the at least one camera, to position and angular orientation in the display view.
4 . The method of claim 3 , further comprising:
detecting, by the processor device, a predetermined calibration marker with known dimensions and at a known location relative the camera; and
determining, by the processor device, the predetermined transformation function based at least on the known dimensions and the known location of the predetermined calibration marker and based on a pixel layout of the display view, the predetermined transformation function representing a rotation-translation transformation from a camera field of view to the display view.
5 . The method of claim 4 , wherein the predetermined marker have a predetermined size and a predetermined shape.
6 . The method of claim 4 , wherein detecting the predetermined marker in the image data is based on image analysis.
7 . The method of claim 4 , wherein the predetermined marker have layouts distinguishable from each other.
8 . The method of claim 2 , further comprising:
positioning, by the processor device, a second virtual assist feature in a second position and second angular orientation in the display view; and
adjusting, by the processor device, each of the virtual assist features in the at least one display view according to the subsequent position and the subsequent angular orientation.
9 . The method of claim 2 , further comprising:
detecting, by the processor device, a set of predetermined targets mounted on the trailer at or near corners of the trailer;
calculating, by the processor device, dimensions of the trailer based on differences in initial positions or subsequent positions for the set of predetermined targets; and
providing, by the processor device, a message including the calculated dimensions of the trailer to a user interface of the vehicle or to a memory storage.
10 . The method of claim 2 , further comprising:
positioning, by the processor device, a three-dimensional virtual assist feature mapped into the display view.
11 . The method of claim 2 , further comprising:
generating, by the processor device, at least one virtual assist feature offset from an outline of the trailer in the display view.
12 . The method of claim 2 , further comprising:
detecting, by the processor device, a new predetermined target in the display view; and
positioning, by the processor device, a new virtual assist feature in the display view.
13 . The method of claim 2 , wherein image data is received from two rearward facing cameras arranged on opposite sides of a cab of the vehicle, and predetermined targets mounted on two sides of the trailer are detected;
wherein virtual assist features are positioned and adjusted in the two display views.
14 . The method of claim 2 , further comprising:
retrieving, by the processor device, the virtual assist feature from a library of predetermined virtual assist features; and
positioning, by the processor device, the retrieved virtual assist feature in the first position and the first angular orientation in the display view.
15 . The method of claim 2 , further comprising:
detecting, by the processor device, in the initial image data, two or more of the at least two predetermined targets, at initial positions and initial angular orientations relative to the at least one camera based on image analysis, predetermined markers have a predetermined shape and are distinguishable from each other;
positioning, by the processor device, a virtual assist feature for each of the predetermined markers in respective first positions and first angular orientations in a display view determined from the initial positions and initial angular orientations and a predetermined transformation function configured to transform a location in a camera field of view to pixels in the display view;
detecting, by the processor device, in subsequent image data, the at least two predetermined targets at subsequent positions and subsequent angular orientations relative to the at least one camera based on image analysis;
adjusting, by the processor device, the virtual assist feature in the display view according to the subsequent positions and subsequent angular orientations relative to the at least one camera and the predetermined transformation function;
in each of the initial image data and the and subsequent image data, calculating, by the processor device, dimensions of the trailer based on differences in each of the initial positions and the subsequent positions for the predetermined targets; and
providing, by the processor device, for each of the initial image data and the subsequent image data, a message including the calculated dimensions of the trailer to a user interface of the vehicle or to a memory storage.
16 . A vehicle comprising the processor device to perform the method of claim 2 .
17 . A non-transitory computer program product comprising program code for performing, when executed by the processor device, the method of claim 2 .
18 . A control system comprising one or more control units configured to perform the method of claim 2 .
19 . A non-transitory computer-readable storage medium comprising instructions, which when executed by the processor device, cause the processor device to perform the method of claim 2 .