System and methods for autonomously backing a vehicle to a trailer
In some embodiments, techniques are provided for autonomously backing a vehicle (such as a Class 8 truck) to a trailer. In some embodiments, environment sensors such as image sensors and range sensors mounted to the vehicle detect a trailer and determine distances between the trailer and the vehicle as well as relative angles of the trailer and the vehicle. In some embodiments, the vehicle determines a path to the trailer based on this information, and autonomously controls braking, torque, and/or steering of the vehicle to autonomously back the vehicle along the determined path.
1 . A method of autonomously backing a vehicle to a trailer, the method comprising:
determining, by an autonomous backing module of the vehicle, a trailer to target in an image captured by a sensor positioned on the vehicle;
determining, by the autonomous backing module, a distance to the trailer, an angle of an axis of the trailer, and an angle of an axis of the vehicle including:
detecting, by the autonomous backing module, a left edge of a front surface of the trailer and a right edge of the front surface of the trailer in an edge map generated from the image;
determining, by the autonomous backing module, a first distance from the vehicle to the left edge and a second distance from the vehicle to the right edge in a depth map generated from the image by cross-referencing locations in the edge map against information from the depth map, wherein cross-referencing locations in the edge map against information from the depth map includes identifying depth discontinuities in the depth map and verifying that locations of the depth discontinuities in the depth map correspond to locations of the left edge and the right edge detected on the edge map to confirm presence of one or more of the left edge or the right edge as detected from the edge map;
determining, by the autonomous backing module, the angle of the axis of the trailer and the angle of the axis of the vehicle based on the first distance and the second distance;
determining, by the autonomous backing module, a path to the trailer based on the distance to the trailer, the angle of the axis of the trailer, and the angle of the axis of the vehicle; and
transmitting, by the autonomous backing module, one or more commands to components of the vehicle to autonomously control the vehicle to back along the determined path to the trailer.
2 . The method of claim 1 , wherein determining the trailer to target includes:
presenting, by an operator interface device of the vehicle, an image captured by a first environment sensor mounted to a rear portion of the vehicle; and
receiving, by the operator interface device, an input from an operator indicating a location within the image that corresponds to the front surface of the trailer to target.
3 . The method of claim 1 , wherein detecting the left edge of the front surface of the trailer comprises finding an edge to the left of the location indicated by the operator, and wherein detecting the right edge of the front surface of the trailer comprises finding an edge to the right of the location indicated by the operator.
4 . The method of claim 1 , wherein determining the path to the trailer includes:
determining a result that is a multi-order polynomial function;
wherein the function is defined in a Cartesian space having an origin at the center of the front surface of the trailer, and an X-axis parallel to an axis of the trailer; and
wherein the function includes terms that represent:
an X-Y position of a rear axle of the vehicle; and
an X-Y position of a front axle of the vehicle.
5 . The method of claim 1 , wherein transmitting, by the autonomous backing module, one or more commands to components of the vehicle to autonomously control the vehicle to back along the determined path to the trailer includes:
transmitting commands to one or more of a braking control module, a steering control module, and a torque request module.
6 . The method of claim 1 , wherein the vehicle is a Class 8 truck.
7 . A vehicle configured to autonomously back to a trailer, the vehicle comprising:
a braking control module for electronically controlling a brake system;
a steering control module for electronically controlling a steering system;
a torque request module for electronically causing the vehicle to produce a requested amount of torque; and
an electronic control module (ECM) configured to:
determine a trailer to target in an image captured by a sensor positioned on the vehicle;
determine a distance to the trailer, an angle of an axis of the trailer, and an angle of an axis of the vehicle including:
detecting a left edge of a front surface of the trailer and a right edge of the front surface of the trailer in an edge map generated from the image;
determining a first distance from the vehicle to the left edge and a second distance from the vehicle to the right edge in a depth map generated from the image by cross-referencing locations in the edge map against information from the depth map, wherein cross-referencing locations in the edge map against information from the depth map includes identifying depth discontinuities in the depth map and verifying that locations of the depth discontinuities in the depth map correspond to locations of the left edge and the right edge detected on the edge map to confirm presence of one or more of the left edge or the right edge as detected from the edge map;
determining the angle of the axis of the trailer and the angle of the axis of the vehicle based on the first distance and the second distance;
determine a path to the trailer based on the distance to the trailer, the angle of the axis of the trailer, and the angle of the axis of the vehicle; and
transmit one or more commands to the braking control module, the steering control module, and the torque request module to autonomously control the vehicle to back along the determined path to the trailer.
8 . The vehicle of claim 7 , further comprising an operator interface device and a first environment sensor, wherein the first environment sensor is mounted to a rear portion of the vehicle, and wherein determining a trailer to target includes;
presenting, by the operator interface device, an image captured by the first environment sensor; and
receiving, by the operator interface device, an input from an operator indicating a location within the image that corresponds to the front surface of the trailer to target.
9 . The vehicle of claim 8 , wherein detecting the left edge of the front surface of the trailer comprises finding an edge to the left of the location indicated by the operator, and wherein detecting the right edge of the front surface of the trailer comprises finding an edge to the right of the location indicated by the operator.
10 . The vehicle of claim 7 , wherein determining the path to the trailer includes:
determining a result that is a third order polynomial function with a zeroth term missing and a first order term missing;
wherein the function is defined in a Cartesian space having an origin at the center of the front surface of the trailer, and an X-axis parallel to an axis of the trailer; and
wherein the function includes terms that represent:
an X-Y position of a rear axle of the vehicle; and
an X-Y position of a front axle of the vehicle.
11 . The vehicle of claim 7 , wherein the vehicle is a Class 8 truck.
12 . A non-transitory computer-readable medium having computer-executable instructions stored thereon that, in response to execution by an electronic control unit (ECU) of a vehicle, cause the vehicle to perform actions for autonomously backing to a trailer, the actions comprising:
determining, by the ECU, a trailer to target in an image captured by a sensor positioned on the vehicle;
determining, by the ECU, a distance to the trailer, an angle of an axis of the trailer, and an angle of an axis of the vehicle including:
detecting, by the ECU, a left edge of a front surface of the trailer and a right edge of the front surface of the trailer in an edge map generated from the image;
determining, by the ECU, a first distance from the vehicle to the left edge and a second distance from the vehicle to the right edge in a depth map generated from the image by cross-referencing locations in the edge map against information from the depth map, wherein cross-referencing locations in the edge map against information from the depth map includes identifying depth discontinuities in the depth map to and verifying that locations of the depth discontinuities in the depth map correspond to locations of the left edge and the right edge detected on the edge map confirm presence of one or more of the left edge or the right edge as detected from the edge map;
determining, by the ECU, the angle of the axis of the trailer and the angle of the axis of the vehicle based on the first distance and the second distance;
determining, by the ECU, a path to the trailer based on the distance to the trailer, the angle of the axis of the trailer, and the angle of the axis of the vehicle; and
transmitting, by the ECU, one or more commands to components of the vehicle to autonomously control the vehicle to back along the determined path to the trailer.
13 . The computer-readable medium of claim 12 , wherein determining the trailer to target includes:
presenting, by an operator interface device of the vehicle, an image captured by a first environment sensor mounted to a rear portion of the vehicle; and
receiving, by the operator interface device, an input from an operator indicating a location within the image that corresponds to the front surface of the trailer to target.
14 . The computer-readable medium of claim 12 , wherein detecting the left edge of the front surface of the trailer comprises finding an edge to the left of the location indicated by the operator, and wherein detecting the right edge of the front surface of the trailer comprises finding an edge to the right of the location indicated by the operator.
15 . The computer-readable medium of claim 12 , wherein determining the path to the trailer includes:
determining a result that is a third order polynomial function with a zeroth term missing and a first order term missing;
wherein the function is defined in a Cartesian space having an origin at the center of the front surface of the trailer, and an X-axis parallel to an axis of the trailer; and
wherein the function includes terms that represent;
an X-Y position of a rear axle of the vehicle; and
an X-Y position of a front axle of the vehicle.
16 . The computer-readable medium of claim 3 ,
wherein transmitting, by the autonomous backing module, one or more commands to components of the vehicle to autonomously control the vehicle to back along the determined path to the trailer includes:
transmitting commands to one or more of a braking control module, a steering control module, and a torque request module.
17 . The computer-readable medium of claim 12 , wherein the vehicle is a Class 8 truck.