Method and device for autonomously plugging a charging plug into a charging socket of a vehicle
Autonomously plugging a charging plug installed on a programmable robot arm into a charging socket of a transportation vehicle. The method includes a camera to locate the charging socket by image analysis of acquired images of the charging socket; determining pose data to determine a pose of the located charging socket using a PnP algorithm; moving the charging plug to an intermediate position; creating a depth image of the charging socket corresponding to the current position of the charging plug when the charging plug is in the intermediate position; performing a comparison between the determined depth image and a reference depth image corresponding to a reference position to determine a positional deviation between the current position of the charging plug and the reference position; moving the charging plug to a plug-in position; and plugging the charging plug into the charging socket from the plug-in position.
1 . A method for autonomously plugging a charging plug into a charging socket of a transportation vehicle, wherein the charging plug is installed on a programmable robot arm of a charging station, the method comprising:
camera-based localization of the charging socket by image analysis of images of the charging socket;
obtaining pose data based on the analyzed images to determine a pose of the localized charging socket using a PnP algorithm;
moving the charging plug to an intermediate position based on the determined pose data;
creating a depth image of the charging socket indicating a current position of the charging plug when the charging plug is in the intermediate position;
performing a comparison between the determined depth image and a reference depth image corresponding to a reference position to determine a positional deviation between the current position of the charging plug and the reference position;
moving the charging plug to a plug-in position from the current position of the charging plug based on the determined positional deviation so as to plug the charging plug into the charging socket.
2 . The method of claim 1 , wherein the image analysis is performed by a neural network pre-trained using data on position features of the charging socket.
3 . The method of claim 1 , wherein the pose is determined by additional averaging of the pose data.
4 . The method of claim 1 , wherein the reference depth image is a reference depth image taken from a predefined distance and/or position relative to the charging socket.
5 . The method of claim 1 , wherein the positional deviation is determined using an ICP algorithm.
6 . The method of claim 1 , further comprising:
digitizing the environment of the charging station as a digital map including a charging socket area in which the charging socket is located; and
modifying the digital map into a modified digital map without the charging socket area,
wherein the plugging the charging plug into the charging socket is performed based on the modified digital map.
7 . A non-transitory computer-readable medium including a computer program, comprising instructions which cause an apparatus as to perform the method of claim 1 .
8 . The non-transitory computer readable medium of claim 7 , wherein the image analysis is performed by a neural network pre-trained using data on position features of the charging socket.
9 . The non-transitory computer readable medium of claim 7 , wherein the pose is determined by additional averaging of the pose data.
10 . The non-transitory computer readable medium of claim 7 , wherein the reference depth image is a reference depth image taken from a predefined distance and/or position relative to the charging socket.
11 . The non-transitory computer readable medium of claim 7 , wherein the positional deviation is determined using an ICP algorithm.
12 . An apparatus for autonomously moving and plugging a charging plug into a charging socket of a transportation vehicle, comprising:
a charging station with a programmable robot arm on which the charging plug ( 10 ) is installed;
a camera system mounted on the robot arm and configured for camera-based localization of the charging socket by image analysis of images of the charging socket and for creating a depth image of the charging socket indicating a current position when the charging plug has been moved to an intermediate position at a distance from the charging socket;
a computational unit for determining pose data for determining a pose of the localized charging socket using a PnP algorithm to analyze images of the charging socket; and
a comparison unit for performing a comparison between the created depth image and a reference depth image corresponding to a reference position to determine a positional deviation between a current position of the charging plug and the reference position,
wherein the charging plug is moved to the intermediate position based on the determined pose data, and
wherein the charging plug is moved to a plug-in position from the intermediate position of the charging plug based on the determined positional deviation so as to plug the charging plug into the charging socket.
13 . The apparatus of claim 12 , wherein the charging station is a mobile charging station.
14 . The apparatus of claim 12 , wherein the image analysis is performed by a neural network pre-trained using data on position features of the charging socket.
15 . The apparatus of claim 12 , wherein the pose is determined by additional averaging of the pose data.
16 . The apparatus of claim 12 , wherein the reference depth image is a reference depth image taken from a predefined distance and/or position relative to the charging socket.
17 . The apparatus of claim 12 , wherein the positional deviation is determined using an ICP algorithm.