Techniques for extrinsic sensor calibration using mobile device
Techniques are described for calibrating a coordinate frame of a remote sensing sensor of a perception system of a machine with respect to a coordinate frame of the machine. An application of a mobile device, such as a smartphone, a tablet computing device, or a laptop, or a web service detects a visual fiducial tag and a feature of the machine in an image by using an optical sensor of a digital camera system. The fiducial tag detection provides the coordinate transformation between the mobile device and a housing of the remote sensing sensor, e.g., radar sensor, lidar sensor, sonar sensor, or camera. Detecting a feature of the machine, e.g., a traction component such as a wheel or track, a brake light, a headlight, a turn signal, or a logo, provided constraints for the transformation between the device and machine.
1 . A computer-implemented method for calibrating a perception system of a machine, the method comprising:
detecting, using an optical sensor of a digital camera system, a first fiducial tag coupled with a remote sensing sensor, wherein the remote sensing sensor is coupled with a machine frame of the machine;
detecting, using the optical sensor, a feature of the machine;
determining, using the first fiducial tag, a first coordinate transformation between the optical sensor and the remote sensing sensor;
determining, using the feature, a second coordinate transformation between the optical sensor and the machine frame;
determining, using the first coordinate transformation and the second coordinate transformation, a third coordinate transformation between the remote sensing sensor and the machine frame; and
calibrating, using the third coordinate transformation, the perception system of the machine.
2 . The computer-implemented method of claim 1 , wherein the feature includes a traction component, a brake light, a headlight, a turn signal, or a logo.
3 . The computer-implemented method of claim 1 , wherein the digital camera system is part of a mobile device.
4 . The computer-implemented method of claim 3 , wherein detecting, using the optical sensor of the digital camera system, the first fiducial tag mounted on the remote sensing sensor includes:
detecting, from an image generated by the digital camera system, the first fiducial tag and the feature.
5 . The computer-implemented method of claim 1 , comprising:
displaying, on a user interface, a model of the machine; and
receiving, as user input to the user interface, a selection of the model of the machine.
6 . The computer-implemented method of claim 1 , comprising:
displaying, on a user interface, an outline; and
detecting that the machine is positioned within the outline.
7 . The computer-implemented method of claim 6 , wherein detecting, using the optical sensor, the feature of the machine includes:
detecting, using the optical sensor, the feature of the machine when the machine is positioned within the outline without extending beyond the outline.
8 . The computer-implemented method of claim 1 , comprising:
receiving, as user input to the user interface, a location on the user interface of the feature of the machine.
9 . The computer-implemented method of claim 1 , wherein the first coordinate transformation includes a transform between the first fiducial tag and the remote sensing sensor.
10 . A computer-implemented method for calibrating a perception system of a machine, the method comprising:
detecting, using a first image generated by an optical sensor of a digital camera system, a first fiducial tag coupled with a remote sensing sensor and a second fiducial tag mounted on a machine frame of the machine, wherein the remote sensing sensor is coupled with the machine frame;
detecting, using a second image generated by the optical sensor of the digital camera system, the second fiducial tag and a feature of the machine;
determining, using the first fiducial tag and the second fiducial tag, a first coordinate transformation between the optical sensor and the remote sensing sensor;
determining, using the second fiducial tag and the feature, a second coordinate transformation between the optical sensor and the machine frame;
determining, using the first coordinate transformation and the second coordinate transformation, a third coordinate transformation between the remote sensing sensor and the machine frame; and
calibrating, using the third coordinate transformation, the perception system of the machine.
11 . The computer-implemented method of claim 10 , wherein the feature includes a traction component, a brake light, a headlight, a turn signal, or a logo.
12 . The computer-implemented method of claim 10 , wherein the digital camera system is part of a mobile device.
13 . The computer-implemented method of claim 12 , comprising:
displaying, on a user interface, a model of the machine; and
receiving, as user input to the mobile device, a selection of the model of the machine.
14 . The computer-implemented method of claim 10 , comprising:
receiving, as user input to the user interface, a location on the user interface of the feature of the machine.
15 . The computer-implemented method of claim 10 , comprising:
displaying, on a user interface, an outline; and
detecting that the machine is positioned within the outline.