Stereo-Camera Calibration
Stereo-Camera Main | Platform | Calibration |
We use the open source camera and IMU calibration toolbox Kalibr (commit #2999e186ec) to compute the intrinsic calibration of the RealSense cameras as well as their extrinsics. As our device is not hardware synchronized, it is crucial to leverage as much software/network synchronization as possible. We perform spatio-temporal calibration between the cameras and the two IMUs embedded in the Realsense and the Ouster sensor. The temporal offsets between measurements of the Realsense IMU and Ouster IMU with respect to the Realsense cameras are estimated using batch, continuous-time, maximum-likelihood estimation. We provide this calibration dataset along with the main dataset.
The Ouster LiDAR synchronizes with the recording computer using the Precision Time Protocol (PTP), which achieves sub-microsecond accuracy.
To get the most accurate timestamps for the IMU messages of the Realsense D435i, we used the following setup:
- Ubuntu 18.04
- Linux kernel
4.15.0-74-generic
- librealsense version
2.32.1.0
- realsense-ros version
2.2.7
- Firmware version
0.5.10.13.00
We found the Realsense to be sensitive to the right configuration, in particular for the IMU messages, hence care needs to be taken to achieve a reliable configuration.
We also found that the beam intrinsics of the Ouster we used differ from the designed values. Hence we provided the JSON-formatted beam atitude and azimuth offsets, in degress, in file beam_intrinsics.json
Calibration files
Here you can find the files used for the Kalibr calibration:
rooster_2020-03-11-10-05-35_0.bag
calibration rosbag, including RealSense camera images and Realsense and Ouster IMU messages captured in front of an april board (Note: Realsense IMU messages are at 650 Hz)rooster_2020-03-11-09-59-32_0.bag
calibration rosbag, including RealSense camera images and Realsense IMU messages captured in front of an april board (Note: Realsense IMU messages are at 250 Hz and 400 Hz for accelerometer and gyroscope, respectively)rooster_2020-03-11-09-52-47_0.bag
calibration rosbag, including RealSense camera images and Realsense IMU messages captured in front of an april board (Note: Realsense IMU messages are at 650 Hz)
Note: In all calibration rosbags, the Ouster was covered to avoid the reflection of LiDAR beams in the camera images.
april_6x6.yaml
description of the checkerboard used for calibrationcalibrate.sh
simple bash script to perform the calibrationimu.yaml
configuration file for the RealSense IMUouster_imu.yaml
configuration file for the Ouster IMU
To reproduce the Kalibr report, just make the script executable and run it:
chmod +x calibrate.sh
./calibrate.sh rooster_2020-03-11-10-05-35_0.bag
This will produce the following files (also available here):
camchain_D435i_SN844212071220rooster_2020-03-11-10-05-35_0
: this file contains the intrinsic and extrinsic calibration parameters between the left and the right cameras.results-realsense_imu-cam-rooster_2020-03-11-10-05-35_0.txt
: this file contains the intrinsic and extrinsic parameters between the right and the left cameras relative to the built-in IMU.results-ouster_imu-cam_2020-03-11-10-05-35_0.txt
: this file contains the intrinsic and extrinsic parameters between the right and the left cameras relative to the Ouster IMU.
Time Offset between LIDAR and Camera
After the experiments we noticed that the two sensor’s timestamps slowly drift relative to one another. As mentioned above the Ouster sensor is synchronised to the PC using PTP. We put this drift down to the RealSense - probably from the point its driver is started. Using the two sensor’s IMUs it is possible to estimate the relative drift using maximum correlation (breaking the data into 30 second chunks).
Below are plots of the relative drift over time for experiments 1 and 2. Relative drift is about 58.78 ms/h.
A csv of the time offsets has been added to the folder of each experiment. The file format is:
sec,nsec
- the time the offset should be applied.time_offset_nsec
- the time offset (nanoseconds).
This offset should be applied in the following manner:
time_camera_corrected = time_camera - time_offset