Stereo-Camera Main Platform Calibration

We use the open source camera and IMU calibration toolbox Kalibr (commit #2999e186ec) to compute the intrinsic calibration of the RealSense cameras as well as their extrinsics. As our device is not hardware synchronized, it is crucial to leverage as much software/network synchronization as possible. We perform spatio-temporal calibration between the cameras and the two IMUs embedded in the Realsense and the Ouster sensor. The temporal offsets between measurements of the Realsense IMU and Ouster IMU with respect to the Realsense cameras are estimated using batch, continuous-time, maximum-likelihood estimation. We provide this calibration dataset along with the main dataset.

The Ouster LiDAR synchronizes with the recording computer using the Precision Time Protocol (PTP), which achieves sub-microsecond accuracy.

To get the most accurate timestamps for the IMU messages of the Realsense D435i, we used the following setup:

We found the Realsense to be sensitive to the right configuration, in particular for the IMU messages, hence care needs to be taken to achieve a reliable configuration.

We also found that the beam intrinsics of the Ouster we used differ from the designed values. Hence we provided the JSON-formatted beam atitude and azimuth offsets, in degress, in file beam_intrinsics.json

Calibration files

Here you can find the files used for the Kalibr calibration:

  • rooster_2020-03-11-10-05-35_0.bag calibration rosbag, including RealSense camera images and Realsense and Ouster IMU messages captured in front of an april board (Note: Realsense IMU messages are at 650 Hz)
  • rooster_2020-03-11-09-59-32_0.bag calibration rosbag, including RealSense camera images and Realsense IMU messages captured in front of an april board (Note: Realsense IMU messages are at 250 Hz and 400 Hz for accelerometer and gyroscope, respectively)
  • rooster_2020-03-11-09-52-47_0.bag calibration rosbag, including RealSense camera images and Realsense IMU messages captured in front of an april board (Note: Realsense IMU messages are at 650 Hz)

Note: In all calibration rosbags, the Ouster was covered to avoid the reflection of LiDAR beams in the camera images.

To reproduce the Kalibr report, just make the script executable and run it:

chmod +x calibrate.sh
./calibrate.sh rooster_2020-03-11-10-05-35_0.bag

This will produce the following files (also available here):

Time Offset between LIDAR and Camera

After the experiments we noticed that the two sensor’s timestamps slowly drift relative to one another. As mentioned above the Ouster sensor is synchronised to the PC using PTP. We put this drift down to the RealSense - probably from the point its driver is started. Using the two sensor’s IMUs it is possible to estimate the relative drift using maximum correlation (breaking the data into 30 second chunks).

Below are plots of the relative drift over time for experiments 1 and 2. Relative drift is about 58.78 ms/h.

linear_fit_both

A csv of the time offsets has been added to the folder of each experiment. The file format is:

  • sec,nsec - the time the offset should be applied.
  • time_offset_nsec - the time offset (nanoseconds).

This offset should be applied in the following manner: time_camera_corrected = time_camera - time_offset