Direct Visual Lidar Calib 实践

系统环境

ubuntu2004
ros noetic

依赖

ROS1/ROS2
PCL
OpenCV
GTSAM 优化库
Ceres 优化库
Iridescence 图形化显示
SuperGlue [optional]

安装

Install dependencies

sudo apt install libomp-dev libboost-all-dev libglm-dev libglfw3-dev libpng-dev libjpeg-dev

Install GTSAM

git clone https://github.com/borglab/gtsam
cd gtsam && git checkout 4.2a9
mkdir build && cd build
# For Ubuntu 22.04, add -DGTSAM_USE_SYSTEM_EIGEN=ON
cmake .. -DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF \
         -DGTSAM_BUILD_TESTS=OFF \
         -DGTSAM_WITH_TBB=OFF \
         -DGTSAM_BUILD_WITH_MARCH_NATIVE=OFF
make -j$(nproc)
sudo make install

如果报错重新安装或者检查gtsam是否重复安装

libdirect_visual_lidar_calibration.so: undefined symbol: _ZNK5gtsam15NonlinearFactor5rekeyERKSt6vectorImSaImEE

ldd install/lib/libdirect_visual_lidar_calibration.so

Install Ceres

git clone --recurse-submodules https://github.com/ceres-solver/ceres-solver
cd ceres-solver
git checkout e47a42c2957951c9fafcca9995d9927e15557069
mkdir build && cd build
cmake .. -DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF -DUSE_CUDA=OFF
make -j$(nproc)
sudo make install

Install Iridescence for visualization

git clone https://github.com/koide3/iridescence --recursive
mkdir iridescence/build && cd iridescence/build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install

Install SuperGlue (Optional)

Warning

SuperGlue is not allowed to be used for commercial purposes. You must carefully check and follow its licensing conditions.

pip3 install numpy opencv-python torch matplotlib
git clone https://github.com/magicleap/SuperGluePretrainedNetwork.git

echo 'export PYTHONPATH=$PYTHONPATH:/path/to/SuperGluePretrainedNetwork' >> ~/.bashrc
source ~/.bashrc

Build direct_visual_lidar_calibration ATTACH

# ROS1
cd ~/catkin_ws/src
git clone https://github.com/koide3/direct_visual_lidar_calibration.git --recursive
cd .. && catkin_make
# ROS2
cd ~/ros2_ws/src
git clone https://github.com/koide3/direct_visual_lidar_calibration.git --recursive
cd .. && colcon build

编译报错

CMake Error at /usr/local/lib/cmake/iridescence/iridescence-config.cmake:43 (add_library):
add_library cannot create ALIAS target "Iridescence::Iridescence" because
target "Iridescence::iridescence" is imported but not globally visible.

解决方法

https://github.com/koide3/direct_visual_lidar_calibration/issues/120

https://github.com/koide3/direct_visual_lidar_calibration/issues/94

/usr/bin/ld: cannot find -lfmt
make: *** [Makefile:141: all] Error 2

解决方法

sudo apt-get install libfmt-dev

运行

预处理 ATTACH

rosrun direct_visual_lidar_calibration preprocess --data_path /data/cjtong/rosbag/calib

ros1用户如果跑livox数据集livox_ros1.tar.gz会报错

[ERROR] [1742887033.079032080]: Client [/rostopic_815016_1742887029912] wants topic /camera_info to have datatype/md5sum [sensor_msgs/CameraInfo/c9a58c1b0b154e0e6da7578cb991d214],
but our version has [sensor_msgs/CameraInfo/b5aff04250fbf270b71bef9d82c2132e]. Dropping connection.

解决办法 https://github.com/koide3/direct_visual_lidar_calibration/issues/25

相机内参可以通过plotjuggler查看

rosrun direct_visual_lidar_calibration preprocess -av   --camera_model plumb_bob   --camera_intrinsic 1452.711762456289,1455.877531619469,1265.25895179213,1045.818593664107   --camera_distortion_coeffs -0.04203564850455424,0.0873170980751213,0.002386381727224478,0.005629700706305988,-0.04251149335870252 /data/cjtong/rosbag/calib /data/cjtong/rosbag/calib

粗标

自动

rosrun direct_visual_lidar_calibration find_matches_superglue.py /data1/cjtong/ros_ws/rosbag/calib
rosrun direct_visual_lidar_calibration initial_guess_auto /data/cjtong/rosbag/calib

手动

rosrun direct_visual_lidar_calibration initial_guess_manual /data/cjtong/rosbag/calib

精标 ATTACH

rosrun direct_visual_lidar_calibration calibrate /data/cjtong/rosbag/calib

结果

校准完成后，打开/data/cjtong/calib/calib.json文件。

标定结果T_lidar_camera: [x， y, z, qx, qy, qz, qw]，将相机帧中的3D点转换为LiDAR帧（即p_lidar = T_lidar_camera * p_camera）。 calib.json还包含相机参数，手动/自动初值结果（init_T_lidar_camera和init_T_lidar_camera_auto），以及一些元数据。