Event camera based pose estimation in stereo visual-inertial odometry with multi-state constraint Kalman filter

Date

2025-8-26

Author

Keskin, Abdulbaki

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

436
views

0
downloads

Cite This

This thesis presents ESS-MSCKF (Event-Supported Stereo Multi-State Constraint Kalman Filter), an efficient stereo visual–inertial odometry pipeline that combines event cameras, conventional grayscale cameras, and an IMU. The system fuses features from stereo grayscale frames and motion-compensated event frames, called Images of Warped Events (IWEs), with inertial data in a filter-based framework. Built on the widely used MSCKF, the proposed method benefits from the high temporal resolution and robustness to brightness changes of event cameras, while also exploiting the spatial structure and stability of traditional images. A key contribution is a GPU-based motion compensation method that warps events to a reference time using odometry estimates. This generates sharp IWEs instead of the blurrier time surfaces typically employed. To the best of our knowledge, ESS-MSCKF is the first stereo event-based odometry pipeline with a filter back-end. We evaluate ESS-MSCKF on diverse datasets, including MVSEC, HKU, M3ED, and VECtor, and benchmark it against state-of-the-art methods. Results show that ESS-MSCKF achieves competitive accuracy while requiring significantly less computation, making it suitable for real-time applications. In addition to other ablation studies, we perform a key comparison of three input configurations—event-only, intensity-only, and hybrid—which demonstrates consistent performance improvements with the hybrid approach. Overall, the findings highlight that event data are crucial for motion estimation in low-light, high-speed, or high dynamic range environments, while grayscale frames provide greater reliability in low-motion and well-textured scenes.

Subject Keywords

Visual-inertial odometry, Event-based vision, Motion compensation, High dynamic range, State estimation

URI

https://hdl.handle.net/11511/116041

Collections

Graduate School of Natural and Applied Sciences, Thesis