Layer-wise structural and mechanistic interpretation of OpenVLA using controlled perturbations

Date

2026-9-02

Author

Yıldız, Turgay

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

60
views

0
downloads

Cite This

Limitations of conventional policies in robot learning have driven the adoption of large-scale pretrained models, known as foundation models, leading to the emergence of Vision-Language- Action (VLA) models. Although VLA models have alleviated some of these significant limitations, such as generalization through foundation model backbones, they also inherit flaws, including an inability to comprehend negation. Additionally, this adaptation further degrades some already limited capabilities. Furthermore, VLAs introduce novel issues such as the integration of perception and control, alongside prompt-induced and action-related limitations. Therefore, current VLA models face significant challenges. To this end, this thesis aims to explore the interpretability-related limitations of OpenVLA under a unified framework. Moreover, it adopts log-probability-based probing to the VLA setting to alleviate the limitations of conventional linear probing, enabling a comparative analysis. Both methods are employed in a layer-wise manner via task-specific controlled perturbations. Our findings demonstrate that (1) OpenVLA neglects the fixed part of the prompts, focusing primarily on instructions; (2) negation has a negligible effect on action prediction; (3) language following exhibits greater sensitivity to spatial properties than to colors; (4) the model is most sensitive to translations, followed by gripper and then rotations; (5) the first and second halves of the architecture appear to process different types of information, with the 27th and 32nd layers requiring distinct attention. Ultimately, the insights gained from this thesis are expected to inform novel architectural design, more effective training strategies and improved data design in embodied AI.

Subject Keywords

Autoregressive vision-language-action models, OpenVLA, Linear probing, Log-probability, Task-specific controlled perturbations

URI

https://hdl.handle.net/11511/120984

Collections

Graduate School of Informatics, Thesis