The effect of language input in semantic segmentation and grading of colorectal cancer with language-vision models.

Date

2024-7-29

Author

Şehlaver, Sina

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

129
views

62
downloads

Cite This

Accurately diagnosing and grading colorectal cancer from histopathological images is crucial for effective treatment. Traditionally, this grading relies on visual examination by pathologists, a process that is inherently time-consuming, subjective, and prone to inter-observer variability. Deep learning offers a promising avenue for automating this process, but two major challenges remain: limited availability of large, annotated datasets and a lack of dedicated semantic segmentation models for this task. This study proposes a modified architecture of MaskCLIP, a powerful vision-language model, fine-tuned on our annotated colorectal cancer image dataset for semantic segmentation and grading of colorectal cancer images. This marks the first application of such a model to this specific task, offering a way to leverage textual information in histopathological analysis. We conduct a comparative analysis of vision-language and vision-only MaskCLIP variants to understand the impact of language guidance on model performance. By investigating whether integrating MaskCLIP as a vision-language module surpasses the capabilities of traditional vision-only models, we aim to uncover the potential benefits of incorporating textual data in histopathological analysis. This research provides valuable insights into the role of language in enhancing deep learning models for colorectal cancer analysis, particularly for semantic segmentation.

Subject Keywords

Deep Learning, Semantic Segmentation, Colorectal Cancer, Vision-language models, Histopathology

URI

https://hdl.handle.net/11511/110567

Collections

Graduate School of Natural and Applied Sciences, Thesis