A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers - Mines Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers

Matteo Bastico
David Ryckelynck
Laurent Corté
Yannick Tillier

Résumé

When it comes to clinical images, automatic segmentation has a wide variety of applications and a considerable diversity of input domains, such as different types of Magnetic Resonance Images and Computerized Tomography scans. This heterogeneity is a challenge for cross-modality algorithms that should equally perform independently of the input image type fed to them. Often, segmentation models are trained using a single modality, preventing generalization to other types of input data without resorting to transfer learning techniques. Furthermore, the multi-modal or cross-modality architectures proposed in the literature frequently require registered images, which are not easy to collect in clinical environments, or need additional processing steps, such as synthetic image generation. In this work, we propose a simple framework to achieve fair image segmentation of multiple modalities using a single conditional model that adapts its normalization layers based on the input type, trained with non-registered interleaved mixed data. We show that our framework outperforms other crossmodality segmentation methods, when applied to the same 3D UNet baseline model, on the Multi-Modality Whole Heart Segmentation Challenge. Furthermore, we define the Conditional Vision Transformer encoder, based on the proposed cross-modality framework, and we show that it brings significant improvements to the resulting segmentation, up to 6.87% of Dice accuracy, with respect to its baseline reference. The code to reproduce our experiments and the trained model weights are publicly available at https: //github.com/matteo-bastico/MI-Seg.
Fichier principal
Vignette du fichier
ICCVW_arxiv.pdf (4.6 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY - Paternité

Dates et versions

hal-04309111 , version 1 (05-12-2023)

Licence

Paternité

Identifiants

Citer

Matteo Bastico, David Ryckelynck, Laurent Corté, Yannick Tillier, Etienne Decencière. A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers. International Conference on Computer Vision Workshops (ICCVW) 2023, Oct 2023, Paris, France. pp.4128-4138, ⟨10.1109/ICCVW60793.2023.00446⟩. ⟨hal-04309111⟩
48 Consultations
22 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More