Semantic Segmentation using Foundation Models for Cultural Heritage: an Experimental Study on Notre-Dame de Paris - Chantier scientifique Notre-Dame de Paris
Communication Dans Un Congrès Année : 2023

Semantic Segmentation using Foundation Models for Cultural Heritage: an Experimental Study on Notre-Dame de Paris

Résumé

The zero-shot performance of foundation models has captured a lot of attention. Specifically, the Segment Anything Model (SAM) has gained popularity in computer vision due to its label-free segmentation capabilities. Our study proposes using SAM on cultural heritage data, specifically images of Notre-Dame de Paris, with a controlled vocabulary. SAM can successfully identify objects within the cathedral. To further improve segmentation, we utilized Grounding DINO to detect objects and CLIP to automatically add labels from the segmentation masks generated by SAM. Our study demonstrates the usefulness of foundation models for zero-shot semantic segmentation of cultural heritage data.
Fichier principal
Vignette du fichier
ICCV workshop paper.pdf (1.38 Mo) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04275484 , version 1 (08-11-2023)

Identifiants

  • HAL Id : hal-04275484 , version 1

Citer

Kévin Réby, Anaïs Guillem, Livio De Luca. Semantic Segmentation using Foundation Models for Cultural Heritage: an Experimental Study on Notre-Dame de Paris. 4th ICCV Workshop on Electronic Cultural Heritage, Computer Vision Foundation, Oct 2023, Paris, France. https://openaccess.thecvf.com/content/ICCV2023W/e-Heritage/html/Reby_Semantic_Segmentation_Using_Foundation_Models_for_Cultural_Heritage_an_Experimental_ICCVW_2023_paper.html. ⟨hal-04275484⟩

Relations

108 Consultations
97 Téléchargements

Partager

More