Pré-Publication, Document De Travail Année : 2025

A rescaling-invariant Lipschitz bound based on path-metrics for modern ReLU network parameterizations

Résumé

Lipschitz bounds on neural network parameterizations are important to establish generalization, quantization or pruning guarantees, as they control the robustness of the network with respect to parameter changes. Yet, there are few Lipschitz bounds with respect to parameters in the literature, and existing ones only apply to simple feedforward architectures, while also failing to capture the intrinsic rescaling-symmetries of ReLU networks. This paper proves a new Lipschitz bound in terms of the so-called path-metrics of the parameters. Since this bound is intrinsically invariant with respect to the rescaling symmetries of the networks, it sharpens previously known Lipschitz bounds. It is also, to the best of our knowledge, the first bound of its kind that is broadly applicable to modern networks such as ResNets, VGGs, U-nets, and many more.
Fichier principal
Vignette du fichier
path_metrics_arxiv.pdf (1.3 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04584311 , version 1 (23-05-2024)
hal-04584311 , version 2 (23-05-2024)
hal-04584311 , version 3 (11-01-2025)

Identifiants

  • HAL Id : hal-04584311 , version 3

Citer

Antoine Gonon, Nicolas Brisebarre, Elisa Riccietti, Rémi Gribonval. A rescaling-invariant Lipschitz bound based on path-metrics for modern ReLU network parameterizations. 2025. ⟨hal-04584311v3⟩
176 Consultations
97 Téléchargements

Partager

More