Schedule of Events | Symposia

Generalization Gradients in Deep Vision Models: Insights from Shepard’s Universal Law

Poster Session F - Tuesday, April 1, 2025, 8:00 – 10:00 am EDT, Back Bay Ballroom/Republic Ballroom

Daniel Carstensen¹ (daniel_carstensen@brown.edu), Steven M. Frankland², Serra E. Favila¹; ¹Brown University, ²Dartmouth College

Deep vision models share striking similarities (and differences) to the primate visual system. Understanding the factors that determine these similarities and differences is crucial to creating better models of visual processing in biological systems. Here, we use Shepard’s universal law of generalization as a measure of representational alignment. According to the universal law, the perceived similarity between two stimuli should decrease as a concave function of their psychological distance, forming a characteristic generalization gradient. Recent work has found such generalization gradients for sets of natural images and corresponding human similarity judgments [Marjieh et al., 2024]. We asked if these gradients naturally appear when we use the embedding space of deep vision models as a model of the psychological space proposed by Shepard. Drawing on the same image sets, we selected a diverse range of deep vision models and extracted image embeddings from each model. We then computed the pairwise distance matrix of the embeddings and examined the resulting generalization gradients. We found that the relationship between embedding distance and human similarity judgments exhibited a concave generalization gradient for most image sets regardless of model choice, with some models showing this pattern across all sets. Notably, models overestimated the distance between a significant number of image pairs. These results show that deep vision models often acquire an embedding space consistent with Shepard’s law, indicating some alignment with primate visual processing. However, the models' tendency to overestimate image distances suggests key differences that warrant further investigation.

Topic Area: PERCEPTION & ACTION: Vision