Pith. sign in

REVIEW 3 major objections 5 minor 49 references

Bound by semanticity: universal laws governing the generalization-identification tradeoff

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Finite semantic resolution forces a universal Pareto front between generalization and identification.

desk verdict The closed-form results for step-function similarity are correct and worth publishing, but the 'universal' claim is broader than the theorems actually prove. read the letter →

arxiv 2506.14797 v1 pith:EABVS5LM submitted 2025-06-01 cs.LG cs.AI

Marco Nurisso , Jesseba Fernando , Raj Deshpande , Alan Perotti , Raja Marjieh , Steven M. Frankland , Richard L. Lewis , Taylor W. Webb
show 4 more authors
Declan Campbell Francesco Vaccarino Jonathan D. Cohen Giovanni Petri
This is my paper
classification cs.LGcs.AI
keywords generalization-identificationtradeoffsemanticresolutionrepresentationalsimilarityParetofrontballmeasuremulti-itemcapacityneuralnetworksfinite-resolutionconstraint
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that the tension between generalization and identification is not an empirical accident but a formal consequence of finite semantic resolution. It models a system whose similarity judgments follow a step function: perfect similarity within a ball of radius $\varepsilon$, and a small noise value outside. For this model the probability of correct generalization is $p_S = 1/2 + \langle b\rangle - \langle b\rangle^2 - \mathrm{Var}(b)$ and the probability of correct identification is $p_I = 1 - \langle b\rangle/2$, where $b$ is the measure of the ball of radius $\varepsilon$ around a probe. These formulas trace a Pareto front in the $(p_S,p_I)$ plane that is independent of the geometry of the stimulus space, and the $n$-input extension predicts identification accuracy that collapses roughly as $1/n$. The same laws are reported in a minimal ReLU network, a convolutional network, and several large language and vision-language models.

What carries the argument

The load-bearing object is the constant similarity function of Definition 1, $g_{\varepsilon;\Delta}(x,y) = \mathbb{1}_{B_\varepsilon(x)}(y) + \Delta\,\mathbb{1}_{M\setminus B_\varepsilon(x)}(y)$, together with the ball measure $b_p(\varepsilon) = \nu(B_\varepsilon(p))$ and its average and variance over the stimulus distribution. The ratio choice rule $D_i = g(x_i,p)/\sum_k g(x_k,p)$ converts these similarities into decision probabilities. Theorem 1's identity then does the work: it reduces the two probabilities $p_S$ and $p_I$ to one scalar $\langle b(\varepsilon)\rangle$ (plus a variance penalty), which is why the Pareto front is universal in homogeneous spaces. The same machinery, with expected values over $b_p(\varepsilon)$, yields the $n$-item formulas and the $1/n$ capacity collapse.

What would settle it

Measure a trained model's raw similarity function $g(d)$ as a function of representational distance; fit its resolution $\varepsilon$ and noise $\Delta$, then compare the model's observed $(p_S,p_I)$ across a range of thresholds to Equations (3) and (4). A system whose similarity decays linearly or exponentially should deviate from the predicted front in exactly the way Proposition 1 predicts, which would settle whether the front is universal across decay shapes or specific to the step function.

Watch

Extended reading notes

Core claim

The central claim is that finite resolution $\varepsilon$ in computing similarity pins a system's performance to closed-form curves. For the constant similarity function $g_{\varepsilon;\Delta}(x,y) = 1$ when $d(x,y) \le \varepsilon$ and $\Delta$ otherwise, Theorem 1 gives $p_S(\varepsilon) = 1/2 + \langle b(\varepsilon)\rangle - \langle b(\varepsilon)\rangle^2 - \mathrm{Var}(b(\varepsilon))$ and $p_I(\varepsilon) = 1 - \langle b(\varepsilon)\rangle/2$; noise shifts these via Theorem 2, and Theorem 3 gives the $n$-item generalizations with a $1/n$ identification collapse. In homogeneous spaces the variance term vanishes and both probabilities are parameterized by $\langle b(\varepsilon)\rangle$, so all models sit on one universal Pareto curve. The authors find that empirical training trajectories of a toy ReLU network follow this curve, with a resolution boundary that self-organizes during learning, and that large language models and vision-language models show resolution-limited accuracy in temporal and spatial judgments. The intended conclusion is that finite-resolution similarity is an emergent informational constraint on any system, biological or artificial, that builds structured representations for generalization.

Load-bearing premise

That real similarity functions are well described by the step-like constant form $g_{\varepsilon;\Delta}$, since Proposition 1 shows a linearly decaying similarity function yields a different Pareto curve with different coefficients; the theorems therefore establish universality only within one decay shape, unless the qualitative front is shown to be insensitive to that shape.

Editorial extensions

If this is right

  • Generalization is maximized when the resolution ball covers half the stimulus space, $\langle b(\varepsilon)\rangle = 1/2$; increasing resolution beyond that point hurts both $p_S$ and $p_I$.
  • In homogeneous spaces every model's performance lies on a single Pareto curve parameterized by $\langle b(\varepsilon)\rangle$, making model-independent comparisons possible.
  • Spatial heterogeneity, captured by $\mathrm{Var}(b(\varepsilon))$, lowers similarity-test accuracy relative to the homogeneous curve, so non-uniform stimulus distributions make generalization harder.
  • The $n$-item formulas predict a sharp $1/n$ collapse of identification accuracy, explaining multi-object reasoning limits in large vision-language models as a consequence of resolution rather than architecture or scale.
  • A ReLU network trained on a semantic similarity task self-organizes a resolution boundary, and its $(p_S,p_I)$ trajectory tracks the theoretical curve, showing the tradeoff emerges from ordinary gradient-based learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The exact coefficients of the Pareto front likely depend on the shape of the decay, not only on $\varepsilon$; the paper's own Proposition 1 demonstrates this for linear decay, so the 'universal' part of the claim is best read as the existence and qualitative location of the tradeoff rather than a single numeric curve.
  • Editorial inference: The $1/n$ identification collapse is a directly testable prediction for human working memory: multi-item visual identification accuracy should fall as roughly $1/(b n)$ with $b$ fixed by the task's resolution, connecting the theory to classic set-size effects.
  • Editorial inference: One could use the fitted ball measure $b(\varepsilon)$ as a cheap diagnostic for foundation models: a model's measured resolution should predict its multi-object failure point before running large-scale benchmarks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper formalizes a tradeoff between generalization and identification in representation-learning systems. It defines probabilities pS and pI via a Luce choice model on similarity functions g, and considers a constant step similarity g_{ε,Δ}(x,y) = 1_{d(x,y)≤ε} + Δ1_{d(x,y)>ε}. Under this model, Theorem 1 derives pS(ε) = 1/2 + <b(ε)> - <b(ε)>^2 - Var(b(ε)) and pI(ε) = 1 - (1/2)<b(ε)> for two items; Theorem 2 adds a nonzero noise level Δ; Theorem 3 extends the formulas to n items and predicts a 1/n collapse of identification performance. Proposition 1 computes the analogous front for linearly decaying similarity on the circle, obtaining different coefficients. The paper claims the resulting Pareto front is universal for any model with finite-resolution decaying similarity and independent of input-space geometry, and it reports empirical support in a toy ReLU network, a ResNet-50, several LLMs, and two VLMs.

Significance. The theoretical derivation is a clean exercise: for the step similarity function, the two-item success probabilities reduce to expectations of the ball-measure b_p(ε) and its square, with the variance term as the only geometry-dependent quantity. The closed forms for noise and for n items are useful, and the paper explicitly provides a separate proposition for linear decay, which is a genuine attempt to probe robustness. The empirical sections cover diverse architectures and the code is released, both of which are strengths. However, the headline claim of universality is substantially broader than what the theorems prove, since only the step-shaped similarity is analyzed in the theorems and Proposition 1 shows the front changes under linear decay. The significance of the paper as a contribution therefore depends on whether it can be reframed as a theorem about step-like resolution rather than a universal law for all distance-decaying similarities.

major comments (3)
  1. [Abstract, Theorem 1, Proposition 1] The claim that the laws hold 'for any model whose representational similarity decays with finite semantic resolution' is not supported by the theorems. Theorems 1-3 are derived only for the constant step similarity function of Definition 1. For a generic distance-decaying similarity g(r), the derivation in Appendix A.2, specifically Eq. (18), integrates expressions such as g(r1)/(g(r1)+g(r2)) against the push-forward measure S_p, so the result depends on the full functional form of g and not only on the ball measure b_p(ε). Proposition 1 is an in-paper counterexample to the universal claim: on the circle, linear decay gives pS = 1/2 + b − (3/2 − log 2)b^2 and pI = 1 − (1 − log 2)b, which differs from Theorem 1's pS = 1/2 + b − b^2 and pI = 1 − b/2. The paper should either restrict the universality claim to step-like similarity functions or prove a theorem showing the front is insensitive to the decay shape; without that, the abstract's 'any model' wording is too strong.
  2. [Abstract, Theorem 1, Eq. (3)] The statement that the Pareto front is 'independent of input space geometry' is not supported by Eq. (3), which contains the term −Var(b(ε)). The universal curve that is independent of M and ν holds only in the homogeneous case Var(b(ε)) = 0. The main text does acknowledge the variance term, but the abstract and introduction do not state this necessary condition. The paper should qualify the geometry-independence claim explicitly, for example by saying the front is universal in homogeneous spaces.
  3. [Section 4, Figure 4, Proposition 1] The empirical validation in the toy model is partly circular. The dashed theoretical curve in Figure 4 is drawn using a noise level Δ that is estimated from the learned similarity function, and the black curve is obtained from Proposition 1, which is selected after observing that the learned similarity functions appear approximately linear. Thus the close match between the empirical trajectories and the theoretical curves is in part a fitting procedure rather than a parameter-free prediction. The paper should distinguish between curves derived from independently fixed parameters and curves fitted to the data, and adjust the strength of the claim that the empirical trajectories 'closely follow' the theoretical predictions.
minor comments (5)
  1. [Section 2, paragraph after Eq. (1)] In the definition of the induced distance d on M, the second argument should be Φ^{-1}(y), not Φ^{-1}(x): 'd(x, y) := dS(Φ−1(x), Φ−1(y))'.
  2. [Section 4, first paragraph] The phrase 'a simple con model' appears to be a typo; it should read 'toy model'.
  3. [Appendix A.2.2, Eq. (30)] The last differential in Eq. (30) should be dν(x2), not dν(x1), because the integration is over the pair (x1, x2).
  4. [Appendix A.6.2] There is a typo in 'the the decrease' in the results paragraph; it should be 'the decrease'.
  5. [Theorem 3, Eq. (8)] Equation (8) is undefined when b_p(ε) = 0, since both the numerator and denominator vanish. The paper uses the convention 0/0 = 1/2 for maximally uncertain decisions elsewhere, but this convention is not stated for Eq. (8); the limiting interpretation should be spelled out.

Circularity Check

3 steps flagged · score 4.0 of 10

Theorems 1-3 are self-contained computations for the step similarity of Definition 1, but the abstract's 'any model' universality is the step ansatz itself; the toy and VLM validations fit (epsilon, Delta) or b(epsilon) and then present the fitted curve as confirmation.

  1. self definitional [Abstract and Section 3, Theorem 1 with Definition 1; contrasted with Proposition 1]
    ""For any model whose representational similarity between inputs decays with finite semantic resolution ε, we derive closed-form expressions that pin its probability of correct generalization pS and identification pI to a universal Pareto front independent of input space geometry." ... "Definition 1. The constant similarity function with resolution ε and noise ∆ is gε;∆(x, y) = 1 Bε(x)(y) + ∆1 M \ Bε(x)(y)." ... Proposition 1: "pS(ε) = 1/2 + b(ε) − (3/2 − log(2))b(ε)^2, pI(ε) = 1 − (1 − log(2))b(ε).""

    The closed forms of Theorem 1 are derived for exactly the indicator/step function of Definition 1; they express pS and pI as functions of the ball measure b(ε) only for that specific g. The abstract's 'any model whose representational similarity decays with finite semantic resolution' is therefore not a consequence of finite-resolution decay but of the step ansatz chosen in Definition 1. The paper's own Proposition 1 for a linearly decaying similarity gives a different Pareto front (coefficients 3/2 − log 2 and log 2 instead of 1 and 1/2), so the claimed 'universal' front is equivalent to the step-function definition, not to the general premise stated in the abstract.

  2. fitted input called prediction [Section 4, toy neural network paragraph]
    ""Estimating this noise scale ∆ and using that in the equations given by Theorem 2, shows that the corresponding dashed curve accurately predicts the value of pI at which the training stops.""

    The noise scale ∆ is estimated from the learned similarity function, and then plugged into Theorem 2, which is an algebraic identity for the constant similarity function g_{ε;∆}. Since pI in Theorem 2 is a deterministic function of the fitted (ε, ∆), reading off pI from that fitted curve is curve-fitting, not an independent prediction. The agreement is forced by the fit rather than by an out-of-sample test of the theory.

1 more flagged steps
  1. fitted input called prediction [Appendix A.6.4, VLM color similarity task, Results paragraph]
    ""In particular, we observe a decreasing identification performance which, in both models, can be fitted using the theoretical curve of Theorem 3 (main text). The fitted parameter b(ε) suggests the presence of a larger effective resolution for Gemma and a lower one for Qwen.""

    Here b(ε) is a free parameter fitted to the measured pI(n) data using Theorem 3. Theorem 3 is a closed-form map from b(ε) to pI for the step similarity function, so a one-parameter fit will always trace some curve of that family; the success of the fit reports the fitted b(ε) value rather than independently confirming the law. Presenting this as evidence of an 'emergent resolution' renames the fitted parameter as the discovered phenomenon.

full rationale

The core mathematical derivation is not circular: Theorems 1, 2, and 3 are exact computations from the explicitly stated constant similarity function of Definition 1, with proofs given in the appendix, and no fitted parameter enters the closed-form expressions themselves. The self-citation to Frankland et al. [23] supplies task definitions and the phrase 'Miller's Law', but the theorems do not rest on that citation for their validity; they are self-contained given (M, d, Σ, ν) and g_{ε;Δ}. The main circularity signals are at the edges. First, the abstract's universality claim ('any model whose representational similarity decays with finite semantic resolution' and 'independent of input space geometry') is broader than what is proven: the Pareto front is derived for the step-like g of Definition 1, and the paper's own Proposition 1 shows a different front for a linearly decaying similarity, so the 'universal law' is effectively the ansatz written as a result. Second, the toy model validation fits the noise scale Δ from the learned similarity and then uses the theorem to 'predict' pI, which is a fitted-input-called-prediction. Third, the VLM color experiment fits b(ε) with Theorem 3 to the identification curve and then reports the fitted value as evidence of an emergent resolution; this is a one-parameter fit, not an out-of-sample prediction. These issues do not invalidate the self-contained theorem proofs, but they do mean part of the empirical confirmation reduces by construction, and the headline universality claim overstates the scope of Definition 1. A score of 4 reflects partial circularity in the validation and framing while recognizing that the central derivations are internally consistent computations.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The theory introduces no new physical or mathematical entities. The only inputs are the model parameters epsilon and Delta, plus the empirically fitted ball mass b(epsilon) and noise Delta. The core assumptions are the distance-only similarity, the step-function resolution model, and the Luce choice rule, all stated explicitly.

free parameters (2)
  • Effective resolution ball mass b(epsilon) = Fitted per task, e.g., Gemma vs Qwen in the color task
    In Appendix A.6.4, b(epsilon) is fitted using Theorem 3 to the VLM identification data, then used to draw the theoretical curves and estimate noise levels. This is an in-sample fit, not an independent prediction.
  • Noise level Delta in the toy model = Estimated from learned similarity functions
    Section 4 estimates Delta from the toy network's similarity noise and uses Theorem 2 to predict the final pI, so the comparison is not fully independent of the data it is meant to explain.
assumptions (5)
  • domain assumption Similarity depends only on distance: g(x,y) = g(d(x,y))
    Stated in Section 2: 'we assume for simplicity that g depends only on the distance between the stimuli'. The theorems are built on this assumption.
  • domain assumption Resolution-limited similarity has a step form: g_{epsilon,Delta}(x,y) = 1 inside the ball, Delta outside
    Definition 1. The closed forms in Theorems 1 to 3 are computed only for this function, so the claimed universal front is specific to this decay shape.
  • domain assumption Choice follows Luce's ratio rule: D_i = g(x_i,p) / sum_k g(x_k,p)
    Equation (1), citing Luce 1959. The probabilities pS and pI are derived from this decision function, which is an assumption about how models convert similarities to choices.
  • standard math Absolute continuity of ball-mass functions b_p
    Required in Appendix A.1 to ensure no ties via Lemma 1 and to use the fundamental theorem of calculus in the proofs. It excludes pathological metric measure spaces.
  • ad hoc to paper Convention 0/(0+0) = 1/2 for maximally uncertain decisions
    Introduced in the proof of Theorem 1 in Appendix A.2.1 to handle the case where both items fall outside the resolution region. It is a modeling convention rather than a derived result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bound by semanticity: universal laws governing the generalization-identification tradeoff." pith.science (2026). https://pith.science/paper/EABVS5LM

@misc{pith2026250614797,
  author       = {Pith},
  title        = {Pith review of: Bound by semanticity: universal laws governing the generalization-identification tradeoff},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EABVS5LM}},
  note         = {Machine review of arXiv:2506.14797}
}
abstract

Intelligent systems must deploy internal representations that are simultaneously structured -- to support broad generalization -- and selective -- to preserve input identity. We expose a fundamental limit on this tradeoff. For any model whose representational similarity between inputs decays with finite semantic resolution $\varepsilon$, we derive closed-form expressions that pin its probability of correct generalization $p_S$ and identification $p_I$ to a universal Pareto front independent of input space geometry. Extending the analysis to noisy, heterogeneous spaces and to $n>2$ inputs predicts a sharp $1/n$ collapse of multi-input processing capacity and a non-monotonic optimum for $p_S$. A minimal ReLU network trained end-to-end reproduces these laws: during learning a resolution boundary self-organizes and empirical $(p_S,p_I)$ trajectories closely follow theoretical curves for linearly decaying similarity. Finally, we demonstrate that the same limits persist in two markedly more complex settings -- a convolutional neural network and state-of-the-art vision-language models -- confirming that finite-resolution similarity is a fundamental emergent informational constraint, not merely a toy-model artifact. Together, these results provide an exact theory of the generalization-identification trade-off and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike.

Figures

Figures reproduced from arXiv: 2506.14797 by the authors.

Figure 1
Figure 1. a. On the left, exponential similarity functions centered on two stimuli x1, x2 ∈ M, with the black line indicating the decision function g(x1, p)/(g(x1, p) + g(x2, p)) with no resolution (see Section 2 for explanation). On the center and right, the same quantities are shown in the case of the presence of finite resolution. Notice that the model becomes uncertain for probes far away from stimuli x1, x2. b. Visualiza… view at source ↗
Figure 2
Figure 2. a. The region in (pS, pI ) plane where the model’s performances lie (Theorem 1). The black line is pa￾rameterized by the resolution ε and rep￾resents the behaviour of the model in homogeneous spaces. b. Effect of het￾erogeneity Var(b(ε)) on the similarity test performance. This curve exhibits three regimes as a function of the ball’s resolution ε. Low ε regime. For small resolutions, the similarity func￾tions act li… view at source ↗
Figure 3
Figure 3. a. Similarity-identification curves for different values of n and parameterized by bp(ε) ∈ [0, 1], as described by Equations (7) and (8). b. The colored curves correspond to similarity￾identification values as the number of inputs n varies, for some fixed values of bp(ε). c. Similarity (top) and identification (bottom) dependence on n for different resolutions. multiple stimuli [1]. On the one hand, these tasks typi… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Emergent resolution and tradeoff in toy architecture. (pS, pI ) results for the toy model (a) of Section 4 with 50 inputs. b. The orange curve shows the average training trajectory for a purely reconstruction loss. The orange insets show the learned (average) similarit…
Figure 5
Figure 5. Figure 5: Empirical resolution tradeoffs across realistic neural architectures. (a) a CNN fine￾tuned on bird recognition shows tradeoff between species identification and generalization to phylo￾genetic similarity as a function of the weights of generalization α and of the resol…
Figure 6
Figure 6. Figure 6: Visualization of the distance matrix (left) and the learned similarity matrices through [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: Different training trajectories of the toy model with different latent dimensions, visualized [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Evolutionary similarity between species obtained from (a) bird phylogeny and (b) the [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: (a) Average generalization and identification performance for different fixed thresholds [PITH_FULL_IMAGE:figures/full_fig_p024_9.png]
Figure 10
Figure 10. Figure 10: Similarity and identification performances of three LLMs on the interval of years [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Examples of image inputs for the spatial resolution task. [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Examples of image inputs for the color similarity task. Panels (a) and (d) represent two [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: a. Similarity and identification probabilities for the color test explained in Appendix A.6.4. In the identification plot, the dashed curves are the theoretical curves of Theorem 3 (main text) fitted to the data. b. Token score associated to the wrong responses, as a …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 28 canonical work pages

  1. [1]

    Understanding the limits of vision language models through the lens of the binding problem

    Declan Campbell, Sunayana Rane, Tyler Giallanza, Camillo Nicol \`o De Sabbata, Kia Ghods, Amogh Joshi, Alexander Ku, Steven Frankland, Tom Griffiths, Jonathan D Cohen, et al. Understanding the limits of vision language models through the lens of the binding problem. Advances in Neural Information Processing Systems, 37: 0 113436--113460, 2024

    work page 2024

  2. [2]

    G. E. Hinton, J. L. McClelland, and D. E. Rumelhart. Parallel distributed processing: Explorations in the microstructure of cognition. MIT Press, 1986

    work page 1986

  3. [3]

    Geoffrey E. Hinton. Learning distributed representations of concepts. In Proceedings of Eighth Annual Conference of the Cognitive Science Society, 1986. URL https://www.cs.toronto.edu/ hinton/absps/families.pdf

    work page 1986

  4. [4]

    Tensor product variable binding and the representation of symbolic structures in connectionist systems

    Paul Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46 0 (1--2): 0 159--216, 1990. doi:10.1016/0004-3702(90)90007-M

    doi:10.1016/0004-3702(90)90007-m 1990

  5. [5]

    Adina L. Roskies. The binding problem. Neuron, 24, 1999. URL https://www.cell.com/neuron/fulltext/S0896-6273(00)80817-X?_returnURL=https

    work page 1999

  6. [6]

    On the binding problem in artificial neural networks

    Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. ArXiv, 2020. URL https://arxiv.org/pdf/2012.05208

    arXiv 2020

  7. [7]

    A feature-integration theory of attention

    Anne M Treisman and Garry Gelade. A feature-integration theory of attention. Cognitive psychology, 12 0 (1): 0 97--136, 1980

    work page 1980

  8. [8]

    Stimulus and response generalization: Deduction of the generalization gradient from a trace model

    Roger Shepard. Stimulus and response generalization: Deduction of the generalization gradient from a trace model. Psychological Review, 1958. URL https://psycnet.apa.org/record/1959-09346-001

    work page 1958

Show all 49 references
  1. [9]

    Toward a universal law of generalization for psychological science

    Roger N Shepard. Toward a universal law of generalization for psychological science. Science, 237 0 (4820): 0 1317--1323, 1987

    1987

  2. [10]

    Discrimination of mnemonic similarity is associated with short-term and long-term memory precision

    Weizhen Xie, Tianye Ma, Sanikaa Thakurdesai, Inik Kim, and Weiwei Zhang. Discrimination of mnemonic similarity is associated with short-term and long-term memory precision. Memory & Cognition, pages 1--13, 2024

    2024

  3. [11]

    Schurgin, John T

    Mark W. Schurgin, John T. Wixted, and Timothy F. Brady. Psychophysical scaling reveals a unified theory of visual memory strength. Nature Human Behaviour, 2020. URL https://www.nature.com/articles/s41562-020-00938-0

    2020

  4. [12]

    Ivan Tomic and Paul M. Bays. Perceptual similarity judgments do not predict the distribution of errors in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 50, 2024. URL https://psycnet.apa.org/doiLanding?doi=10.1037

    2024

  5. [13]

    Efficient compression in color naming and its evolution

    Noga Zaslavsky, Charles Kemp, Terry Regier, and Naftali Tishby. Efficient compression in color naming and its evolution. Proceedings of the National Academy of Sciences, 115 0 (31): 0 7937--7942, 2018

    2018

  6. [14]

    Shepard's universal law supported by honeybees in spatial generalization

    Ken Cheng. Shepard's universal law supported by honeybees in spatial generalization. Psychological Science, 11 0 (5): 0 403--408, 2000. doi:10.1111/1467-9280.00278. URL https://doi.org/10.1111/1467-9280.00278

  7. [15]

    Hebart, Charles Y

    Martin N. Hebart, Charles Y. Zheng, Francisco Pereira, and Chris I. Baker. Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nature Human Behaviour, 4, 2020. URL https://www.nature.com/articles/s41562-020-00951-3#citeas

    2020

  8. [16]

    Tenenbaum and Thomas L

    Joshua B. Tenenbaum and Thomas L. Griffiths. Generalization, similarity, and Bayesian inference. Behavioral and Brain Sciences, 24 0 (4): 0 629--640, August 2001. doi:10.1017/s0140525x01000061

    2001 doi

  9. [17]

    Chris R. Sims. Efficient coding explains the universal law of generalization in human perception. Science, 360, 2018. URL https://www.science.org/doi/10.1126/science.aaq1118

    2018 doi

  10. [18]

    Deep learning and the information bottleneck principle

    Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pages 1--5. Ieee, 2015

    2015

  11. [19]

    Chris R. Sims. Rate-distortion theory and human perception. Cognition, 152, 2016. URL https://www.sciencedirect.com/science/article/pii/S0010027716300750

    2016

  12. [20]

    Pereira, and William Bialek

    Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method. ArXiv, 1999. URL https://arxiv.org/pdf/physics/0004057

    1999 arXiv

  13. [21]

    Opening the black box of deep neural networks via information

    Ravid Schwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. ArXiv, 2017. URL https://arxiv.org/pdf/1703.00810

    2017 arXiv

  14. [22]

    Self-attention limits working memory capacity of transfomer-based models

    Dongyu Gong and Hantao Zhang. Self-attention limits working memory capacity of transfomer-based models. ArXiv, 2024. URL https://arxiv.org/pdf/2409.10715

    2024 arXiv

  15. [23]

    No coincidence, george: Processing limits in cognitive function reflect the curse of generalization, Oct 2021

    Steven M Frankland, Taylor Webb, Richard L Lewis, and Jonathan D Cohen. No coincidence, george: Processing limits in cognitive function reflect the curse of generalization, Oct 2021. URL osf.io/preprints/psyarxiv/cjuxb_v1

    2021

  16. [24]

    On the local behavior of spaces of natural images

    Gunnar Carlsson, Tigran Ishkhanov, Vin De Silva, and Afra Zomorodian. On the local behavior of spaces of natural images. International journal of computer vision, 76: 0 1--12, 2008

    2008

  17. [25]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  18. [26]

    Individual choice behavior, volume 4

    R Duncan Luce. Individual choice behavior, volume 4. Wiley New York, 1959

    1959

  19. [27]

    Toy models of superposition

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition. arXiv preprint arXiv:2209.10652, 2022

    2022 arXiv

  20. [28]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770--778, 2016. doi:10.1109/CVPR.2016.90

    2016 doi

  21. [29]

    The Caltech-UCSD Birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011

    2011

  22. [30]

    TimeTree 5 : An expanded resource for species divergence times

    Sudhir Kumar, Morgan Suleski, Jessica M Craig, Anna E Kasprowicz, Maxwell Sanderford, Mingfeng Li, Glen Stecher, and S Blair Hedges. TimeTree 5 : An expanded resource for species divergence times. Molecular Biology and Evolution, 39 0 (8): 0 msac174, 2022. doi:10.1093/molbev/msac174

    2022 doi

  23. [31]

    Gemma 2: Improving open language models at a practical size

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  24. [32]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  25. [33]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  26. [34]

    Gemma Team. Gemma 3. 2025 a . URL https://goo.gle/Gemma3Report

    2025

  27. [35]

    Qwen2.5-vl, January 2025 b

    Qwen Team. Qwen2.5-vl, January 2025 b . URL https://qwenlm.github.io/blog/qwen2.5-vl/

    2025

  28. [36]

    Neural theory for few-shot learning of naturalistic stimuli

    Ben Sorscher, Surya Ganguli, and Haim Sompolinsky. Neural theory for few-shot learning of naturalistic stimuli. Proceedings of the National Academy of Sciences, 119 0 (12): 0 e2112410119, 2022. doi:10.1073/pnas.2112410119

    2022 doi

  29. [37]

    Giovanni Petri, Sebastian Musslick, and Jonathan D. Cohen. An information-theoretic approach to reward rate optimization in the tradeoff between controlled and automatic processing in neural network architectures. eLife, 13, 2024. URL https://elifesciences.org/reviewed-preprints/93251

    2024

  30. [38]

    Topological limits to the parallel processing capability of network architectures

    Giovanni Petri, Sebastian Musslick, Biswadip Dey, Kayhan \"O zcimder, David Turner, Nesreen K Ahmed, Theodore L Willke, and Jonathan D Cohen. Topological limits to the parallel processing capability of network architectures. Nature Physics, 17 0 (5): 0 646--651, 2021

    2021

  31. [39]

    Michael Lesnick, Sebastian Musslick, Biswadip Dey, and Jonathan D. Cohen. A formal framework for cognitive models of multitasking. PsyArXiv, 2020. URL https://osf.io/preprints/psyarxiv/7yzdn_v1

    2020

  32. [40]

    Separability and geometry of object manifolds in deep neural networks

    Uri Cohen, SueYeon Chung, Daniel D Lee, and Haim Sompolinsky. Separability and geometry of object manifolds in deep neural networks. Nature Communications, 11 0 (1): 0 746, 2020. doi:10.1038/s41467-020-14578-5

    2020 doi

  33. [41]

    A thesaurus for a neural population code

    Elad Ganmor, Ronen Segev, and Elad Schneidman. A thesaurus for a neural population code. eLife, 2015. URL https://elifesciences.org/articles/06134.pdf

    2015

  34. [42]

    Combinatorial neural codes from a mathematical coding theory perspective

    Carina Curto, Vladimir Itskov, Katherine Morrison, Zachary Roth, and Judy L Walker. Combinatorial neural codes from a mathematical coding theory perspective. Neural computation, 25 0 (7): 0 1891--1925, 2013

    1925

  35. [43]

    Lake and Marco Baroni

    Brenden M. Lake and Marco Baroni. Human-like systematic generalization through a meta-learning neural network. Nature, 623, 2023. URL https://www.nature.com/articles/s41586-023-06668-3

    2023

  36. [44]

    Fodor and Zenon W

    Jerry A. Fodor and Zenon W. Pylyshyn. Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 1998. URL https://www.sciencedirect.com/science/article/abs/pii/0010027788900315

    1998

  37. [45]

    Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks

    Alexandra M Proca, Fernando E Rosas, Andrea I Luppi, Daniel Bor, Matthew Crosby, and Pedro AM Mediano. Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks. PLoS computational biology, 20 0 (6): 0 e1012178, 2024

    2024

  38. [46]

    An introduction to integration and measure theory

    Ole A Nielsen. An introduction to integration and measure theory. (No Title), 1997

    1997

  39. [47]

    Real analysis: modern techniques and their applications

    Gerald B Folland. Real analysis: modern techniques and their applications. John Wiley & Sons, 1999

    1999

  40. [48]

    Graham, Donald E

    Ronald L. Graham, Donald E. Knuth, and Oren Patashnik. Concrete Mathematics . Addison-Wesley, 1989. ISBN 0-201-14236-8

    1989

  41. [49]

    Qwen technical report

    Jinze Bai, Yixuan Yang, Yingqi Chai, Victor Ling, Aohan Yang, Zhiyuan Lei, Junyang Huang, Yonggang Tan, Xiubo Liu, Zhijian Yang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.