# Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning

DevFeed: [Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning](<https://devfeed.tech/articles/slower-generalization-faster-memorization-a-sweet-spot-in-algorithmic-learning-116702.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.14659>)

Author: Shin So, Kyelim Lee, Albert No

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Cellular automaton](<https://devfeed.tech/topics/cellular-automaton.md>)

Tags: [cellular-automata](<https://devfeed.tech/tags/cellular-automata.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [generation](<https://devfeed.tech/tags/generation.md>), [input](<https://devfeed.tech/tags/input.md>), [intermediate](<https://devfeed.tech/tags/intermediate.md>), [learning](<https://devfeed.tech/tags/learning.md>), [random](<https://devfeed.tech/tags/random.md>), [relationships](<https://devfeed.tech/tags/relationships.md>), [rules](<https://devfeed.tech/tags/rules.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [validation](<https://devfeed.tech/tags/validation.md>)

## AI overview

Small Transformers reached high exact-match validation accuracy on Needleman-Wunsch matrices in the fewest optimizer updates with an intermediate dataset size; larger datasets converged more slowly. A random-suffix probe found that models fit the structured matrices before memorizing random suffixes at larger sizes. In elementary cellular automata, lower-order predictive signals helped explain learnability: finite-sample fluctuations in Rule 90 created output biases and single-input dependencies absent from the uniform population, and these signals weakened as datasets grew. Training with such biases accelerated generalization on an unchanged uniform validation distribution. The findings suggest sampling fluctuations can help models acquire rules, though the evidence here is limited to the described tasks and experiments.

## Source excerpt

arXiv:2605.14659v2 Announce Type: replace Abstract: Once generalization becomes possible, does adding more data always make learning faster? In Needleman-Wunsch matrix generation, we find that small Transformers reach high exact-match validation accuracy in the fewest optimizer updates at an intermediate dataset size, while larger datasets converge more slowly. A random-suffix probe shows that, at larger dataset sizes, models fit the structured matrices before memorizing their random suffixes, suggesting that shared rule learning aids training-set fitting. To examine what makes rules easier to learn, we study elementary cellular automata (ECA), in which each binary cell updates from its own state and those of its two neighbors. These experiments identify lower-order predictive signals-output biases or predictive relationships involving only a few input bits-as a key factor in learnability. In Rule 90, finite-sample fluctuations create output biases and single-input dependencies absent from the uniform population. These signals weaken as the dataset grows. Biased training that introduces these signals accelerates generalization on an unchanged uniform validation distribution. These results support an explanation for faster convergence at intermediate dataset sizes: sampling fluctuations can facilitate rule acquisition, and larger datasets weaken this advantage.