# Random Initialization as a Baseline for Time Series Foundation Models

DevFeed: [Random Initialization as a Baseline for Time Series Foundation Models](<https://devfeed.tech/articles/random-initialization-as-a-baseline-for-time-series-foundation-models-97003.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.30950>)

Author: Pinar Sungu Isiacik, Georgiana Ifrim

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Time Series](<https://devfeed.tech/topics/time-series.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [time-series](<https://devfeed.tech/tags/time-series.md>), [time-series-foundation-models](<https://devfeed.tech/tags/time-series-foundation-models.md>)

## AI overview

Across seven publicly released TSFM instances, pretrained weights improved mean UEA multivariate time series classification accuracy over architecture-matched random initialization by 1.43-5.19 percentage points (2.58 points on average). Each comparison used frozen representations and four classifiers, with five random initializations; computational exclusions left 183 paired dataset-level comparisons. Three of seven instance-level Wilcoxon tests had raw p-values below 0.05, but only NuTime remained significant after Holm correction. A pooled test across pairs was significant, though the authors treat corrected per-instance tests as primary evidence. ROCKET, Hydra, and Quant were also competitive without large-scale pretraining. The findings support evaluating pretrained weights separately from architecture, alongside random-initialization and strong non-pretrained baselines.

## Source excerpt

arXiv:2610.30950v1 Announce Type: new Abstract: Time series foundation models (TSFMs) use self-supervised pretraining to learn reusable representations for downstream tasks such as multivariate time series classification (MTSC). We examine how much pretrained weights contribute beyond the architecture itself by comparing seven publicly released instances from four TSFM families with architecture-matched randomly initialized counterparts on the UEA MTSC archive. For each instance, five random initializations are evaluated using four classifiers on frozen representations: ridge classification, logistic regression, RBF SVM, and random forest. After model-specific exclusions due to computational constraints, the study comprises 183 paired dataset-level comparisons. Pretraining improves mean accuracy for all seven instances, but the gains are modest, ranging from 1.43 to 5.19 percentage points, with an overall average improvement of 2.58 points. Three of seven per-instance Wilcoxon signed-rank tests have raw $p$-values below 0.05, but only NuTime remains significant after Holm correction. A secondary pooled test across all 183 pairs gives $p < 0.001$, while the corrected per-instance tests remain the primary evidence. We also compare with ROCKET, Hydra, and Quant, which remain competitive without requiring large-scale pretraining. These findings suggest that MTSC evaluations should distinguish the effect of pretrained weights from that of the underlying architecture and should include both architecture-matched random initialization and strong non-pretrained baselines.