# saddle

Published articles for saddle.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## On the Complexity of Finite-Sum Nonconvex-Strongly-Concave Minimax Optimization with Exact Proximal Oracles

DevFeed: [On the Complexity of Finite-Sum Nonconvex-Strongly-Concave Minimax Optimization with Exact Proximal Oracles](<https://devfeed.tech/articles/on-the-complexity-of-finite-sum-nonconvex-strongly-concave-minimax-optimization-with-exact-proximal-oracles-116986.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.04806>)

Author: Yuxing Peng, Zhiqing Tang, Weijia Jia

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [computational complexity](<https://devfeed.tech/topics/computational-complexity.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [component](<https://devfeed.tech/tags/component.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [learning](<https://devfeed.tech/tags/learning.md>), [linear](<https://devfeed.tech/tags/linear.md>), [lower-bounds](<https://devfeed.tech/tags/lower-bounds.md>), [minimax](<https://devfeed.tech/tags/minimax.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [problems](<https://devfeed.tech/tags/problems.md>), [queries](<https://devfeed.tech/tags/queries.md>), [responses](<https://devfeed.tech/tags/responses.md>), [robust](<https://devfeed.tech/tags/robust.md>), [saddle](<https://devfeed.tech/tags/saddle.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

For finite-sum nonconvex-strongly-concave minimax optimization under the stated smoothness, condition-number, and initialization assumptions, exact component proximal access and unrestricted adaptivity do not improve polynomial query complexity. The paper proves a worst-case lower bound of Ω(n + min{√n κ, n^(3/4)√κ} LΔ/ε²) component queries for randomized adaptive algorithms given exact component values, gradients, and proximal responses. A method using only component gradients achieves the same rate up to a log²(2+κ) factor, including initialization. In the constructed hard instances, all components allow proximal steps at the same step sizes, but each response reveals hidden directions only until the first direction assigned to another component. The abstract notes that Zhou concurrently established the same rate for zero-respecting first-order methods.

### Source excerpt

arXiv:2611.04806v1 Announce Type: new Abstract: Finite-sum nonconvex-strongly-concave minimax problems arise in adversarial training and distributionally robust learning. Their known lower bounds cover only linear-span algorithms, zero-respecting algorithms, or a restricted proximal protocol. They leave open the power of exact component proximal steps, whose responses are global saddle points of regularized component problems and can expose directions that gradients do not reveal. We show that neither unrestricted adaptivity nor exact proximal access improves the polynomial complexity. We consider $n$ components with joint mean-squared smoothness $L$ and condition number $\kappa$, started from a point with primal gap at most $\Delta$ and joint gradient norm at most $\sqrt{2L\Delta}$. For $\varepsilon^2/(L\Delta)$ below a universal constant, every randomized adaptive algorithm with exact component values, gradients, and proximal responses needs $\Omega(n+\min\{\sqrt n\,\kappa,n^{3/4}\sqrt\kappa\}L\Delta/\varepsilon^2)$ component queries in the worst case to find an $\varepsilon$-stationary point of the value function. A method using only component gradients attains this rate up to a factor $\log^2(2+\kappa)$, including initialization. Zhou (arXiv:2610.04944) concurrently proves the same rate for zero-respecting first-order methods. In our hard instances, every component admits proximal steps with the same step sizes, and a response for a component reveals hidden directions only up to the first one assigned to another component.

## Finite Spectral Depth Controls Gaussian Entrance Rank and Avoidance of the First Gradient-Flow Saddle

DevFeed: [Finite Spectral Depth Controls Gaussian Entrance Rank and Avoidance of the First Gradient-Flow Saddle](<https://devfeed.tech/articles/finite-spectral-depth-controls-gaussian-entrance-rank-and-avoidance-of-the-first-gradient-flow-saddle-117005.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05022>)

Author: Yanlin Long

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [rust-ai](<https://devfeed.tech/topics/rust-ai.md>)

Tags: [bypass](<https://devfeed.tech/tags/bypass.md>), [condition](<https://devfeed.tech/tags/condition.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [flow](<https://devfeed.tech/tags/flow.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [matrix](<https://devfeed.tech/tags/matrix.md>), [muon](<https://devfeed.tech/tags/muon.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [newton](<https://devfeed.tech/tags/newton.md>), [rank](<https://devfeed.tech/tags/rank.md>), [saddle](<https://devfeed.tech/tags/saddle.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

In continuous-time matrix factorization, applying a finite number of Newton-Schulz iterations to the normalized matrix gradient (Muon) can change the rank selected early in training and let trajectories bypass the first nonzero stationary point reached by gradient flow. With an additional residual-inertia condition, the authors prove bypass for all sufficiently large finite iteration depths and give a sufficient threshold; a constructed example has threshold four.

### Source excerpt

arXiv:2611.05022v1 Announce Type: new Abstract: Matrix factorization provides a simple model for studying the training dynamics of neural networks. Recent studies of gradient flow under small initialization have analyzed how trajectories escape from a neighborhood of the origin and approach their first nonzero stationary point. In this paper, we study continuous-time dynamics that applies a finite number of Newton-Schulz iterations to the normalized matrix gradient (Muon). We show that the number of Newton-Schulz steps can change the rank selected during the early stages of training and allow the resulting trajectory to bypass the first nonzero stationary point approached by gradient flow. Under an additional residual-inertia condition, we prove that bypass occurs for all sufficiently large finite Newton-Schulz depths and provide an explicit sufficient threshold. We also construct an example in which the bypass threshold is four.

## Gradient Descent on Two ReLU Neurons: Global Landscape and Bifurcation Dynamics

DevFeed: [Gradient Descent on Two ReLU Neurons: Global Landscape and Bifurcation Dynamics](<https://devfeed.tech/articles/gradient-descent-on-two-relu-neurons-global-landscape-and-bifurcation-dynamics-111672.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.42118>)

Author: Binghua Li, Mengzhe Li, Denny Wu, Tianhao Wang

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [computer](<https://devfeed.tech/tags/computer.md>), [computer-science](<https://devfeed.tech/tags/computer-science.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [easy](<https://devfeed.tech/tags/easy.md>), [flow](<https://devfeed.tech/tags/flow.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [index](<https://devfeed.tech/tags/index.md>), [learning](<https://devfeed.tech/tags/learning.md>), [model](<https://devfeed.tech/tags/model.md>), [network](<https://devfeed.tech/tags/network.md>), [neural](<https://devfeed.tech/tags/neural.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [point](<https://devfeed.tech/tags/point.md>), [relu](<https://devfeed.tech/tags/relu.md>), [saddle](<https://devfeed.tech/tags/saddle.md>), [specialization](<https://devfeed.tech/tags/specialization.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [student](<https://devfeed.tech/tags/student.md>), [symmetry](<https://devfeed.tech/tags/symmetry.md>), [theory](<https://devfeed.tech/tags/theory.md>)

### AI overview

In a two-neuron ReLU teacher-student model with isotropic Gaussian inputs, population gradient flow from small initialization first drives the student neurons toward a bisector saddle, then they escape and specialize toward the teacher neurons. The analysis identifies an "easy" bisector direction carrying the leading signal and a "hard" splitting direction governing specialization. For the population squared-loss landscape, every critical point with both student weights nonzero is either a global minimum or a saddle. These results describe multi-phase symmetry breaking in this specific model.

### Source excerpt

arXiv:2610.42118v1 Announce Type: new Abstract: Understanding how gradient descent learns features remains a central challenge in neural network theory. We study this question in a minimal multi-index setting that already exhibits rich multi-phase gradient dynamics: a well-specified two-neuron ReLU teacher-student model under isotropic Gaussian inputs, trained by population gradient flow from small initialization. We show that the dynamics are organized by two directions: the "easy" bisector direction, which carries the leading signal, and the "hard" splitting direction, which governs specialization. To characterize the loss structure and learning behavior, we analyze the population squared-loss landscape and show that every critical point with both student weights nonzero is either a global minimum or a saddle. We then track the gradient-flow trajectory, showing that student neurons first collapse toward a bisector saddle before escaping and specializing to teacher neurons. Our results provide both a landscape and dynamics account of the multi-phase symmetry-breaking behavior that arises in a simple multi-index model and under standard algorithmic and architectural choices.

## Non-Commutative Lie-Algebraic State-Space Models: Finite-Division Variety Laws, Semidirect Cascades, and Gradient Saddle Escape

DevFeed: [Non-Commutative Lie-Algebraic State-Space Models: Finite-Division Variety Laws, Semidirect Cascades, and Gradient Saddle Escape](<https://devfeed.tech/articles/non-commutative-lie-algebraic-state-space-models-finite-division-variety-laws-semidirect-cascades-and-gradient-saddle-escape-103829.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.00329>)

Author: Zeyu Jia

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [algebra](<https://devfeed.tech/tags/algebra.md>), [commutator](<https://devfeed.tech/tags/commutator.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [imu](<https://devfeed.tech/tags/imu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [linear](<https://devfeed.tech/tags/linear.md>), [models](<https://devfeed.tech/tags/models.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [saddle](<https://devfeed.tech/tags/saddle.md>), [sharp](<https://devfeed.tech/tags/sharp.md>), [state](<https://devfeed.tech/tags/state.md>), [structured](<https://devfeed.tech/tags/structured.md>), [symmetry](<https://devfeed.tech/tags/symmetry.md>), [tangent](<https://devfeed.tech/tags/tangent.md>)

### AI overview

The paper proposes Lie-SSMs, which parameterize recurrent updates on Lie algebra tangent spaces using the matrix exponential to model non-commutative group structure. It reports that single-layer homogeneous contracting SO(3) and SU(2) heads obey finite-division limits, while depth-2 semidirect and Schreier-cocycle cascades represent several groups, including S5, in logarithmic sequence-length depth. The authors say contracting coupling collapses exponentially without Directional RMSNorm, and analyze a commutator saddle and coset-centroid dilution; dense prefix loss and factored readouts are used in reported group-word experiments. Across the listed tasks, depth-2 Lie-SO(3) models achieve 95.69% on UCI HAR, 87.16% on HAPT-PT (88.32% for 30-subject episodes), and 84.23% under displacement on REALDISP. The abstract also reports a 26.03x GPU scan speedup. These are claims and results as summarized in the supplied abstract.

### Source excerpt

arXiv:2610.00329v3 Announce Type: replace Abstract: Structured state-space models (SSMs) achieve linear-time inference and logarithmic-time parallel training via associative scans, yet homogeneous diagonal transitions generate solvable semigroups unable to track non-solvable group words or 3D rigid-body manifolds. We develop a theoretical and empirical framework for Non-Commutative Lie-Algebraic State-Space Models (Lie-SSMs), parameterizing recurrent updates on Lie algebra tangent spaces via the matrix exponential. First, via an o-minimal multi-rate cylindrical flag collar, we establish an unconditional single-layer finite-division criterion for homogeneous contracting $SO(3)$ and $SU(2)$ heads, proving odd-prime Sylow abelianity, odd-$p$-core centrality $[u^2,x]=e$, exponent-30 commutator laws ($[u^{30},v^{30}]=e$ in $SO(3)^m$ and $[u^{30},v^{30}]\in Z(G),[u^{30},v^{30}]^2=e$ in $SU(2)^m,SO(4)^m$, excluding $S_5$), and sharp threshold $d^*(S_5)=d^*(A_6)=5$ in $SO(5)$. Second, Depth-2 semidirect and Schreier $2$-cocycle cascades resolve $S_5$, $2I\cong SL_2(F_5)$, and $\mathrm{Heis}(F_p)$ in $O(\log T)$ depth, whereas unnormalized contracting coupling collapses exponentially without Directional RMSNorm. Third, we characterize the $O(\varepsilon^{c-1})$ Chen--Magnus--Hall--Witt commutator saddle and $1/|[G,G]|$ coset-centroid dilution, proving $\varepsilon=0$ symmetry breaking and local Polyak--\L{}ojasiewicz convergence ($\theta_{\max}<\pi$), solving $S_5$ ($99.90\%/99.40\%$ at $T=128$) and $D_{16}$ ($100.00\%$) via dense prefix loss and factored readouts. Across $Q_8,A_5,S_5,D_{16}$, Dyck-2, $SE_2(3)$ Left-Jacobian odometry, GPU scans ($26.03\times$ speedup), and three 3D IMU cohorts (UCI HAR, HAPT-PT 30-subject CV, and REALDISP), Depth-2 Lie-$SO(3)$ cascades reach $95.69\%\pm 0.14\%$ on HAR, $87.16\%\pm 0.64\%$ ($88.32\%\pm 0.93\%$ 30-subject episodes) on HAPT-PT, and $84.23\%\pm 1.00\%$ under displacement on REALDISP.

## Vendor-driven upgrades saddle IT leaders with huge 'software tax'

DevFeed: [Vendor-driven upgrades saddle IT leaders with huge 'software tax'](<https://devfeed.tech/articles/vendor-driven-upgrades-saddle-it-leaders-with-huge-software-tax-95248.md>)

Original publisher: [Read original article](<https://www.cio.com/article/4238087/vendor-driven-upgrades-saddle-it-leaders-with-huge-software-tax.html>)

Author: Grant Gross

Published: 2026-10-22T10:01:00Z

Content type: news

Language: en

Sources: [CIO](<https://devfeed.tech/sources/cio.md>)

Topics: [vendor lock-in](<https://devfeed.tech/topics/vendor-lock-in.md>), [Legacy Modernization](<https://devfeed.tech/topics/legacy-modernization.md>)

Tags: [000](<https://devfeed.tech/tags/000.md>), [about](<https://devfeed.tech/tags/about.md>), [also](<https://devfeed.tech/tags/also.md>), [better](<https://devfeed.tech/tags/better.md>), [budgeting](<https://devfeed.tech/tags/budgeting.md>), [budgeting-cloud-computing-enterprise-applications-it-management-saas](<https://devfeed.tech/tags/budgeting-cloud-computing-enterprise-applications-it-management-saas.md>), [but](<https://devfeed.tech/tags/but.md>), [cases](<https://devfeed.tech/tags/cases.md>), [ceo](<https://devfeed.tech/tags/ceo.md>), [changes](<https://devfeed.tech/tags/changes.md>), [cios](<https://devfeed.tech/tags/cios.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [company](<https://devfeed.tech/tags/company.md>), [control](<https://devfeed.tech/tags/control.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [customers](<https://devfeed.tech/tags/customers.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [enterprise-applications](<https://devfeed.tech/tags/enterprise-applications.md>), [it](<https://devfeed.tech/tags/it.md>), [it-management](<https://devfeed.tech/tags/it-management.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [saas](<https://devfeed.tech/tags/saas.md>), [saddle](<https://devfeed.tech/tags/saddle.md>), [software](<https://devfeed.tech/tags/software.md>), [storytype-newsanalysis](<https://devfeed.tech/tags/storytype-newsanalysis.md>), [tax](<https://devfeed.tech/tags/tax.md>), [upgrades](<https://devfeed.tech/tags/upgrades.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

A survey commissioned by Origina found that vendor-driven upgrades, migrations and licensing changes consume about 28% of large organizations' software budgets--nearly $12.9 million annually on average--and 95% of surveyed IT leaders said they had paid for an upgrade they believed would reduce performance. The survey covered large US and UK organizations. The text attributes buyers' limited leverage to switching costs, threatened loss of support and vendors' contract expertise. Suggested responses include modular, open standards and headless architecture, feasibility studies before renewals, shorter contracts, third-party or internal support, and sharing purchasing experiences. These approaches have tradeoffs: short contracts can cost more up front, and switching may be difficult or more expensive than staying. One CIO described a different experience, negotiating vendor-managed upgrades and patches, while still facing the staff burden of tracking patching.

### Source excerpt

CIOs experiencing sticker shock from recent software bills should be questioning how they got there. That's because IT leaders at large enterprises are paying a huge so-called software tax for vendor-driven upgrades, migrations, and licensing changes they would not have chosen for their organizations. About 28% of software budgets at large organizations -- an average of nearly $12.9 million per year -- are being eaten up by vendor-pushed initiatives, according to a survey commissioned by independent software maintenance partner Origina. Ninety-five percent of IT leaders surveyed said their organizations have paid for a vendor-driven upgrade in the past three years that they believed would actually reduce performance, according to the survey, which covered US and UK organizations with annual revenues higher than $1 billion or with more than 10,000 employees. The sheer size of this "software tax" surprised Tomás O'Leary, Origina's CEO, even though the company expected to see substantial costs for vendor-driven upgrades. "Software was supposed to make businesses better, and it does," he says. "But it's also imposing a hidden cost on companies, quite a significant one. I'm quite taken aback because we know this exists, but we never quantified it before." When vendors control your IT spend Big vendors have several major advantages over software customers, with the cost of switching or refusing to upgrade topping the list, O'Leary notes. In many cases, CIOs are faced with a loss of support if they turn down vendor-driven upgrades, leaving them with little room to refuse, he says. CIOs dependent on VMware are familiar with this squeeze play, as are those facing SAP deadlines. "CIOs are left in a situation where the alternative may be worse, so upgrading is the lesser of two evils," O'Leary adds. "When people buy technology, they don't realize how long this stuff is going to hang around. Nobody realized this thing was going to be embedded, and the dependency was going to be g