# Block-Sparse Gauss-Newton Optimization of Finite Basis Physics-Informed Neural Networks: Sparse Automatic Differentiation Approach

DevFeed: [Block-Sparse Gauss-Newton Optimization of Finite Basis Physics-Informed Neural Networks: Sparse Automatic Differentiation Approach](<https://devfeed.tech/articles/block-sparse-gauss-newton-optimization-of-finite-basis-physics-informed-neural-networks-sparse-automatic-differentiation-approach-128678.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13231>)

Author: Denis Korolev, Adrian Hill, Jeeeun Lee

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Physics-guided deep learning](<https://devfeed.tech/topics/physics-guided-deep-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Operations research and optimization](<https://devfeed.tech/topics/operations-research-and-optimization.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [automatic](<https://devfeed.tech/tags/automatic.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-na](<https://devfeed.tech/tags/cs-na.md>), [domain-decomposition](<https://devfeed.tech/tags/domain-decomposition.md>), [graph-coloring](<https://devfeed.tech/tags/graph-coloring.md>), [linear-algebra](<https://devfeed.tech/tags/linear-algebra.md>), [math-na](<https://devfeed.tech/tags/math-na.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [physics-informed](<https://devfeed.tech/tags/physics-informed.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>)

## AI overview

The paper uses FBPINN overlapping domain decomposition to exploit block sparsity in the residual Jacobian and Gauss-Newton matrix. Graph coloring groups structurally orthogonal Jacobian columns or rows for sparse automatic differentiation, reducing differentiation cost and memory while retaining structurally nonzero derivatives. The compressed Jacobian is assembled directly in block-sparse form, allowing sparse direct solvers or preconditioned iterative methods. In reported multiscale PDE experiments, the method achieves high accuracy and substantially faster optimization than first-order baselines, while remaining tractable at large problem sizes.

## Source excerpt

arXiv:2611.13231v1 Announce Type: cross Abstract: In this work, we develop an efficient Gauss-Newton optimization approach for finite basis physics-informed neural networks (FBPINNs). The key ingredient is the overlapping domain decomposition architecture of FBPINNs, which localizes the influence of each subnetwork and induces block sparsity in both the residual Jacobian and the corresponding Gauss-Newton matrix. We characterize this structure and exploit it through sparse automatic differentiation, using graph coloring to partition structurally orthogonal Jacobian columns or rows. This reduces differentiation costs and memory requirements while preserving all structurally nonzero derivative information. The resulting compressed Jacobian representations allow us to assemble the Gauss-Newton system directly in block-sparse form, enabling the use of sparse direct solvers and preconditioned iterative methods. In this way, the otherwise expensive second-order optimization problem is reduced to scalable sparse linear algebra. We demonstrate the approach on multiscale PDEs, where accurate approximation requires rich neural network parametrizations and large numbers of collocation points. Numerical experiments show that our block-sparse Gauss-Newton method attains high accuracy and substantially accelerates optimization relative to first-order baselines while remaining computationally tractable for large problem sizes.