# Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football

DevFeed: [Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football](<https://devfeed.tech/articles/strategic-multi-agent-learning-for-interpretable-action-valuation-of-all-players-in-football-103834.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.05961>)

Author: Kenjiro Ide, Taiga Someya, Kohei Kawaguchi, Keisuke Fujii

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Sports](<https://devfeed.tech/topics/sports.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [defensive](<https://devfeed.tech/tags/defensive.md>), [football](<https://devfeed.tech/tags/football.md>), [game](<https://devfeed.tech/tags/game.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [independent](<https://devfeed.tech/tags/independent.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [learning](<https://devfeed.tech/tags/learning.md>), [linear](<https://devfeed.tech/tags/linear.md>), [markov](<https://devfeed.tech/tags/markov.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [multi-agent-system](<https://devfeed.tech/tags/multi-agent-system.md>), [position](<https://devfeed.tech/tags/position.md>), [positioning](<https://devfeed.tech/tags/positioning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [valuation](<https://devfeed.tech/tags/valuation.md>)

## AI overview

The proposed MPE-inspired framework values actions for all 22 football players by modeling each possession as a finite-horizon dynamic game. It uses Expandable Decision-Making States (EDMS) and decomposes Q-values into a successor-feature basis and linear reward weights; the basis is initialized with linear temporal-difference learning and refined nonlinearly from observed transitions. On tracking and event data from 95 J1 League matches, the study compares it with an independent reinforcement-learning baseline using Position and Velocity States (PVS) and EDMS. With EDMS fixed, the baseline most often favored forward movement (99.21% of evaluated off-ball states), while the proposed method's most frequent direction accounted for 17.63%, indicating more varied valuations. Qualitative analysis shows context-dependent off-ball values but identifies limitations in representing defensive positioning. The abstract describes an approach that supports analysis under alternative tactical reward weights.

## Source excerpt

arXiv:2610.05961v2 Announce Type: replace Abstract: Valuing player actions in football, a complex multi-agent system, requires accounting for strategic interactions among 22 players, including off-ball movements and defensive positioning. Existing reinforcement-learning-based valuation methods commonly incorporate other players through the observed state while retaining independent-agent value formulations. To our knowledge, we introduce the first Markov perfect equilibrium (MPE)-inspired framework for valuing the actions of all 22 players using real football tracking and event data, formulating each possession as a finite-horizon multi-agent dynamic game. To improve interpretability, the framework uses Expandable Decision-Making States (EDMS) and decomposes each Q-value into a successor-feature basis and a linear reward-weight vector. The value basis is estimated through a two-stage procedure combining linear temporal-difference initialization with nonlinear iterative refinement from observed state-action transitions. Using tracking and event data from 95 J1 League matches, we compare the proposed formulation with an independent reinforcement learning baseline under both Position and Velocity States (PVS) and EDMS representations. With EDMS held fixed, the independent baseline assigns the highest value to forward movement in 99.21% of evaluated off-ball states, whereas the most frequent direction under the proposed formulation accounts for 17.63%. Qualitative analyses illustrate context-dependent off-ball valuations and highlight limitations of the current state representation for defensive positioning. Overall, the framework provides an MPE-inspired approach to interpretable action valuation in football, enabling analysis under alternative tactical reward weights.