# Minerva: Reinforcement Learning with Verifiable Rewards for Cyber Threat Intelligence LLMs

DevFeed: [Minerva: Reinforcement Learning with Verifiable Rewards for Cyber Threat Intelligence LLMs](<https://devfeed.tech/articles/minerva-reinforcement-learning-with-verifiable-rewards-for-cyber-threat-intelligence-llms-123091.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.00513>)

Author: Md Tanvirul Alam, Aritran Piplai, Ionut Cardei, Nidhi Rastogi, Peter J Worth Jr

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Threat Hunting & Intel](<https://devfeed.tech/topics/threat-hunting-intel.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cyber](<https://devfeed.tech/tags/cyber.md>), [cyber-threat-intelligence](<https://devfeed.tech/tags/cyber-threat-intelligence.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llms](<https://devfeed.tech/tags/llms.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rlvr](<https://devfeed.tech/tags/rlvr.md>), [structured-outputs](<https://devfeed.tech/tags/structured-outputs.md>), [threat-intelligence](<https://devfeed.tech/tags/threat-intelligence.md>)

## AI overview

Minerva applies reinforcement learning with verifiable rewards to cyber threat intelligence tasks, using standards and community resources to check structured outputs and identifier predictions deterministically. Its dataset and training pipeline pair multiple CTI subtasks with task-specific verifiers. To address sparse rewards during rollouts, MinervaRL generates additional verified trajectories and distills them into the model. Across four backbones and 12 benchmarks, it raises mean score by 15.8 percentage points over the corresponding base models and 4.3 points over GRPO. These are reported aggregate results; the supplied text does not provide benchmark-level results or further methodological detail.

## Source excerpt

arXiv:2602.00513v4 Announce Type: replace Abstract: Cyber threat intelligence (CTI) analysts routinely convert noisy, unstructured security artifacts into standardized, automation-ready representations. Although large language models (LLMs) show promise for this task, existing approaches remain brittle when producing structured CTI outputs and have largely relied on supervised fine-tuning (SFT). In contrast, CTI standards and community-maintained resources define canonical identifiers and schemas that enable deterministic verification of model outputs. We leverage this structure to study reinforcement learning with verifiable rewards (RLVR) for CTI tasks. We introduce Minerva, a unified dataset and training pipeline spanning multiple CTI subtasks, each paired with task-specific verifiers that score structured outputs and identifier predictions. To address reward sparsity during rollout, we propose MinervaRL, an auxiliary self-training mechanism that generates additional verified trajectories and distills them back into the model. Averaged across four backbones and 12 CTI benchmarks, MinervaRL improves the mean score by 15.8 percentage points over the corresponding base models and by 4.3 points over GRPO. The project page is available at https://maveryn.github.io/minerva-tmlr/.