# ReconPotBench: Evaluating LLM Honeypots in Multi-Round Linux Reconnaissance

DevFeed: [ReconPotBench: Evaluating LLM Honeypots in Multi-Round Linux Reconnaissance](<https://devfeed.tech/articles/reconpotbench-evaluating-llm-honeypots-in-multi-round-linux-reconnaissance-139305.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20528>)

Author: Qianli Wang, Junchao Wu, Binkai Ou, Lidia S. Chao, Derek F. Wong

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Reliable AI outputs](<https://devfeed.tech/topics/reliable-ai-outputs.md>), [consistency](<https://devfeed.tech/topics/consistency.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [honeypots](<https://devfeed.tech/tags/honeypots.md>), [linux](<https://devfeed.tech/tags/linux.md>), [llm](<https://devfeed.tech/tags/llm.md>), [reconnaissance](<https://devfeed.tech/tags/reconnaissance.md>)

## AI overview

ReconPotBench evaluates whether Linux honeypots stay believable when reconnaissance commands depend on earlier results. It includes 1,032 three-round interactions across eight categories, with reference outputs from a real Linux environment, and compares Cowrie with open- and closed-source LLMs on authenticity, cross-round consistency, and reconnaissance value. Authenticity declines with interaction depth, and both types of LLM struggle to maintain coherent state. LLM honeypots outperform Cowrie in authenticity and reconnaissance value, while Cowrie is more consistent due to its rigid templates. Preserving network relationships across rounds is harder than tracking explicit entities such as ports.

## Source excerpt

arXiv:2611.20528v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to build interactive honeypots, yet existing evaluations mainly assess isolated command outputs and cannot determine whether a system remains believable when later commands depend on earlier results. We present ReconPotBench, a multi-round benchmark for Linux reconnaissance honeypots. It contains 1{,}032 three-round reconnaissance interactions across eight reconnaissance categories, with reference outputs collected from a real Linux environment. Under a unified testing protocol, we compare Cowrie, open-source LLMs, and closed-source LLMs along three dimensions: round-level authenticity, cross-round contextual consistency, and hacker-oriented reconnaissance value. Results show that multi-round evaluation exposes failures hidden by single-round testing, as authenticity declines with interaction depth and both open-source and closed-source LLMs struggle to maintain coherent state across rounds. LLM-based honeypots nevertheless outperform Cowrie in authenticity and reconnaissance value, while Cowrie remains more consistent because of its rigid templated behavior. Category-level analysis further shows that tasks requiring systems to preserve network relationships across rounds are harder than tasks centered on explicit entities such as ports. ReconPotBench enables reproducible evaluation of LLM honeypots across authenticity, entity consistency, and hacker-perspective utility in dependent multi-round interactions. The dataset is available at: https://github.com/qianli-wang/ReconPotBench