# P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior

DevFeed: [P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior](<https://devfeed.tech/articles/p-contextbench-from-personal-fact-recall-to-personalized-agent-behavior-111999.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.40959>)

Author: Grant Nasir Brown, Abdul Basir Bari, Anmol Tadikonda, Isabel Song

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [context](<https://devfeed.tech/topics/context.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [all](<https://devfeed.tech/tags/all.md>), [app](<https://devfeed.tech/tags/app.md>), [art](<https://devfeed.tech/tags/art.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [cases](<https://devfeed.tech/tags/cases.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [constraint](<https://devfeed.tech/tags/constraint.md>), [context](<https://devfeed.tech/tags/context.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [digital](<https://devfeed.tech/tags/digital.md>), [personal](<https://devfeed.tech/tags/personal.md>), [recall](<https://devfeed.tech/tags/recall.md>)

## AI overview

P-ContextBench evaluates whether language models can use fragmented personal histories to recover facts, track changing states, infer patterns from routines, and handle requests that conflict with a persona's constraints. It represents 100 synthetic personas through evolving mobile-app traces rather than static profiles, and includes 10,088 hidden facts, 18,261 trace fragments, and 4,810 verified test cases. Its validation pipeline checks plausibility, evidence recoverability and preservation, leakage, test grounding, and human-judge agreement. The abstract reports that cost-effective state-of-the-art models score below 60% on temporal tracking, pattern inference, and constraint-aware decisions; fewer than 10% of cases are answered correctly across all three repeated runs, and even a high-performing model remains inconsistent.

## Source excerpt

arXiv:2610.40959v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for personal tasks, yet evaluating their safe use of personal context remains difficult because such contexts are private, fragmented, dynamic, and socially situated. We introduce P-ContextBench, a realistic, verifiable, and scalable benchmark for evaluating personal-context understanding. P-ContextBench constructs population-structured, life-story-grounded digital humans from demographic seeds reflecting the U.S. population. Rather than using explicit, static profiles, P-ContextBench embeds traits, routines, goals, and constraints as implicit evidence across fragmented, evolving synthetic mobile-app records. It evaluates five capabilities: single-source fact recovery, multi-source fact recovery, temporal state tracking, inference of higher-level personal patterns from routine behavior, and safe handling of requests conflicting with persona-specific constraints through compliance, warning, clarification, or refusal. A multi-stage validation pipeline checks persona plausibility, context recoverability, information leakage, evidence preservation, test-case grounding, and human-judge agreement. We instantiate P-ContextBench with 100 personas, 10,088 hidden facts, 18,261 app-trace fragments, and 4,810 verified test cases. The pipeline supports configurable generation across scales and demographic groups. Cost-effective state-of-the-art (SOTA) LLMs relevant to personal agents achieve below 60% accuracy on temporal tracking, higher-level pattern inference, and constraint-aware action-boundary decisions, with fewer than 10% of test cases answered correctly across all three repeated runs. Even a high-performing SOTA model remains inconsistent on these capabilities. P-ContextBench provides measurable targets for improving the accuracy, reliability, and safety of LLM-based personal agents.