# Interpretable Feature Discovery from Unstructured Data Using LLMs and Statistical Choice Models

DevFeed: [Interpretable Feature Discovery from Unstructured Data Using LLMs and Statistical Choice Models](<https://devfeed.tech/articles/interpretable-feature-discovery-from-unstructured-data-using-llms-and-statistical-choice-models-108014.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38082>)

Author: Baptiste Arnaudo, Amine Bennouna, S\\'ebastien Martin

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Feature Engineering](<https://devfeed.tech/topics/feature-engineering.md>), [Data analysis](<https://devfeed.tech/topics/data-analysis.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [audit](<https://devfeed.tech/tags/audit.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [choice](<https://devfeed.tech/tags/choice.md>), [combines](<https://devfeed.tech/tags/combines.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [decision-making](<https://devfeed.tech/tags/decision-making.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [hiring](<https://devfeed.tech/tags/hiring.md>), [idea](<https://devfeed.tech/tags/idea.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llms](<https://devfeed.tech/tags/llms.md>), [models](<https://devfeed.tech/tags/models.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [unstructured-data](<https://devfeed.tech/tags/unstructured-data.md>)

## AI overview

The method discovers interpretable features by asking an LLM to explain differences in pairwise item comparisons, then combining those rationales in a statistical choice model to identify relevant features and score each item against them. On synthetic cover letters, it reportedly recovers known features significantly better than direct LLM feature extraction. On ChangeMyView, it finds features associated with persuasion with predictive accuracy comparable to established baselines. The rationales can also audit LLM decisions by decomposing an overall score into interpretable features. The supplied abstract does not specify implementation details or the discovered features.

## Source excerpt

arXiv:2610.38082v1 Announce Type: cross Abstract: Feature extraction from unstructured data typically relies on opaque embeddings unsuited to interpretable decision-making. Interpretable features, on the other hand, typically require costly, and often infeasible, hand-engineering. Large language models (LLMs) offer a natural bridge, as they can read raw data and describe it in interpretable terms. One could simply ask an LLM to extract features from a large sample of the dataset, but context limitations and diluted attention can lead to unsatisfactory results. We introduce a method that automatically discovers interpretable, decision-relevant features from unstructured data. Our key idea is to perform pairwise comparisons between items, asking an LLM to articulate, in interpretable terms, the features distinguishing each pair. We treat these explanations, or rationales, as data: we develop a statistical choice model that combines these rationales across the comparisons to identify relevant features and estimate per-item, per-feature scores. We first test our method on a synthetic hiring dataset of cover letters with known ground-truth features and show that it significantly outperforms direct LLM feature extraction in feature recovery. We then apply it to the ChangeMyView (Reddit) debate dataset, where it discovers several interpretable features associated with persuasion while reaching predictive accuracy comparable to established baselines. We also show how the same rationales can be used to audit LLM decisions, yielding a holistic score whose decomposition over interpretable features reveals what the LLM rewards.