# I Don't Care How You Hide It, I Care What You Do: Runtime Detection of Malicious ML Models

DevFeed: [I Don't Care How You Hide It, I Care What You Do: Runtime Detection of Malicious ML Models](<https://devfeed.tech/articles/i-don-t-care-how-you-hide-it-i-care-what-you-do-runtime-detection-of-malicious-ml-models-122787.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.08745>)

Author: Hala Ali, Andrew Case

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [AI software supply chain security](<https://devfeed.tech/topics/ai-software-supply-chain-security.md>), [Threat Research](<https://devfeed.tech/topics/threat-research.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [behavioral-detection](<https://devfeed.tech/tags/behavioral-detection.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [detection](<https://devfeed.tech/tags/detection.md>), [ebpf](<https://devfeed.tech/tags/ebpf.md>), [evasion](<https://devfeed.tech/tags/evasion.md>), [exfiltration](<https://devfeed.tech/tags/exfiltration.md>), [false-positives](<https://devfeed.tech/tags/false-positives.md>), [fickling](<https://devfeed.tech/tags/fickling.md>), [formats](<https://devfeed.tech/tags/formats.md>), [frameworks](<https://devfeed.tech/tags/frameworks.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [instrumentation](<https://devfeed.tech/tags/instrumentation.md>), [loading](<https://devfeed.tech/tags/loading.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [malicious](<https://devfeed.tech/tags/malicious.md>), [median](<https://devfeed.tech/tags/median.md>), [ml](<https://devfeed.tech/tags/ml.md>), [ml-models](<https://devfeed.tech/tags/ml-models.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [runtime-detection](<https://devfeed.tech/tags/runtime-detection.md>)

## AI overview

ModelShield detects malicious ML models by attributing runtime system behavior to model operations. It instruments framework APIs and uses eBPF syscall tracing across spawned threads and processes, then classifies attributed syscall sequences with 195 patterns mapped to MITRE ATT&CK. The implementation supports 17 frameworks and 36 formats. In the reported evaluation, it had zero false positives; false negatives were limited to payloads that produced no malicious runtime behavior. It detected all 29 tested evasion variants, while existing scanners did not achieve complete coverage. In scans of 145,000 public-platform models, it found 41 malicious models on Hugging Face; ModelScan, PickleScan, Fickling, and ModelAudit detected 29%, 56%, 80%, and 93% of these, respectively. Runtime analysis took a median 12.3 seconds, with 90% of scans completing within 35.5 seconds.

## Source excerpt

arXiv:2611.08745v1 Announce Type: new Abstract: Pre-trained machine learning (ML) models shared through public platforms can be weaponized to execute malicious payloads during loading or inference. Existing scanners prove inadequate for detecting malicious models as they rely on static analysis and function denylists, support limited serialization formats, and miss attacks that replace denylisted functions with equivalent alternatives, exploit unsupported loading paths, or abuse legitimate framework APIs. To address this problem, we present ModelShield, the first runtime behavioral detection system for malicious ML models that attributes observed system-level behavior to specific model operations. It combines application-level instrumentation of ML framework APIs with eBPF-based syscall tracing, tracking model behavior across spawned threads and processes to separate model-triggered activity from environment noise. Attributed syscall sequences are classified using 195 detection patterns mapped to MITRE ATT&CK techniques. We implemented ModelShield to support 17 ML frameworks and 36 formats. On an independent dataset, it is the only evaluated tool to achieve zero false positives, with false negatives limited to payloads that produce no malicious runtime behavior. It detected all 29 published evasion variants spanning function substitution, loading-path obfuscation, and TensorFlow API abuse, where no existing scanner achieved complete coverage. Across 145,000 models scanned from public platforms, it captured 41 malicious models on Hugging Face, including a reverse-shell payload and a payload that performed host reconnaissance and outbound exfiltration. Among these, ModelScan, PickleScan, Fickling, and ModelAudit detected only 29%, 56%, 80%, and 93%, respectively. Despite runtime overhead, ModelShield has a median analysis time of 12.3 seconds, with 90% of scans completing within 35.5 seconds, making it practical for hub-scale screening.