# On-device AI

Artificial intelligence and machine-learning inference performed locally on a user's device rather than in the cloud.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Hybrid intelligence on the Web: Introducing Aion Instruct and Prompt API in Microsoft Edge

DevFeed: [Hybrid intelligence on the Web: Introducing Aion Instruct and Prompt API in Microsoft Edge](<https://devfeed.tech/articles/hybrid-intelligence-on-the-web-introducing-aion-instruct-and-prompt-api-in-microsoft-edge-146576.md>)

Original publisher: [Read original article](<https://blogs.windows.com/msedgedev/2026/11/19/hybrid-intelligence-on-the-web-introducing-aion-instruct-and-prompt-api-in-microsoft-edge/>)

Author: Patrick Brosset

Published: 2026-11-19T23:08:46Z

Content type: release

Language: en

Sources: [Microsoft Edge](<https://devfeed.tech/sources/microsoft-edge-blog.md>)

Topics: [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Edge](<https://devfeed.tech/topics/edge.md>), [Web](<https://devfeed.tech/topics/web.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-apis](<https://devfeed.tech/tags/ai-apis.md>), [api](<https://devfeed.tech/tags/api.md>), [edge](<https://devfeed.tech/tags/edge.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [microsoft-edge](<https://devfeed.tech/tags/microsoft-edge.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [on-device-ai](<https://devfeed.tech/tags/on-device-ai.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [sites](<https://devfeed.tech/tags/sites.md>), [the-web](<https://devfeed.tech/tags/the-web.md>), [uncategorized](<https://devfeed.tech/tags/uncategorized.md>), [web](<https://devfeed.tech/tags/web.md>), [web-platform](<https://devfeed.tech/tags/web-platform.md>)

### AI overview

Microsoft Edge is positioning on-device AI as one part of a hybrid approach: sites can use browser models when available and suitable, then switch to a configured cloud provider for broader device coverage or more powerful inference. Edge's built-in model is moving from Phi-4-mini to Aion Instruct, a 2.3B-parameter model that Microsoft says is smaller, faster, and available on more devices while offering comparable quality for common web tasks. It downloads only when first needed on devices that can run it; users can disable on-device AI in Edge settings, which removes the model and stops future downloads. Developers can test the Prompt API on live sites through an Edge origin trial. The experimental Decisions API, in preview in Edge Canary and Dev, returns probabilities and confidence for yes/no, multiple-choice, or scoring questions about text in one pass without generating text; both its API and implementation may change.

### Source excerpt

Rapidly evolving AI capabilities are redefining what sites can do for users on the web. With on-device AI in Microsoft The post Hybrid intelligence on the Web: Introducing Aion Instruct and Prompt API in Microsoft Edge appeared first on Microsoft Edge Blog.

## Label-Free Role-Aware Segmentation of Classroom Recordings: Recovering the Teacher from Transferable Structure

DevFeed: [Label-Free Role-Aware Segmentation of Classroom Recordings: Recovering the Teacher from Transferable Structure](<https://devfeed.tech/articles/label-free-role-aware-segmentation-of-classroom-recordings-recovering-the-teacher-from-transferable-structure-143413.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23609>)

Author: Nitin Jain, Tim F\\"utterer, Babette B\\"uhler, \\c{C}a\\u{g}atay Y{\\i}ld{\\i}z

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [audio](<https://devfeed.tech/tags/audio.md>), [classroom](<https://devfeed.tech/tags/classroom.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sd](<https://devfeed.tech/tags/cs-sd.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [device](<https://devfeed.tech/tags/device.md>), [diarization](<https://devfeed.tech/tags/diarization.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [english](<https://devfeed.tech/tags/english.md>), [error](<https://devfeed.tech/tags/error.md>), [f1](<https://devfeed.tech/tags/f1.md>), [failures](<https://devfeed.tech/tags/failures.md>), [faster](<https://devfeed.tech/tags/faster.md>), [flag](<https://devfeed.tech/tags/flag.md>), [gdpr](<https://devfeed.tech/tags/gdpr.md>), [human-review](<https://devfeed.tech/tags/human-review.md>), [inference](<https://devfeed.tech/tags/inference.md>), [recordings](<https://devfeed.tech/tags/recordings.md>), [structure](<https://devfeed.tech/tags/structure.md>)

### AI overview

The paper proposes an on-device pipeline that identifies teacher speech without labels at inference or per-session calibration, addressing a constraint in which classroom audio must remain on the recording device. It combines voice activity detection, pretrained speaker embeddings, agglomerative clustering, and logistic regression that assigns the teacher role from session-relative structure rather than voice identity. In leave-one-teacher-out evaluation on 292 recordings from 63 teachers, teacher F1 was 0.81 for English and 0.87 for German, exceeding the reported enrollment-based baselines and 30-second supervised enrollment ceiling. On a commodity smartphone, performance matched pyannote while running 4.8 times faster at comparable peak memory. Filtering audio raised teacher word error rate only slightly, from 0.26 to 0.28. Failures were concentrated where teacher and student voices were acoustically inseparable; the method can flag such sessions for human review.

### Source excerpt

arXiv:2611.23609v1 Announce Type: new Abstract: How much the teacher talks and when are among the most informative and costly-to-measure signals of teaching. Yet, classroom audio is among the most sensitive data an AI system can process: recordings capture minors and fall under GDPR, preventing raw audio from leaving the recording device. We present a teacher-student segmentation pipeline designed to meet exactly this constraint: label-free at inference, running on a recording device, and requiring no per-session calibration. The pipeline chains a lightweight voice activity detector, compactly pretrained speaker embeddings, agglomerative clustering, and a small logistic regression that assigns the teacher role based on session-relative structure rather than voice identity. We show that our structural cue transfers across classrooms, languages, and acoustic conditions. In leave-one-teacher-out cross-validation on 292 recordings from 63 teachers across English and German corpora, our pipeline achieves teacher F1 of 0.81 and 0.87, respectively, outperforming both enrollment-based baselines and a 30-second supervised enrollment ceiling. On a commodity smartphone, its performance matches pyannote, a strong neural diarization tool, while running 4.8x faster at comparable peak memory at a real-time factor of ~0.18. We also find that transcribing filtered audio only marginally increases teacher's word error rate from $0.26$ to 0.28. Residual failures concentrate where teacher and student voices are not acoustically separable. However, we show how to flag these sessions label-free at inference so that they can be routed to human review, enabling selective, privacy-preserving deployment.

## Hypnos: A Dual-Phase End-to-End Platform for Scalable Continual Learning at the Edge

DevFeed: [Hypnos: A Dual-Phase End-to-End Platform for Scalable Continual Learning at the Edge](<https://devfeed.tech/articles/hypnos-a-dual-phase-end-to-end-platform-for-scalable-continual-learning-at-the-edge-142817.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24637>)

Author: Guilherme Guedes, Marco Apolinario Lainez, Charlotte Frenkel

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [tinyml](<https://devfeed.tech/topics/tinyml.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [fpga](<https://devfeed.tech/topics/fpga.md>)

Tags: [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [charger](<https://devfeed.tech/tags/charger.md>), [charging](<https://devfeed.tech/tags/charging.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [edge](<https://devfeed.tech/tags/edge.md>), [fpga](<https://devfeed.tech/tags/fpga.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [mcu](<https://devfeed.tech/tags/mcu.md>), [memory](<https://devfeed.tech/tags/memory.md>), [online](<https://devfeed.tech/tags/online.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [platform](<https://devfeed.tech/tags/platform.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [risc-v](<https://devfeed.tech/tags/risc-v.md>), [sample](<https://devfeed.tech/tags/sample.md>), [sensor](<https://devfeed.tech/tags/sensor.md>), [sleep](<https://devfeed.tech/tags/sleep.md>), [system-on-chip](<https://devfeed.tech/tags/system-on-chip.md>)

### AI overview

Hypnos splits edge continual learning between a low-resource "wake" phase and a better-resourced "sleep" phase. A RISC-V microcontroller handles inference and lightweight, gradient-free prototype updates as sensor samples arrive; a smart charger uses rehearsal-based gradient updates for consolidation when more energy and memory are available. On CORe50 NICv2-391, deployed on a ZCU104 FPGA and synthesized in 28-nm technology, it reports 78.6%-81.6% accuracy and 7.79-10.54 seconds per sample, depending on activation precision. Compared with quantized latent replay, it reports 3.8x fewer learning steps and 22x less learning compute, with higher accuracy and comparable latency.

### Source excerpt

arXiv:2611.24637v1 Announce Type: cross Abstract: Continual learning (CL) enables neural networks to adapt to non-stationary data streams by mitigating catastrophic forgetting, offering a path to robust, private, and adaptable edge devices. Yet, porting CL to real-world edge devices remains challenging for two reasons: (i) online operation, where sensor data arrives one sample at a time and the device must adapt on-the-fly, and (ii) the low footprint of edge devices, which makes traditional error backpropagation training prohibitive. Inspired by the brain's wake and sleep cycles, the former for rapid adaptation with minimal resources and the latter for long-term consolidation, we introduce Hypnos, an open-source dual-phase platform for efficient CL at the edge. In the wake phase (e.g., daily operation), Hypnos relies on a low-footprint system-on-chip with a RISC-V microcontroller unit (MCU) targeting efficient inference and supporting lightweight gradient-free prototype updates. In the sleep phase (e.g., night-time charging), where wall-plug energy and larger memory budgets are available, a smart charger performs rehearsal-based gradient updates for consolidation. Deployed on a ZCU104 field-programmable gate array (FPGA) and synthesized in a 28-nm process, Hypnos is benchmarked on the CORe50 NICv2-391 protocol. It sets a new state-of-the-art accuracy of 78.6%-81.6% for edge CL systems, with an on-device latency of 7.79s-10.54s per sample depending on activation precision, and requires $3.8\times$ fewer learning steps and $22\times$ less learning compute than quantized latent replay at higher accuracy and comparable latency.

## Learning through Discernment: when a small language model learns from user feedback on the job, and when it decays

DevFeed: [Learning through Discernment: when a small language model learns from user feedback on the job, and when it decays](<https://devfeed.tech/articles/learning-through-discernment-when-a-small-language-model-learns-from-user-feedback-on-the-job-and-when-it-decays-142695.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23716>)

Author: Sirisilp Kongsilp

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [lora](<https://devfeed.tech/topics/lora.md>), [rlhf](<https://devfeed.tech/topics/rlhf.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [job](<https://devfeed.tech/tags/job.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [learning](<https://devfeed.tech/tags/learning.md>), [small-language-model](<https://devfeed.tech/tags/small-language-model.md>), [user](<https://devfeed.tech/tags/user.md>)

### AI overview

In experiments with Qwen3 0.6B and 1.7B plus LoRA adapters, nightly learning from user feedback depended on whether the judge filtered out failures and whether users could see outcomes. A feedback sense that treated polite give-ups as success collapsed in 3 of 5 lives. A version that first checked for a clear user verdict stayed near oracle performance in 4 of 5 lives for two weeks, then faded to self-judging levels by day 60 as clean lessons became scarce. On τ²-bench, it interpreted reactions correctly, but 80% of lessons were failed attempts users had thanked; across 97 lives, the share of failed lessons strongly predicted outcomes (Spearman ρ = −0.81). The abstract concludes that user-guided learning held up only when failures were excluded and users could observe outcomes, and warns that skewed lessons may narrow the model.

### Source excerpt

arXiv:2611.23716v1 Announce Type: cross Abstract: Small language models on devices could keep improving by learning, overnight, from their own daily work. Without a true outcome, however, something must decide which attempts are worth learning from, and both the model's own judgment and the user's reaction can be wrong. We study a small model (Qwen3 0.6B and 1.7B) with a LoRA adapter, fine-tuned each night on the attempts a judge accepts, over 127 lives of up to a year on WebShop and $\tau^2$-bench. We compare the true outcome, the user's stated verdict and the model's own judgment with a separate, frozen "feedback sense" that reads the user's reaction, built in three versions. Judging itself, the model improved and then decayed, over a year and whatever its adapter. A sense that took polite give-ups for success collapsed 3 of 5 lives; one built for discernment, which first asks whether the user gave a clear verdict, kept 4 of 5 near the oracle for two weeks, then faded on clean but scarce lessons, to the level of self-judging by day 60. When the user cannot see the outcome ($\tau^2$-bench), the sense read reactions correctly, yet 80% of its lessons were failures the customer had thanked. Across 97 lives, the share of failed attempts among the lessons predicted the outcome (Spearman $\rho$ = -0.81). In our experiments, learning from users held up only where the judge kept failures out and the user could see the outcome; lopsided lessons may also narrow the model.

## App developers turn to QVAC for AI tooling without the cloud

DevFeed: [App developers turn to QVAC for AI tooling without the cloud](<https://devfeed.tech/articles/app-developers-turn-to-qvac-for-ai-tooling-without-the-cloud-141283.md>)

Original publisher: [Read original article](<https://www.networkworld.com/article/4250425/app-developers-turn-to-qvac-for-ai-tooling-without-the-cloud.html>)

Author: tether

Published: 2026-11-18T21:14:44Z

Content type: article

Language: en

Sources: [Network World](<https://devfeed.tech/sources/network-world.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-apps](<https://devfeed.tech/tags/ai-apps.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [api](<https://devfeed.tech/tags/api.md>), [app](<https://devfeed.tech/tags/app.md>), [apple-silicon](<https://devfeed.tech/tags/apple-silicon.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-machine-learning](<https://devfeed.tech/tags/artificial-intelligence-machine-learning.md>), [backends](<https://devfeed.tech/tags/backends.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data](<https://devfeed.tech/tags/data.md>), [desktop](<https://devfeed.tech/tags/desktop.md>), [developers](<https://devfeed.tech/tags/developers.md>), [development-kit](<https://devfeed.tech/tags/development-kit.md>), [device](<https://devfeed.tech/tags/device.md>), [edge](<https://devfeed.tech/tags/edge.md>), [edge-ai](<https://devfeed.tech/tags/edge-ai.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [engine](<https://devfeed.tech/tags/engine.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [storytype-brandpost](<https://devfeed.tech/tags/storytype-brandpost.md>), [tooling](<https://devfeed.tech/tags/tooling.md>)

### AI overview

Tether's QVAC SDK combines local model inference and fine tuning with tools for offline retrieval-augmented generation, aiming to reduce cloud dependence, per-request costs, and exposure of user data. Its Fabric engine builds on llama.cpp and GGML, with Vulkan, Apple Silicon's Metal and Core ML, and CUDA backends. The SDK's TurboVec integration is described as reducing embedding memory use by up to 8x with 4-bit quantization, helping vector search fit consumer hardware. QVAC also supports quantized open-source models and Tether's compact Psy models; the source cites an 800-million-parameter African-language translation model outperforming a 27-billion-parameter TranslateGemma on key benchmarks. These claims present local AI as more accessible, though the excerpt does not provide benchmark details or specify hardware requirements.

### Source excerpt

In most instances today, adding AI to an app means sending user data to a third-party API, paying per request, and hoping the connection holds. Tether's QVAC is built on the opposite premise. It runs AI models directly on the user's device. Data never leaves the hardware, there are no per-request costs, no rate limits, and no need for an internet connection once a model is downloaded. QVAC SDK: the next milestone in cloud-free AI While open-source tools and standards like llama.cpp, GGML, and ONNX have made local AI much more accessible in recent years, creating new possibilities for consumer devices, the edge AI ecosystem remains fragmented. Against this backdrop, in 2026, Tether released a software development kit (SDK) that pulls together the best open tooling from across different domains of machine learning. The centerpiece of Tether's QVAC SDK is Fabric. Built on llama.cpp, it expands the engine to enable on-device fine tuning and additional AI capabilities--opening access to memory-intensive workloads previously only possible with expensive hardware or cloud services. Building on GGML's modular framework, QVAC Fabric is primarily oriented toward three backends: Vulkan, which covers most consumer-grade desktop and mobile chips; Metal and Core ML for Apple Silicon; and CUDA for higher-end NVIDIA GPU setups. The QVAC SDK empowers developers with a unified interface for building multiplatform AI apps that run on-device. But although significant progress has been made in bringing inference to consumer devices, text generation alone is only one part of the pipeline that makes modern AI useful. Bringing search to edge AI RAG (retrieval-augmented generation) is what equips AI to import and process contextual information to enhance its output. Instead of language models relying solely on their training data to answer questions, RAG lets them answer using a specific set of documents. Implementing RAG into an AI app incurs significant memory overhead versus running a mod

## Flipper One raises more than EUR 10 million on Kickstarter in just 24 hours

DevFeed: [Flipper One raises more than EUR 10 million on Kickstarter in just 24 hours](<https://devfeed.tech/articles/flipper-one-raises-more-than-10-million-on-kickstarter-in-just-24-hours-141168.md>)

Original publisher: [Read original article](<https://www.techspot.com/news/114601-flipper-one-blows-past-kickstarter-goal-24-hours.html>)

Author: Alfonso Maruccia

Published: 2026-11-18T19:54:00Z

Content type: news

Language: en

Sources: [TechSpot](<https://devfeed.tech/sources/techspot.md>)

Topics: [Flipper Zero](<https://devfeed.tech/topics/flipperzero.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [pc](<https://devfeed.tech/topics/pc.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [applications](<https://devfeed.tech/tags/applications.md>), [arm](<https://devfeed.tech/tags/arm.md>), [cellular](<https://devfeed.tech/tags/cellular.md>), [computer](<https://devfeed.tech/tags/computer.md>), [connectivity](<https://devfeed.tech/tags/connectivity.md>), [crowdfunding](<https://devfeed.tech/tags/crowdfunding.md>), [cyberdeck](<https://devfeed.tech/tags/cyberdeck.md>), [desktop](<https://devfeed.tech/tags/desktop.md>), [diy](<https://devfeed.tech/tags/diy.md>), [electronics](<https://devfeed.tech/tags/electronics.md>), [embedded-linux](<https://devfeed.tech/tags/embedded-linux.md>), [ethernet](<https://devfeed.tech/tags/ethernet.md>), [flipper](<https://devfeed.tech/tags/flipper.md>), [flipper-devices](<https://devfeed.tech/tags/flipper-devices.md>), [flipper-one](<https://devfeed.tech/tags/flipper-one.md>), [flipper-zero](<https://devfeed.tech/tags/flipper-zero.md>), [hacking](<https://devfeed.tech/tags/hacking.md>), [kickstarter](<https://devfeed.tech/tags/kickstarter.md>), [linux](<https://devfeed.tech/tags/linux.md>), [networking](<https://devfeed.tech/tags/networking.md>), [security](<https://devfeed.tech/tags/security.md>), [tech-culture](<https://devfeed.tech/tags/tech-culture.md>)

### AI overview

Flipper One raised more than EUR 10.7 million against a EUR 272,981 Kickstarter target. Flipper Devices describes it as a portable Arm-based Linux computer for IP-connected security work, distinct from the Flipper Zero, a low-power microcontroller tool for protocols such as NFC, RFID, sub-GHz radio and UART. Flipper One combines a Rockchip RK3576 system-on-chip with an RP2350 microcontroller, and is presented as supporting Wi-Fi, Ethernet, 5G, satellite communications, software-defined radio, local AI models, desktop mode, network analysis and hardware expansion. The excerpt lists a $399 (EUR 364) preorder price, 25% below estimated retail, and expected shipping from July 2027 in most regions; component and chip shortages could delay production and delivery.

### Source excerpt

The Flipper One Kickstarter campaign is now live, and the developers have already raised more than EUR 10,700,000 against an initial funding target of EUR 272,981. Interest in the new Linux-based hacking tool is clearly strong, but Flipper Devices warns that it is not simply a more powerful replacement for the previously... Read Entire Article

## Edge AI Moves Agricultural Intelligence from the Cloud to the Field

DevFeed: [Edge AI Moves Agricultural Intelligence from the Cloud to the Field](<https://devfeed.tech/articles/edge-ai-moves-agricultural-intelligence-from-the-cloud-to-the-field-140174.md>)

Original publisher: [Read original article](<https://www.eetimes.com/edge-ai-moves-agricultural-intelligence-from-the-cloud-to-the-field/>)

Author: Pat Brans

Published: 2026-11-18T06:59:00Z

Content type: news

Language: en

Sources: [EE Times](<https://devfeed.tech/sources/ee-times.md>)

Topics: [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Jetson](<https://devfeed.tech/topics/jetson.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [agriculture](<https://devfeed.tech/tags/agriculture.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [analog](<https://devfeed.tech/tags/analog.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [autonomous-driving](<https://devfeed.tech/tags/autonomous-driving.md>), [business](<https://devfeed.tech/tags/business.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [community](<https://devfeed.tech/tags/community.md>), [computing](<https://devfeed.tech/tags/computing.md>), [designline](<https://devfeed.tech/tags/designline.md>), [digital](<https://devfeed.tech/tags/digital.md>), [edge](<https://devfeed.tech/tags/edge.md>), [edge-ai](<https://devfeed.tech/tags/edge-ai.md>), [education](<https://devfeed.tech/tags/education.md>), [ee-times](<https://devfeed.tech/tags/ee-times.md>), [ee-times-university](<https://devfeed.tech/tags/ee-times-university.md>), [eelife](<https://devfeed.tech/tags/eelife.md>), [eelive-electronic-design](<https://devfeed.tech/tags/eelive-electronic-design.md>), [eet](<https://devfeed.tech/tags/eet.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [electrical-engineering](<https://devfeed.tech/tags/electrical-engineering.md>), [electronic-components](<https://devfeed.tech/tags/electronic-components.md>), [electronic-engineering-times](<https://devfeed.tech/tags/electronic-engineering-times.md>), [electronics](<https://devfeed.tech/tags/electronics.md>), [embedded-systems](<https://devfeed.tech/tags/embedded-systems.md>), [farm-autonomy](<https://devfeed.tech/tags/farm-autonomy.md>), [fundamentals-courses](<https://devfeed.tech/tags/fundamentals-courses.md>), [learning](<https://devfeed.tech/tags/learning.md>), [mixed-signal](<https://devfeed.tech/tags/mixed-signal.md>), [news](<https://devfeed.tech/tags/news.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nxp](<https://devfeed.tech/tags/nxp.md>), [opinion](<https://devfeed.tech/tags/opinion.md>), [part-search](<https://devfeed.tech/tags/part-search.md>), [physical-ai](<https://devfeed.tech/tags/physical-ai.md>), [power](<https://devfeed.tech/tags/power.md>), [product-search](<https://devfeed.tech/tags/product-search.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [safety](<https://devfeed.tech/tags/safety.md>), [semiconductors](<https://devfeed.tech/tags/semiconductors.md>), [sensors](<https://devfeed.tech/tags/sensors.md>), [smart-agriculture](<https://devfeed.tech/tags/smart-agriculture.md>), [software](<https://devfeed.tech/tags/software.md>), [tech-papers](<https://devfeed.tech/tags/tech-papers.md>), [technology](<https://devfeed.tech/tags/technology.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

Agricultural autonomy requires onboard systems to sense, decide, and act within tight power, heat, cost, and safety limits, especially when connectivity is unreliable or response delays matter. Nvidia describes multi workload platforms combining perception, crop analysis, localization, navigation, obstacle avoidance, and control; Jetson targets embedded AI, while local sensors can handle some filtering and detection and cloud services remain useful for training, simulation, fleet management, and analytics. NXP emphasizes matching processors to tasks: application processors prepare sensor data, NPUs run inference, and MCUs handle control, with performance per watt often more important than peak compute. Safety functions must remain isolated so delayed inference cannot compromise stopping or other critical controls. Models may need location specific updates, but proofs of concept do not establish broad product readiness; deployment still requires safety, security, regulatory compliance, reliability, and protection of farmers' data privacy.

### Source excerpt

NXP and Nvidia tackle the challenges of delivering computing power, energy efficiency, and safety as agricultural machinery becomes autonomous. The post Edge AI Moves Agricultural Intelligence from the Cloud to the Field appeared first on EE Times.

## 180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

DevFeed: [180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers](<https://devfeed.tech/articles/180b-parameters-32gb-of-ram-pushing-moe-inference-to-the-limit-with-transformers-143010.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/moe-expert-streaming>)

Author: Tarek Ziadé

Published: 2026-11-18T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [apple](<https://devfeed.tech/tags/apple.md>), [apple-silicon](<https://devfeed.tech/tags/apple-silicon.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [cache](<https://devfeed.tech/tags/cache.md>), [inference](<https://devfeed.tech/tags/inference.md>), [moe](<https://devfeed.tech/tags/moe.md>), [ram](<https://devfeed.tech/tags/ram.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The excerpt reports a Transformers prototype that streams only router-selected MoE experts from SSD through a bounded device cache, remaps expert IDs to shared cache slots, and reuses `grouped_mm` for expert computation. On a 32 GB M5, after quantizing experts to 3 bits, compressing dense layers, and adding a separate disk path for Qwen3.8-Flash-Next's n-gram table, the system reached about 14 tokens/s--roughly 6% behind Strata on short decode, with slightly faster prefill. It uses about 18 GiB after loading; a 6K prompt peaks at 25.4 GiB, and the author reports needing to close the browser to avoid swapping. The remaining gap is attributed mainly to host round trips and GPU idle time between layers; closing it would require deeper MPS synchronization control unavailable in PyTorch. The API is still under discussion, and the approach currently supports models with the standard experts layout.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## CECAR: Cache & Expert Co-Aware Routing Accelerates On-Device Inference of MoE LLMs

DevFeed: [CECAR: Cache & Expert Co-Aware Routing Accelerates On-Device Inference of MoE LLMs](<https://devfeed.tech/articles/cecar-cache-expert-co-aware-routing-accelerates-on-device-inference-of-moe-llms-137539.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20119>)

Author: Byeongju Kim, Hoonki Lee, Byungjun Kim, Inyul Ra, Yongjin Kim, Yongseong Lee, Sangyeob Kim

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [activation](<https://devfeed.tech/tags/activation.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [cache](<https://devfeed.tech/tags/cache.md>), [can](<https://devfeed.tech/tags/can.md>), [code](<https://devfeed.tech/tags/code.md>), [consumer](<https://devfeed.tech/tags/consumer.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decode](<https://devfeed.tech/tags/decode.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [device](<https://devfeed.tech/tags/device.md>), [devices](<https://devfeed.tech/tags/devices.md>), [dram](<https://devfeed.tech/tags/dram.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llms](<https://devfeed.tech/tags/llms.md>), [moe](<https://devfeed.tech/tags/moe.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [routing](<https://devfeed.tech/tags/routing.md>)

### AI overview

CECAR jointly manages expert routing and caching for on-device MoE inference. It predicts each expert's expected reuse distance with an ML-based cache policy, then prioritizes experts with smaller predicted distances; the authors say this avoids routing instability associated with prior methods that use binary cache presence. On Qwen3-30B-A3B, with non-resident experts fetched from SSD, CECAR reportedly reaches 12.5 tokens/s on a consumer GPU with 16 GB of VRAM, a 4.5x speedup over conventional MoE inference and faster decoding than baselines keeping the full model in DRAM. These results are reported for that model and setup.

### Source excerpt

arXiv:2611.20119v1 Announce Type: cross Abstract: Despite growing interest in on-device LLM deployment, large-scale Mixture-of-Experts (MoE) models remain impractical on resource-constrained devices, as sparse expert activation still requires all expert weights to be memory-resident. To address this, we identify three key observations for on-device MoE inference: (a) cache hit rates are fundamentally bounded even under Belady's optimal policy; (b) these bounds are insufficient for low-latency on-device inference; and (c) prior cache-aware routing based on binary cache presence can induce routing instability. Based on these insights, we propose CECAR (Cache & Expert Co-Aware Routing), a MoE inference system that jointly considers expert routing and cache management. Unlike prior cache-aware routing methods, CECAR uses an ML-based cache policy to predict the expected reuse distance of each expert and prioritizes routing and execution toward experts with smaller predicted reuse distances. On the Qwen3-30B-A3B model, CECAR achieves 12.5 tokens/s on a consumer-grade GPU with only 16 GB of VRAM, where non-resident experts are fetched from SSD, delivering a 4.5x speedup over conventional MoE inference and exceeding the decode speed of baselines that keep the entire model fully resident in DRAM. Code is available at https://github.com/hoonki2002/CECAR.

## SpecRelay: Amortize or Avoid the Round Trip in Edge-Cloud Speculative Decoding

DevFeed: [SpecRelay: Amortize or Avoid the Round Trip in Edge-Cloud Speculative Decoding](<https://devfeed.tech/articles/specrelay-amortize-or-avoid-the-round-trip-in-edge-cloud-speculative-decoding-137368.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18963>)

Author: Jai Jain, Chihyo Ahn, Yong Liu, Charles Chen, Duc N. M Hoang, Ajay Kumar Jaiswal, Fatih Ilhan, Han-Byul Kim, Lauren Hannah, Richa Dixit, Minsik Cho, Yingyan Celine Lin, Hyesoon Kim

Published: 2026-11-17T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Decoding](<https://devfeed.tech/topics/decoding.md>)

Tags: [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [completion](<https://devfeed.tech/tags/completion.md>), [concurrently](<https://devfeed.tech/tags/concurrently.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [device](<https://devfeed.tech/tags/device.md>), [draft](<https://devfeed.tech/tags/draft.md>), [edge](<https://devfeed.tech/tags/edge.md>), [edge-cloud](<https://devfeed.tech/tags/edge-cloud.md>), [full](<https://devfeed.tech/tags/full.md>), [generation](<https://devfeed.tech/tags/generation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [local](<https://devfeed.tech/tags/local.md>), [low-latency](<https://devfeed.tech/tags/low-latency.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>)

### AI overview

SpecRelay tackles round-trip latency in edge-cloud speculative decoding in two ways. For cloud-bound queries, its asynchronous pipeline keeps fixed-budget draft trees in flight while prior verifications are pending; the cloud discards stale requests before running the target model. This schedule alone raises throughput by up to 50% over a reproduced synchronous baseline across multiple benchmarks. An RTT-conditioned Latency-Aware Router also chooses among on-device direct generation, Chain-of-Draft, full reasoning, and cloud speculative decoding under a configurable accuracy-latency objective. It runs alongside draft-model prefill, adding no inference latency. At matched target-model accuracy with 250 ms of added round-trip latency, the full system reaches 1.95x baseline throughput on SPROUT and 1.55x on OlympiadBench, sending 47% and 39% of queries to the cloud, respectively.

### Source excerpt

arXiv:2611.18963v1 Announce Type: new Abstract: Edge-cloud speculative decoding moves autoregressive drafting to an edge device while a larger cloud model verifies draft tokens in parallel, so that edge drafting and cloud verification overlap under a synchronous draft-verify schedule. Beyond low-latency local networks, however, the next verification request of that schedule can be sent only after the previous reply arrives, so every round still waits out a full round trip, and throughput falls as the network RTT grows. To recover this loss, we present SpecRelay, an edge-cloud LLM inference system that addresses this cost at two levels: it amortizes round trips when cloud verification is necessary and avoids them when local reasoning suffices. For cloud-bound queries, an asynchronous verification pipeline keeps fixed-budget draft trees in flight while earlier replies are still outstanding, and the cloud rejects stale requests before the target-model forward pass (the asynchronous schedule). At the query level, an RTT-conditioned router (the Latency-Aware Router) estimates quality and completion latency for three on-device modes (direct generation, Chain-of-Draft, and full reasoning) and cloud speculative decoding, selecting among them under a configurable accuracy-latency objective without adding latency during inference, since the router runs concurrently with the draft-model prefill. The asynchronous schedule alone improves decoding throughput by up to 50% over a reproduced synchronous baseline across multiple benchmarks. At matched target-model accuracy and 250 ms of added round-trip latency, the full system with the router delivers 1.95x and 1.55x the throughput of the same baseline on SPROUT and OlympiadBench, respectively, while sending only 47% and 39% of the queries to the cloud.

## AdaModalX: Synchronization-Free MFM Inference for Distributed Multimodal Mobile Sensing via Progressive Data Orchestration

DevFeed: [AdaModalX: Synchronization-Free MFM Inference for Distributed Multimodal Mobile Sensing via Progressive Data Orchestration](<https://devfeed.tech/articles/adamodalx-synchronization-free-mfm-inference-for-distributed-multimodal-mobile-sensing-via-progressive-data-orchestration-139057.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20464>)

Author: Xiangrui Xu, Sicong Liu, Zhiwen Yu, Zimu Zhou, Bin Guo

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [applications](<https://devfeed.tech/tags/applications.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [completeness](<https://devfeed.tech/tags/completeness.md>), [contribution](<https://devfeed.tech/tags/contribution.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-orchestration](<https://devfeed.tech/tags/data-orchestration.md>), [device](<https://devfeed.tech/tags/device.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [edge](<https://devfeed.tech/tags/edge.md>), [execution](<https://devfeed.tech/tags/execution.md>), [exit](<https://devfeed.tech/tags/exit.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [features](<https://devfeed.tech/tags/features.md>), [inference](<https://devfeed.tech/tags/inference.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [sensing](<https://devfeed.tech/tags/sensing.md>), [synchronization](<https://devfeed.tech/tags/synchronization.md>)

### AI overview

AdaModalX aims to reduce latency in distributed mobile sensing without waiting for every modality to arrive. A lightweight device-side probe lets reliable samples exit locally while uncertain ones continue uploading. At the edge, the system forms candidates from received data and, when needed, imputed features for delayed inputs; a contribution assessor using frozen MFM encoders estimates their utility and triggers the MFM when a candidate meets its threshold. Across four mobile sensing applications, the authors report up to 26.97% higher accuracy and up to 64.75% lower end-to-end latency. These are reported maximum gains; the excerpt does not specify baselines or per-application results.

### Source excerpt

arXiv:2611.20464v1 Announce Type: new Abstract: Multimodal foundation models (MFMs) support generalizable inference in distributed mobile sensing applications by integrating heterogeneous data streams. However, existing MFM inference pipelines typically require complete and synchronized multimodal inputs, whereas mobile sensing streams arrive asynchronously due to network variability and heterogeneous data volumes across modalities. Waiting for all modalities increases end-to-end latency, while inference with incomplete inputs can reduce accuracy. We argue that multimodal inference should depend on the utility of available information rather than modality completeness. Based on this principle, we propose AdaModalX, a progressive multimodal inference framework that decouples inference execution from modality synchronization through runtime utility-aware orchestration. At each device, importance-aware data uploading uses a lightweight probe distilled from the edge-side MFM to allow reliable samples to exit locally while uncertain samples continue uploading. At the edge, on-demand MFM activation constructs candidates from fully received samples and, when needed, imputed features derived from partially received delayed inputs. A lightweight contribution assessor built on frozen MFM encoders estimates candidate utility and activates the MFM once the candidate meets its corresponding utility threshold. Experiments on four mobile sensing applications show that AdaModalX improves the accuracy-latency trade-off, achieving up to 26.97% accuracy improvement and up to 64.75% end-to-end latency reduction.

## Could Mobile AI be Exploited?: Understanding the Exploitations of Trust and Ethics Gap in On-Device Small Language Models

DevFeed: [Could Mobile AI be Exploited?: Understanding the Exploitations of Trust and Ethics Gap in On-Device Small Language Models](<https://devfeed.tech/articles/could-mobile-ai-be-exploited-understanding-the-exploitations-of-trust-and-ethics-gap-in-on-device-small-language-models-139217.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2406.05364>)

Author: Kalyan Nakka, Jimmy Dani, Nitesh Saxena

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [trust and safety](<https://devfeed.tech/topics/trust-and-safety.md>), [trust](<https://devfeed.tech/topics/trust.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [allows](<https://devfeed.tech/tags/allows.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-ai](<https://devfeed.tech/tags/artificial-intelligence-ai.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [cloud-based](<https://devfeed.tech/tags/cloud-based.md>), [content](<https://devfeed.tech/tags/content.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [disinformation](<https://devfeed.tech/tags/disinformation.md>), [ethics](<https://devfeed.tech/tags/ethics.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [exploited](<https://devfeed.tech/tags/exploited.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [mobile-ai](<https://devfeed.tech/tags/mobile-ai.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [small-language-models](<https://devfeed.tech/tags/small-language-models.md>), [smartphones](<https://devfeed.tech/tags/smartphones.md>), [trust](<https://devfeed.tech/tags/trust.md>)

### AI overview

The study reports that evaluated on-device small language models (SLMs) were less trustworthy and had weaker ethical safeguards than their on-server counterparts, producing more stereotypical, unfair, privacy-breaching, and harmful content. It says these risks could be exploited through Open Knowledge (OkNow) attacks: the tested models gave valid responses to potentially unethical ordinary prompts without filters, jailbreaking, or prompt engineering. The abstract notes possible abuse involving societal harm, illegal activity, hate, self-harm, phishing, and disinformation. On-device SLMs may offer privacy and lower latency, but the abstract reports trust and safety tradeoffs; it does not provide model-level results or mitigation guidance.

### Source excerpt

arXiv:2406.05364v3 Announce Type: replace Abstract: In this paper, we present a very first study to investigate trust and ethical implications of on-device artificial intelligence (AI), focusing on small language models (SLMs) amenable for personal devices like smartphones. While on-device SLMs promise enhanced privacy, reduced latency, and improved user experience compared to cloud-based services, we posit that they might also introduce significant risks and vulnerabilities compared to their on-server counterparts. As part of our study, we conduct a systematic evaluation of the state-of-the-art on-devices SLMs, compared to their on-server counterparts, using well-established trust and ethics frameworks. Our results illustrate that on-device SLMs are significantly less trustworthy and lacks ethical safeguards, specifically demonstrating more stereotypical, unfair, privacy-breaching and harmful content generation behavior. Further, these risks of on-device SLMs could be exploited through Open Knowledge (OkNow) Attacks, and we demonstrated them using potentially unethical vanilla prompts, to which the on-device SLMs answer with valid responses without any filters and without the need for any jailbreaking or prompt engineering. These responses can be abused for various harmful and unethical scenarios like: societal harm, illegal activities, hate, self-harm, exploitable phishing content, disinformation and many others.

## Superfluid: Local AI Server for Distributed LLM Inference

DevFeed: [Superfluid: Local AI Server for Distributed LLM Inference](<https://devfeed.tech/articles/superfluid-local-ai-server-for-distributed-llm-inference-134888.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17753>)

Author: Prabod Rathnayaka, Isura Manchanayake, Srikandabala Kogul, Steve Siu, Fabian Waschkowski, Lukas Wesemann

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [llama.cpp](<https://devfeed.tech/topics/llama-cpp.md>), [Ollama](<https://devfeed.tech/topics/ollama.md>)

Tags: [1-5](<https://devfeed.tech/tags/1-5.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [all](<https://devfeed.tech/tags/all.md>), [apache](<https://devfeed.tech/tags/apache.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [api](<https://devfeed.tech/tags/api.md>), [apple](<https://devfeed.tech/tags/apple.md>), [apple-m5](<https://devfeed.tech/tags/apple-m5.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [can](<https://devfeed.tech/tags/can.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [local](<https://devfeed.tech/tags/local.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [server](<https://devfeed.tech/tags/server.md>)

### AI overview

Superfluid coordinates local LLM inference on one device and across a fleet. On a single device, it runs llama.cpp, MLX, or BaseRT in isolated workers; one scheduler prioritizes interactive requests over agent work, reuses shared prompt computation, and durably logs tokens before delivery. Across machines, it exposes one API endpoint, routes work by measured speed, and supports machines joining or failing under load while preserving requests. In the reported tests, an interactive Qwen3.8-27B chat on an Apple M5 Pro produced its first token in 1.5 seconds while eight agents worked, compared with 291-612 seconds for llama-server, Ollama, and mlx_lm.server. A six-machine, three-vendor test answered all 5,041 requests as machines joined and one was killed and rejoined; throughput rose from 112 to 534 tokens/s and median first-token time fell from 99 to 12 seconds. Twelve Mac Studio M3 Ultras sustained 10,656 tokens/s across 38,563 requests from 384 agents without failure.

### Source excerpt

arXiv:2611.17753v1 Announce Type: new Abstract: Running large language models (LLMs) on local devices keeps data private, removes per-token cost, cuts network latency, and keeps working without a connection. Two shifts now strain how such models are served. On one device, several agents share a locally served model with an interactive user. Across devices, a developer or small team often owns several capable machines from different vendors, which sit idle most of the day because every local server serves one runtime on one machine. We present Superfluid, an open-source LLM server built for local inference at both scales. On one device, Superfluid drives llama.cpp, MLX, or BaseRT in isolated worker processes through one scheduler that moves interactive requests ahead of agent work, computes a prompt shared by many requests once, and writes every token to a durable log before it reaches the client. Across devices, one Superfluid server turns a fleet of machines into a single API endpoint. Machines with different hardware serve copies of the same model, receive work according to their measured speed, and can join or fail under load without losing a request. For Qwen3.8-27B on an Apple M5 Pro, a chat sent while eight agents are working receives its first token in 1.5 s, against 291-612 s on llama-server, Ollama, and mlx_lm.server. On six machines from three chip vendors serving 64 simulated users for one hour, Superfluid answers all 5,041 requests while machines join one at a time and one is killed and rejoins; throughput grows from 112 to 534 tok/s and the median first-token time falls from 99 to 12 s. On twelve Mac Studio M3 Ultras it sustains 10,656 tok/s across 38,563 requests from 384 agents without a failure. Superfluid is available under the Apache-2.0 licence at https://github.com/basecompute/superfluid.

## RL$^{\\mathrm{RT}}$: On-Device Reinforcement Learning Runtime

DevFeed: [RL$^{\\mathrm{RT}}$: On-Device Reinforcement Learning Runtime](<https://devfeed.tech/articles/rl-mathrm-rt-on-device-reinforcement-learning-runtime-135161.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18035>)

Author: Zexin Li, Xinwei Gao, Arvind Easwaran

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [tinyml](<https://devfeed.tech/topics/tinyml.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [actor](<https://devfeed.tech/tags/actor.md>), [admission](<https://devfeed.tech/tags/admission.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [learner](<https://devfeed.tech/tags/learner.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [refresh](<https://devfeed.tech/tags/refresh.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [scheduler](<https://devfeed.tech/tags/scheduler.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

RL$^{\mathrm{RT}}$ coordinates a periodic actor and learner sharing an embedded GPU by making their coupling tunable. A refresh bound limits actor releases between weight updates, while a GPU permit with slack-aware admission prioritizes actor work and uses spare time for training on top of a base scheduler. On two Jetson boards, accounting for the real refresh cost, it kept deadline misses for a 16.8-million-parameter policy below 1%, compared with 85-100% without the runtime.

### Source excerpt

arXiv:2611.18035v1 Announce Type: cross Abstract: On-device reinforcement learning runs a periodic actor and a learner on one embedded GPU shared with other workloads. Because the learner produces the actor's weights, existing designs stall the actor or let its policy drift. We present RL$^{\mathrm{RT}}$, an on-device runtime that makes this coupling a tunable parameter: a refresh bound limits actor releases between refreshes, and a GPU permit with slack-aware admission prioritises the actor and fills spare time with training atop any base scheduler. Under the real refresh cost on two Jetson boards, RL$^{\mathrm{RT}}$ keeps a 16.8 M-parameter policy's deadline misses below 1%, against 85 to 100% without it.

## Run Native DLA Inference in Python on NVIDIA Jetson AGX Orin Using cuDLA

DevFeed: [Run Native DLA Inference in Python on NVIDIA Jetson AGX Orin Using cuDLA](<https://devfeed.tech/articles/run-native-dla-inference-in-python-on-nvidia-jetson-agx-orin-using-cudla-133567.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/run-native-dla-inference-in-python-on-nvidia-jetson-agx-orin-using-cudla/>)

Author: Nikshay Shrivastava

Published: 2026-11-13T18:57:10Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>), [Jetson](<https://devfeed.tech/topics/jetson.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [applications](<https://devfeed.tech/tags/applications.md>), [compute](<https://devfeed.tech/tags/compute.md>), [computer-vision-video-analytics](<https://devfeed.tech/tags/computer-vision-video-analytics.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [edge](<https://devfeed.tech/tags/edge.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-jetson](<https://devfeed.tech/tags/nvidia-jetson.md>), [nvidia-jetson-agx-orin](<https://devfeed.tech/tags/nvidia-jetson-agx-orin.md>), [python](<https://devfeed.tech/tags/python.md>), [robotics](<https://devfeed.tech/tags/robotics.md>)

### AI overview

The excerpt describes using `cuda.bindings.cudla` 13.3.0+ to run a TensorRT-compiled DLA loadable from Python on Jetson Orin. JetPack provides `libcudla.so`; inference needs that runtime, while TensorRT is needed only to compile the model. Python 3.10 or later is required. The demonstrated workflow uses hybrid `CUDA_DLA` mode and one CUDA stream to coordinate GPU work with DLA inference: load the model bytes, query tensor descriptors, allocate and register GPU buffers, enqueue the task, synchronize, and copy results back for Python postprocessing. Descriptor fields provide sizes and layouts, but preprocessing and output interpretation remain model-specific. Not every layer is DLA-compatible, so the ONNX graph may need trimming. Standalone mode is unsupported in this release. The excerpt says the cuDLA API is identical on RTX Spark, though its detailed setup is centered on Jetson.

### Source excerpt

AI applications at the edge often need to run multiple neural networks within limited compute and power budgets. NVIDIA Deep Learning Accelerator (DLA)...

## How to Use Apple's fm CLI to Run On-Device LLMs From Python

DevFeed: [How to Use Apple's fm CLI to Run On-Device LLMs From Python](<https://devfeed.tech/articles/how-to-use-apple-s-fm-cli-to-run-on-device-llms-from-python-129387.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/how-to-use-apples-fm-cli-to-run-on-device-llms-from-python/>)

Author: Irfan Alghani Khalid

Published: 2026-11-11T13:30:00Z

Content type: tutorial

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Python](<https://devfeed.tech/topics/python.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [ai-engineer](<https://devfeed.tech/tags/ai-engineer.md>), [apple](<https://devfeed.tech/tags/apple.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cli](<https://devfeed.tech/tags/cli.md>), [device](<https://devfeed.tech/tags/device.md>), [free](<https://devfeed.tech/tags/free.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [llms](<https://devfeed.tech/tags/llms.md>), [local-llm](<https://devfeed.tech/tags/local-llm.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [private](<https://devfeed.tech/tags/private.md>), [prototyping](<https://devfeed.tech/tags/prototyping.md>), [python](<https://devfeed.tech/tags/python.md>), [software-engineer](<https://devfeed.tech/tags/software-engineer.md>), [validation](<https://devfeed.tech/tags/validation.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

Apple's `fm` CLI runs compact Foundation Models locally on Apple Silicon, offering private, no-token-cost inference without network round-trips. The article tests text and image use: the model answered "Ipoh" to a Malaysia capital question and labeled grilled satay as fried, showing why outputs need grounding and validation, especially when they inform decisions. For local prototyping, `fm serve` exposes an OpenAI-compatible endpoint at `http://127.0.0.1:1976/v1`; applications can also invoke `fm respond` through Python's `subprocess`. The server path suits code already using the OpenAI SDK, while subprocess fits a single-machine prototype. Prompts need re-testing when switching to a hosted model, and the workflow assumes Apple Silicon. Native Apple apps intended for users should use the Swift Foundation Models framework. The article recommends hosted models for shared cross-platform backends, long documents, or strong factual accuracy.

### Source excerpt

A Sate Padang vision test and a Malaysia capital question show why the free, private model needs grounding and validation before it leaves prototyping. The post How to Use Apple's fm CLI to Run On-Device LLMs From Python appeared first on Towards Data Science.

## EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities

DevFeed: [EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities](<https://devfeed.tech/articles/engineagent-agentizing-on-device-llm-inference-engines-with-inference-stack-hierarchy-awareness-and-hardware-abilities-128424.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14356>)

Author: Zhengxiang Huang, Yujie Sun, Shengheng Chen, Zhaode Wang, Chaoyue Niu, Zeyu Zhao, Chengfei Lv, Fan Wu, Guihai Chen

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [api](<https://devfeed.tech/tags/api.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [build](<https://devfeed.tech/tags/build.md>), [builder](<https://devfeed.tech/tags/builder.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [completeness](<https://devfeed.tech/tags/completeness.md>), [conversion](<https://devfeed.tech/tags/conversion.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-os](<https://devfeed.tech/tags/cs-os.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [device](<https://devfeed.tech/tags/device.md>), [engines](<https://devfeed.tech/tags/engines.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hierarchy](<https://devfeed.tech/tags/hierarchy.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [on-device](<https://devfeed.tech/tags/on-device.md>)

### AI overview

EngineAgent automates enabling new LLMs in on-device inference engines by separating reusable engine and hardware knowledge from model-specific work. It builds an index of engine layers and their dependencies, validates backend workflows with supported models, then uses a builder to coordinate changes and an auditor to check completeness and debug through device-side validation. In 12 tasks across three mobile LLMs and four engine-backend pairs on CPUs, GPUs, and NPUs, it achieved 75% end-to-end success with either of two underlying LLMs, compared with 33.3% for the strongest coding-agent baseline. Success required compilation, on-device execution, and output-logit agreement with reference implementations. The abstract also reports lower average task time and model-API cost than baselines using the same underlying LLMs.

### Source excerpt

arXiv:2611.14356v1 Announce Type: cross Abstract: Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths. We present EngineAgent, an agentic system for automating on-device LLM model enablement. Its key insight is to separate reusable engine and hardware knowledge from model-specific implementation. Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models. For each new model, a builder uses this knowledge to coordinate cross-layer changes, while an auditor checks implementation completeness and drives iterative debugging through device-side validation. Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.

## Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference

DevFeed: [Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference](<https://devfeed.tech/articles/federated-learning-of-binary-neural-networks-enabling-low-cost-inference-128513.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.15507>)

Author: Nitin Priyadarshini Shankar, Soham Lahiri, Sheetal Kalyani, Saurav Prakash

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [architectures](<https://devfeed.tech/tags/architectures.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [edge-devices](<https://devfeed.tech/tags/edge-devices.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [federated](<https://devfeed.tech/tags/federated.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [inference](<https://devfeed.tech/tags/inference.md>), [memory](<https://devfeed.tech/tags/memory.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [performance](<https://devfeed.tech/tags/performance.md>), [precision](<https://devfeed.tech/tags/precision.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [rotation](<https://devfeed.tech/tags/rotation.md>), [times](<https://devfeed.tech/tags/times.md>), [using](<https://devfeed.tech/tags/using.md>)

### AI overview

FedBNN adapts rotation-aware binarization to federated learning by using global model information to guide clients' local binarization and local rotation information to help the server construct the global binary model. The approach addresses mismatches between local and global models under heterogeneous client data. Across benchmark datasets, architectures, and heterogeneity levels, the authors report accuracy competitive with full-precision federated models and generally better than existing federated binary learning methods. Compared with full-precision models, it reports savings of up to 58x in FLOPs and 32x in memory; the abstract does not provide further experimental details.

### Source excerpt

arXiv:2603.15507v2 Announce Type: replace Abstract: Federated Learning (FL) enables distributed clients to jointly train a global model without sharing their raw data. However, deploying the resulting deep neural networks on resource-constrained edge devices remains challenging due to their inference-time computational and memory requirements. Binary Neural Networks (BNNs) offer substantial efficiency gains by representing weights and activations using a single bit, but achieving competitive accuracy under such aggressive quantization remains difficult. Rotation-aware binarization reduces quantization errors by aligning real-valued weights with their binary representations. Despite its effectiveness in centralized learning, applying rotation-aware binarization in FL is non-trivial due to discrepancies between local and global models, particularly under heterogeneous client data distributions. To address this challenge, we propose FedBNN, a federated rotation-aware binarization framework that leverages global model information to guide local binarization while incorporating locally learned rotation information into server-side construction of the global binary model. Experiments across multiple benchmark datasets, model architectures, and degrees of data heterogeneity demonstrate that the proposed integrated methodology achieves accuracy competitive with full-precision federated models while generally outperforming existing federated binary learning approaches. Overall, FedBNN enables accurate binary inference in heterogeneous federated settings, with savings of up to $58\times$ in FLOPs and $32\times$ in memory over full-precision models, offering a practical balance between predictive performance and deployment efficiency on resource-constrained edge devices.

## Built-in AI APIs

DevFeed: [Built-in AI APIs](<https://devfeed.tech/articles/built-in-ai-apis-129395.md>)

Original publisher: [Read original article](<https://specification.website/spec/foundations/built-in-ai/>)

Author: Joost de Valk

Published: 2026-11-11T00:00:00Z

Content type: tutorial

Language: en

Sources: [The Website Specification](<https://devfeed.tech/sources/the-website-specification.md>)

Topics: [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Web platform](<https://devfeed.tech/topics/web-platform.md>)

Tags: [ai-apis](<https://devfeed.tech/tags/ai-apis.md>), [availability](<https://devfeed.tech/tags/availability.md>), [browser](<https://devfeed.tech/tags/browser.md>), [downloads](<https://devfeed.tech/tags/downloads.md>), [enhancement](<https://devfeed.tech/tags/enhancement.md>), [fallback](<https://devfeed.tech/tags/fallback.md>), [foundations](<https://devfeed.tech/tags/foundations.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [gesture](<https://devfeed.tech/tags/gesture.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [on-device-ai](<https://devfeed.tech/tags/on-device-ai.md>), [optional](<https://devfeed.tech/tags/optional.md>), [server](<https://devfeed.tech/tags/server.md>)

### AI overview

Browsers' on-device AI APIs can add optional language features, but availability, hardware needs, and output quality vary, so the site must work without them. The excerpt identifies Prompt (`LanguageModel`), Summarizer, Translator, and LanguageDetector; availability differs by browser, capability, and--in translation--language pair. Check availability with the same options used for creation, start downloadable models only after an explicit user action, and show progress. Gemini Nano-backed Prompt and Summarizer have substantial storage and RAM requirements; Translator and LanguageDetector use smaller models. Use dedicated task APIs where they fit, retain a server, simpler, or manual fallback, and treat generated output as untrusted text; use schema constraints for structured Prompt API output. The excerpt describes these APIs as draft features available only in desktop Chrome and Edge, with version support varying, and notes nondeterministic results and uneven quality.

### Source excerpt

Use the browser's on-device AI APIs -- Prompt, Summarizer, Translator, Language Detector -- as a progressive enhancement: check availability first, gate model downloads on a user gesture, and always keep a server or manual path.

## Atmosis provides a digestible report, rather than raw environmental data

DevFeed: [Atmosis provides a digestible report, rather than raw environmental data](<https://devfeed.tech/articles/atmosis-provides-a-digestible-report-rather-than-raw-environmental-data-127561.md>)

Original publisher: [Read original article](<https://blog.arduino.cc/2026/11/10/atmosis-provides-a-digestible-report-rather-than-raw-environmental-data/>)

Author: Arduino Team

Published: 2026-11-10T12:18:38Z

Content type: article

Language: en

Sources: [Arduino Blog](<https://devfeed.tech/sources/arduino-blog.md>)

Topics: [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [3d-printed-enclosure](<https://devfeed.tech/tags/3d-printed-enclosure.md>), [adafruit](<https://devfeed.tech/tags/adafruit.md>), [ai](<https://devfeed.tech/tags/ai.md>), [arduino](<https://devfeed.tech/tags/arduino.md>), [arduino-uno-q](<https://devfeed.tech/tags/arduino-uno-q.md>), [arduino-uno-q-dream-lab-challenge](<https://devfeed.tech/tags/arduino-uno-q-dream-lab-challenge.md>), [bot](<https://devfeed.tech/tags/bot.md>), [edge-ai](<https://devfeed.tech/tags/edge-ai.md>), [edge-impulse](<https://devfeed.tech/tags/edge-impulse.md>), [environmental-data](<https://devfeed.tech/tags/environmental-data.md>), [environmental-monitoring](<https://devfeed.tech/tags/environmental-monitoring.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [gemini-llm](<https://devfeed.tech/tags/gemini-llm.md>), [interface](<https://devfeed.tech/tags/interface.md>), [uno-q](<https://devfeed.tech/tags/uno-q.md>)

### AI overview

Atmosis combines an Arduino UNO Q with pressure and altitude, gas, temperature, humidity, and dust sensors, then uses onboard edge AI deployed through Edge Impulse to flag readings that may need attention. For users who want more detail, Google's Gemini LLM can examine readings, consult references, and answer questions. Results are available through a web interface, with optional Telegram bot updates. The project is intended to make localized conditions, such as a microclimate or worksite dust, easier to understand; the excerpt notes that local weather stations may offer more comprehensive data.

### Source excerpt

As part of the Arduino UNO Q Dream Lab Challenge hosted in partnership with DigiKey, makers and innovators were challenged to push the boundaries of what is possible with the dual-brain Arduino® UNO™ Q board and Arduino App Lab. From an incredible pool of creative submissions showcasing the board's unique dual-brain architecture, this entry emerged [...] The post Atmosis provides a digestible report, rather than raw environmental data appeared first on Arduino Blog.

## Sensitivity-Gated Accumulator Caching for On-Device Learning of Low-Bit Hand Networks

DevFeed: [Sensitivity-Gated Accumulator Caching for On-Device Learning of Low-Bit Hand Networks](<https://devfeed.tech/articles/sensitivity-gated-accumulator-caching-for-on-device-learning-of-low-bit-hand-networks-125294.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10991>)

Author: Soroor Ghandali, Kanika Mamgain, Manjot Kaur, Akanksha Shukla, Shashira Guntuka

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [tinyml](<https://devfeed.tech/topics/tinyml.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [learning](<https://devfeed.tech/tags/learning.md>), [networks](<https://devfeed.tech/tags/networks.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [on-device-learning](<https://devfeed.tech/tags/on-device-learning.md>), [quantization](<https://devfeed.tech/tags/quantization.md>)

### AI overview

Sensitivity-gated accumulator caching (SGAC) adapts low-bit network weights while keeping residual state only for selected groups. It uses a cache of 8-bit accumulators to collect sub-step updates for groups ranked by Fisher-weighted squared code step, then writes whole steps into the codes; evicted accumulators are flushed by stochastic rounding, while groups without a cache line do not update. The authors report unbiased write-back and, under a local quadratic model, explain that rounding is worthwhile where expected gradients are large relative to curvature. On a 383k-parameter hand-keypoint network, SGAC uses 8.1 state bits per weight and achieves the highest mean accuracy among nine deployed low-bit networks on two simulated cameras. Across twelve ImageNet-C corruptions, it leads seven deployed methods in average clean and corrupted accuracy, with confidence intervals excluding zero, and matches a per-weight 8-bit accumulator using 56% as much state.

### Source excerpt

arXiv:2611.10991v1 Announce Type: cross Abstract: Small hand-tracking networks are deployed as 4- and 8-bit codes, and they should keep adapting to a user's camera after deployment, on the device and within roughly the memory of the model itself. Quantization-aware training cannot do this cheaply, because it keeps a 32-bit master copy of every weight together with its optimizer state, about 96 bits per weight. Training the codes directly with stochastic rounding needs no copy but injects rounding noise at every step, and the methods that remove this noise keep a residual buffer for every weight. We introduce sensitivity-gated accumulator caching (SGAC), which keeps residual state for only a small and changing set of weight groups. A set-associative cache of 8-bit accumulators collects the sub-step updates of the groups with the largest Fisher-weighted squared code step and writes them into the codes as whole steps; an evicted line is flushed by stochastic rounding, and a group without a line does not move until it is admitted. We prove that the write-back is unbiased and show, under a local quadratic model, that rounding a sub-step update costs loss of the same order as the update's own gain, so that rounding pays only where the expected gradient is large relative to the curvature. On a 383k-parameter hand-keypoint network, SGAC trains with 8.1 bits of state per weight. Scored on the 4/8-bit network that the device runs after adaptation, it has the highest mean accuracy of nine deployed low-bit networks on two simulated cameras. On twelve ImageNet-C corruptions it has the highest average of corrupted and clean accuracy of seven deployed methods, ahead of the PEQA-style, ECO-style and direct low-bit baselines with confidence intervals that exclude zero, and it matches an 8-bit accumulator kept for every weight with 56% of that state.

## Where Should Models Run? A Collaborative Inference Framework for Embodied Intelligence in 6G

DevFeed: [Where Should Models Run? A Collaborative Inference Framework for Embodied Intelligence in 6G](<https://devfeed.tech/articles/where-should-models-run-a-collaborative-inference-framework-for-embodied-intelligence-in-6g-125307.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11023>)

Author: Wanlu Sun, Zhenhui Yuan, Xiang Ji, Yi Han, Qing Li, Gan Zheng

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>)

Tags: [5g](<https://devfeed.tech/tags/5g.md>), [6g](<https://devfeed.tech/tags/6g.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [devices](<https://devfeed.tech/tags/devices.md>), [edge](<https://devfeed.tech/tags/edge.md>), [energy](<https://devfeed.tech/tags/energy.md>), [execution](<https://devfeed.tech/tags/execution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [inference](<https://devfeed.tech/tags/inference.md>), [intelligence](<https://devfeed.tech/tags/intelligence.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llms](<https://devfeed.tech/tags/llms.md>), [models](<https://devfeed.tech/tags/models.md>), [network](<https://devfeed.tech/tags/network.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>)

### AI overview

The optimal place to split embodied AI model inference between a robot and edge server changes with wireless conditions and model structure. The article proposes an edge-centric framework in which the edge coordinates partitioning and execution, addressing robots' limited compute and energy budgets while recognizing that network conditions and privacy requirements affect placement. Simulations and real-world experiments over WiFi and 5G identify latency-minimizing partition points for ResNet-101, Qwen2.5-VL-7B, and OpenVLA-7B across NVIDIA Jetson Nano or AGX Xavier devices and an AGX Thor edge server. The supplied abstract does not report specific partition points or comparative latency values.

### Source excerpt

arXiv:2611.11023v1 Announce Type: new Abstract: Embodied intelligence systems rely on fast and reliable inference to enable real-time perception, reasoning, and control. Recent advances in large-scale artificial intelligence (AI) models, including Large Language Models (LLMs) and Vision-Language-Action (VLA) models, have significantly enhanced the capabilities of embodied agents in complex environments. However, deploying these models remains challenging due to the limited computational resources and energy budgets of robotic platforms. A fundamental question is where inference should be executed. Collaborative inference, which partitions models across end devices, edge servers, and the cloud, has emerged as a promising solution. Yet determining the optimal partitioning point remains an open problem, as it depends on wireless network conditions, model structure, and privacy requirements. This article reviews the state of the art in collaborative inference for embodied intelligence and proposes an edge-centric framework in which the edge coordinates model partitioning and execution. Through both simulation and real-world experiments over WiFi and 5G networks, we demonstrate that the optimal partitioning strategy varies with network conditions and model structure. We further identify latency-minimizing partition points for ResNet-101, Qwen2.5-VL-7B, and OpenVLA-7B in collaborative inference between NVIDIA Jetson Nano or AGX Xavier devices and an AGX Thor edge server under varying wireless conditions. Finally, we discuss the future directions for collaborative inference in 6G-empowered embodied intelligence systems.

## Ubuntu Summit 26.10: Less than a week away!

DevFeed: [Ubuntu Summit 26.10: Less than a week away!](<https://devfeed.tech/articles/ubuntu-summit-26-10-less-than-a-week-away-121772.md>)

Original publisher: [Read original article](<https://ubuntu.com//blog/ubuntu-summit-26-10-less-than-a-week-away>)

Author: Graham Morrison

Published: 2026-11-06T16:43:41Z

Content type: news

Language: en

Sources: [Ubuntu blog](<https://devfeed.tech/sources/ubuntu-blog.md>)

Topics: [Ubuntu](<https://devfeed.tech/topics/ubuntu.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>)

Tags: [accessibility](<https://devfeed.tech/tags/accessibility.md>), [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding](<https://devfeed.tech/tags/ai-coding.md>), [ai-coding-agent](<https://devfeed.tech/tags/ai-coding-agent.md>), [astra](<https://devfeed.tech/tags/astra.md>), [broadcasting](<https://devfeed.tech/tags/broadcasting.md>), [cad](<https://devfeed.tech/tags/cad.md>), [canonical](<https://devfeed.tech/tags/canonical.md>), [community](<https://devfeed.tech/tags/community.md>), [control](<https://devfeed.tech/tags/control.md>), [conversations](<https://devfeed.tech/tags/conversations.md>), [design](<https://devfeed.tech/tags/design.md>), [designing](<https://devfeed.tech/tags/designing.md>), [desktop](<https://devfeed.tech/tags/desktop.md>), [developer](<https://devfeed.tech/tags/developer.md>), [developer-workflows](<https://devfeed.tech/tags/developer-workflows.md>), [summit](<https://devfeed.tech/tags/summit.md>), [ubuntu](<https://devfeed.tech/tags/ubuntu.md>), [ubuntu-summit](<https://devfeed.tech/tags/ubuntu-summit.md>)

### AI overview

Ubuntu Summit 26.10 takes place November 12-13, with a live, interactive stream from London and local watch parties. Its twice-yearly schedule is intended to bring current work and ideas to the community; attendees can ask speakers questions and join interactive activities remotely. The first morning's sessions cover open-source observatory software for exoplanet hunting, ntfy's delivery of four million push notifications per day, an AI coding agent built around Unix workflows, locally processed AI for Ubuntu accessibility, and using LLMs and open-source tools to design a robot. Registration is free.

### Source excerpt

Ubuntu Summit 26.10: Less than a week away! Here in London, the long summer has stepped aside for cooler nights and damp, misty mornings. The trees are turning amber and the sunlight is low on the horizon, which can only mean one thing: the next Ubuntu Summit is just around the corner. In fact, it's [...]

## Ubuntu Summit 26.10: Less than a week away!

DevFeed: [Ubuntu Summit 26.10: Less than a week away!](<https://devfeed.tech/articles/ubuntu-summit-26-10-less-than-a-week-away-121738.md>)

Original publisher: [Read original article](<https://canonical.com//blog/ubuntu-summit-26-10-less-than-a-week-away>)

Author: Graham Morrison

Published: 2026-11-06T16:43:41Z

Content type: news

Language: en

Sources: [Canonical](<https://devfeed.tech/sources/canonical.md>)

Topics: [Ubuntu](<https://devfeed.tech/topics/ubuntu.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>)

Tags: [accessibility](<https://devfeed.tech/tags/accessibility.md>), [ai](<https://devfeed.tech/tags/ai.md>), [astra](<https://devfeed.tech/tags/astra.md>), [backend](<https://devfeed.tech/tags/backend.md>), [cad](<https://devfeed.tech/tags/cad.md>), [coding-agent](<https://devfeed.tech/tags/coding-agent.md>), [delivery](<https://devfeed.tech/tags/delivery.md>), [design](<https://devfeed.tech/tags/design.md>), [development](<https://devfeed.tech/tags/development.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [event](<https://devfeed.tech/tags/event.md>), [exoplanet](<https://devfeed.tech/tags/exoplanet.md>), [hax](<https://devfeed.tech/tags/hax.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [one](<https://devfeed.tech/tags/one.md>), [summit](<https://devfeed.tech/tags/summit.md>), [ubuntu](<https://devfeed.tech/tags/ubuntu.md>), [ubuntu-summit](<https://devfeed.tech/tags/ubuntu-summit.md>)

### AI overview

Ubuntu Summit 26.10 is scheduled for November 12-13, with a live, interactive stream from London and local watch parties. The twice-yearly schedule is presented as a way to surface current work and ideas; attendees can ask speakers questions and join activities remotely. The first morning's listed sessions cover open-source observatory software for exoplanet detection, scaling ntfy to four million daily push notifications, integrating AI coding agents into Unix workflows, locally processed AI for Ubuntu accessibility, and using LLMs and open-source tools to design a robot.

### Source excerpt

Ubuntu Summit 26.10: Less than a week away! Here in London, the long summer has stepped aside for cooler nights and damp, misty mornings. The trees are turning amber and the sunlight is low on the horizon, which can only mean one thing: the next Ubuntu Summit is just around the corner. In fact, it's [...]

[Next page](<https://devfeed.tech/topics/on-device-ai.md?cursor=WyIyMDI2LTExLTA2VDE2OjQzOjQxKzAwOjAwIiwgIjFiM2IzZDZjLWMwODYtNGMxNy04OWY0LTAwYmQ1MmI1NzkzOSJd>)