# AI Architecture

AI architecture is the design of layered, modular AI systems, including orchestration, model inference, data retrieval, monitoring, security, and storage.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Inference pushes AI to smaller sites, making the data layer the deciding factor

DevFeed: [Inference pushes AI to smaller sites, making the data layer the deciding factor](<https://devfeed.tech/articles/inference-pushes-ai-to-smaller-sites-making-the-data-layer-the-deciding-factor-146669.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/20/inference-monolithic-data-center-ai-data-layer-sc26/>)

Author: Sloane Kali Faye

Published: 2026-11-20T20:09:20Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [data centers](<https://devfeed.tech/topics/data-centers.md>), [AI Factory](<https://devfeed.tech/topics/ai-factory.md>)

Tags: [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-data-centers](<https://devfeed.tech/tags/ai-data-centers.md>), [ai-data-layer](<https://devfeed.tech/tags/ai-data-layer.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [alex-bouzari](<https://devfeed.tech/tags/alex-bouzari.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [data-layer](<https://devfeed.tech/tags/data-layer.md>), [ddn](<https://devfeed.tech/tags/ddn.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-inference](<https://devfeed.tech/tags/distributed-inference.md>), [edge-data-centers](<https://devfeed.tech/tags/edge-data-centers.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [enterprise-ai-hyperpod](<https://devfeed.tech/tags/enterprise-ai-hyperpod.md>), [exascaler](<https://devfeed.tech/tags/exascaler.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [heterogeneous-computing](<https://devfeed.tech/tags/heterogeneous-computing.md>), [high-performance-computing](<https://devfeed.tech/tags/high-performance-computing.md>), [high-performance-storage](<https://devfeed.tech/tags/high-performance-storage.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [hpc-infrastructure](<https://devfeed.tech/tags/hpc-infrastructure.md>), [inference](<https://devfeed.tech/tags/inference.md>), [john-furrier](<https://devfeed.tech/tags/john-furrier.md>), [news](<https://devfeed.tech/tags/news.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [parallel-file-systems](<https://devfeed.tech/tags/parallel-file-systems.md>), [rebecca-knight](<https://devfeed.tech/tags/rebecca-knight.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [sc26eventpage](<https://devfeed.tech/tags/sc26eventpage.md>), [scientific-computing](<https://devfeed.tech/tags/scientific-computing.md>), [scientific-simulation](<https://devfeed.tech/tags/scientific-simulation.md>), [supercomputing](<https://devfeed.tech/tags/supercomputing.md>), [thecube](<https://devfeed.tech/tags/thecube.md>)

### AI overview

The excerpt argues that inference is shifting AI infrastructure toward smaller sites near users and data, increasing the importance of a flexible data layer as GPU, memory and SSD costs and shortages constrain budgets. DataDirect Networks CEO Alex Bouzari says software that spans different hardware can help maximize utilization across large AI facilities and smaller 1-5 MW sites. DDN's EXAScaler move to hardware independence reflects that approach; its Enterprise AI HyperPOD combines CPUs, GPUs, networking and storage in a rack for deployment at enterprise sites with available power. The excerpt presents on-premises pods as a way to keep tighter control of company data, and frames data infrastructure as increasingly important alongside accelerated compute. These are Bouzari's views and DDN examples, not an independent comparison of providers.

### Source excerpt

Inference is moving to the center of enterprise AI spending, and the AI data layer is emerging as the piece that decides whether those investments pay off. As GPU costs climb and component shortages persist, the architecture behind AI is shifting from a handful of massive facilities toward a distributed web of smaller sites. That [...] The post Inference pushes AI to smaller sites, making the data layer the deciding factor appeared first on SiliconANGLE.

## GPU-Accelerated Secure Inference for Drug--Target Interaction Prediction

DevFeed: [GPU-Accelerated Secure Inference for Drug--Target Interaction Prediction](<https://devfeed.tech/articles/gpu-accelerated-secure-inference-for-drug-target-interaction-prediction-145088.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25664>)

Author: Sikha Pentyala, Steven Golob, Daniil Filienko, Ming-Chun Kao, Martine De Cock

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Cryptography](<https://devfeed.tech/topics/cryptography.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [accelerated](<https://devfeed.tech/tags/accelerated.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [co-design](<https://devfeed.tech/tags/co-design.md>), [communication](<https://devfeed.tech/tags/communication.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [competition](<https://devfeed.tech/tags/competition.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [custom](<https://devfeed.tech/tags/custom.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [deep-learning-models](<https://devfeed.tech/tags/deep-learning-models.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-accelerated](<https://devfeed.tech/tags/gpu-accelerated.md>), [inference](<https://devfeed.tech/tags/inference.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [target](<https://devfeed.tech/tags/target.md>)

### AI overview

The paper's first-place iDASH 2026 Track III solution combines an MPC-friendly DTI model with GPU protocols for two-party secure inference. SlimDTA reduces operations on the secret drug, especially nonlinear operations that require communication between servers. The GPU framework evaluates linear layers using additive secret sharing and nonlinear layers using function secret sharing, with a custom CUDA comparison kernel. The authors report under 10 ms per prediction on average, with secure predictions matching the plaintext model to within 0.02%.

### Source excerpt

arXiv:2611.25664v1 Announce Type: cross Abstract: Deep learning models that predict drug--target interactions (DTI) can speed up drug discovery, but the candidate drugs they score are valuable intellectual property that their owners cannot reveal. Secure multiparty computation (MPC) lets two servers compute a prediction on a secret-shared drug, without either server learning the drug. However, secure inference of deep models is slow. Non-linear operations require communication between the servers, and MPC arithmetic does not map naturally to GPUs. In this paper, we present our solution to Track~III of the iDASH 2026 privacy and security competition, which asked for fast two-party secure DTI inference on GPUs. Our solution is built on the co-design of MPC-friendly models and MPC protocols. On the model side, we design SlimDTA, a DTI model that reduces the operations on the secret drug, and in particular the non-linear operations that require communication. On the protocol side, we build a GPU framework that evaluates the linear layers with additive secret sharing and the non-linear layers with function secret sharing, including a custom CUDA kernel for the comparisons. Our solution won first place in the competition, at under 10,ms per prediction on average, and its secure predictions match the plaintext model to within 0.02%.

## Four AI Architecture Patterns for Regulated Industries

DevFeed: [Four AI Architecture Patterns for Regulated Industries](<https://devfeed.tech/articles/four-ai-architecture-patterns-for-regulated-industries-144360.md>)

Original publisher: [Read original article](<https://www.databricks.com/blog/ai-architecture-patterns-regulated-industries>)

Author: Molly Nagamuthu

Published: 2026-11-19T19:30:00Z

Content type: article

Language: en

Sources: [Databricks](<https://devfeed.tech/sources/databricks.md>)

Topics: [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-architecture](<https://devfeed.tech/tags/agent-architecture.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-architecture](<https://devfeed.tech/tags/ai-architecture.md>), [ai-native](<https://devfeed.tech/tags/ai-native.md>), [ai-transformation](<https://devfeed.tech/tags/ai-transformation.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audit-trail](<https://devfeed.tech/tags/audit-trail.md>), [auditability](<https://devfeed.tech/tags/auditability.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [automated](<https://devfeed.tech/tags/automated.md>), [automation](<https://devfeed.tech/tags/automation.md>), [autonomy](<https://devfeed.tech/tags/autonomy.md>), [control](<https://devfeed.tech/tags/control.md>), [data-strategy](<https://devfeed.tech/tags/data-strategy.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [regulated-industries](<https://devfeed.tech/tags/regulated-industries.md>)

### AI overview

The article presents four ways to place agents within regulated systems while keeping controls and decision authority explicit: agents may investigate a deterministic control without deciding its result; generate candidates judged against objective, approved acceptance tests; assemble evidence and recommendations for an accountable human decision; or act within a defined operating envelope, with policy checks and escalation for exceptions. It stresses that passing validation does not prove the acceptance contract covers every production behavior, and that consequential recommendations need traceable evidence and specific explanations. In the access example, deterministic policy services validate conditions before Unity Catalog enforces a time-bound grant; out-of-envelope requests go to a human reviewer. The patterns can be combined. The article also names Databricks components for governance, agent composition, connectivity, and audit evidence, while noting that institution-specific implementations vary.

### Source excerpt

Enterprise AI cannot replace every deterministic control. These four patterns help...

## Where should AI workloads run? Probably next to the data.

DevFeed: [Where should AI workloads run? Probably next to the data.](<https://devfeed.tech/articles/where-should-ai-workloads-run-probably-next-to-the-data-144043.md>)

Original publisher: [Read original article](<https://thenewstack.io/ai-workloads-data-gravity/>)

Author: Shubham Thakur

Published: 2026-11-19T16:30:00Z

Content type: article

Language: en

Sources: [The New Stack](<https://devfeed.tech/sources/the-new-stack.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [data gravity](<https://devfeed.tech/topics/data-gravity.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [data mobility](<https://devfeed.tech/topics/data-mobility.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [ML model versioning](<https://devfeed.tech/topics/ml-model-versioning.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [data lake](<https://devfeed.tech/topics/data-lake.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [acceldata](<https://devfeed.tech/tags/acceldata.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [access-controls](<https://devfeed.tech/tags/access-controls.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-state](<https://devfeed.tech/tags/agent-state.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-governance](<https://devfeed.tech/tags/ai-governance.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-stack](<https://devfeed.tech/tags/ai-stack.md>), [ai-strategy](<https://devfeed.tech/tags/ai-strategy.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [apache](<https://devfeed.tech/tags/apache.md>), [aws](<https://devfeed.tech/tags/aws.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [data](<https://devfeed.tech/tags/data.md>), [ec2](<https://devfeed.tech/tags/ec2.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [post-contributed](<https://devfeed.tech/tags/post-contributed.md>), [prices](<https://devfeed.tech/tags/prices.md>), [sponsor-acceldata](<https://devfeed.tech/tags/sponsor-acceldata.md>), [sponsored-post-contributed](<https://devfeed.tech/tags/sponsored-post-contributed.md>)

### AI overview

AI workload placement should account for where data already lives: repeated large reads can make cross-region or cross-environment transfers costly and slow, while moving compute may avoid full-dataset copies. The article recommends evaluating each job against residency and access rules, compatible accelerator and runtime capacity, performance, and total cost, including transfer, storage, queueing, and engineering overhead. Workloads differ: bursty training may suit cloud capacity, residency-bound jobs must stay in an approved jurisdiction, and transaction-near inference must meet latency targets. Hybrid setups can keep data-heavy or constrained work local while using cloud elasticity where worthwhile, provided identity, policy, observability, and runtime compatibility span environments. Open table formats can support in-place reads across engines, but portability depends on target support. Sovereignty also covers prompts, retrieval indexes, orchestration state, and access controls, not only GPU location. Revisit placement as prices, data, or deadlines change; if no option meets policy and capacity requirements, queue or reject the job.

### Source excerpt

AWS raised prices on EC2 Capacity Blocks, its reserved GPU capacity, by about 15% in January and about 20% in The post Where should AI workloads run? Probably next to the data. appeared first on The New Stack.

## AI's next build-out: Inference racks that resize themselves for every new model

DevFeed: [AI's next build-out: Inference racks that resize themselves for every new model](<https://devfeed.tech/articles/ai-s-next-build-out-inference-racks-that-resize-themselves-for-every-new-model-143792.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/19/memory-pooling-helps-liqid-fix-ai-inference-tokenomics-sc26/>)

Author: Ryan Stevens

Published: 2026-11-19T15:43:30Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-data-centers](<https://devfeed.tech/tags/ai-data-centers.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-inference-economics](<https://devfeed.tech/tags/ai-inference-economics.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-servers](<https://devfeed.tech/tags/ai-servers.md>), [ai-supercomputers](<https://devfeed.tech/tags/ai-supercomputers.md>), [application](<https://devfeed.tech/tags/application.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [composable-infrastructure](<https://devfeed.tech/tags/composable-infrastructure.md>), [compute-express-link](<https://devfeed.tech/tags/compute-express-link.md>), [constraint](<https://devfeed.tech/tags/constraint.md>), [cube-event-coverage](<https://devfeed.tech/tags/cube-event-coverage.md>), [cxl](<https://devfeed.tech/tags/cxl.md>), [digital-twins](<https://devfeed.tech/tags/digital-twins.md>), [disaggregated-memory](<https://devfeed.tech/tags/disaggregated-memory.md>), [energy-efficiency](<https://devfeed.tech/tags/energy-efficiency.md>), [exascale-computing](<https://devfeed.tech/tags/exascale-computing.md>), [gemma-allen](<https://devfeed.tech/tags/gemma-allen.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-pooling](<https://devfeed.tech/tags/gpu-pooling.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [graphics-processing-unit](<https://devfeed.tech/tags/graphics-processing-unit.md>), [heterogeneous-computing](<https://devfeed.tech/tags/heterogeneous-computing.md>), [high-performance-computing](<https://devfeed.tech/tags/high-performance-computing.md>), [high-performance-storage](<https://devfeed.tech/tags/high-performance-storage.md>), [high-speed-networking](<https://devfeed.tech/tags/high-speed-networking.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [hpc-infrastructure](<https://devfeed.tech/tags/hpc-infrastructure.md>), [hpc-unites](<https://devfeed.tech/tags/hpc-unites.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infra](<https://devfeed.tech/tags/infra.md>), [john-furrier](<https://devfeed.tech/tags/john-furrier.md>), [liqid](<https://devfeed.tech/tags/liqid.md>), [liqid-as-a-service](<https://devfeed.tech/tags/liqid-as-a-service.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-pooling](<https://devfeed.tech/tags/memory-pooling.md>), [national-laboratories](<https://devfeed.tech/tags/national-laboratories.md>), [news](<https://devfeed.tech/tags/news.md>), [parallel-file-systems](<https://devfeed.tech/tags/parallel-file-systems.md>), [quantum-computing](<https://devfeed.tech/tags/quantum-computing.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [sc26eventpage](<https://devfeed.tech/tags/sc26eventpage.md>), [sumit-puri](<https://devfeed.tech/tags/sumit-puri.md>), [thecube](<https://devfeed.tech/tags/thecube.md>)

### AI overview

Liqid proposes pooling GPUs and memory in a rack and dynamically assigning them to servers according to model or workload, instead of fixing each server's GPU-to-memory ratio. The company says delivering only the memory an application needs can stretch constrained supply. It measures inference economics in tokens per second, tokens per watt and tokens per dollar. At SC26, Liqid announced Liqid as a Service, a fully managed, single-tenant offering, and said it is moving toward preconfigured appliances. The excerpt presents these as Liqid's plans and rationale; it provides no independent performance results.

### Source excerpt

Memory pooling is fast becoming one of the sharpest tools in AI inference economics, with companies racing to wring more output from every graphics processing unit. Most AI servers still ship with a fixed GPU-to-memory ratio that holds for the life of the machine. But that is a poor fit when nobody knows what model [...] The post AI's next build-out: Inference racks that resize themselves for every new model appeared first on SiliconANGLE.

## How to Oversee AI Agents at Scale With a Three-Tier Architecture

DevFeed: [How to Oversee AI Agents at Scale With a Three-Tier Architecture](<https://devfeed.tech/articles/how-to-oversee-ai-agents-at-scale-with-a-three-tier-architecture-143637.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/how-to-oversee-ai-agents-at-scale-with-a-three-tier-architecture/>)

Author: Barr Moses

Published: 2026-11-19T13:30:00Z

Content type: opinion

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [human review](<https://devfeed.tech/topics/human-review.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [scalable-architecture](<https://devfeed.tech/topics/scalable-architecture.md>)

Tags: [agent-architecture](<https://devfeed.tech/tags/agent-architecture.md>), [agent-observability](<https://devfeed.tech/tags/agent-observability.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-evaluation](<https://devfeed.tech/tags/ai-evaluation.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [human-review](<https://devfeed.tech/tags/human-review.md>), [humans](<https://devfeed.tech/tags/humans.md>), [impact](<https://devfeed.tech/tags/impact.md>), [oversight](<https://devfeed.tech/tags/oversight.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [scale](<https://devfeed.tech/tags/scale.md>), [supervisor](<https://devfeed.tech/tags/supervisor.md>)

### AI overview

The article proposes a three-tier setup for scaling agent oversight: task-performing agents, an independent automated supervisor for runtime monitoring and reversible actions, and humans who set policy and approve high-impact changes. It argues that manual review cannot keep pace with agents and may miss silent failures, so the supervisor should use separate compute and credentials, independent telemetry, and append-only audit logs. Humans define task-level autonomy rules; the supervisor checks them against independently computed metrics, can pause or buffer a batch, and escalates irreversible fixes for approval. In the currency-conversion example, a spike in transactions using a fallback rate triggers a batch hold and a proposed schema-mapping patch for an engineer to approve. The author recommends keeping the supervisor simple, limiting it to easy-to-undo actions, and having humans spot-check its decisions.

### Source excerpt

Separate runtime oversight from human review: an autonomous supervisor handles reversible fixes, while humans approve high-impact changes. The post How to Oversee AI Agents at Scale With a Three-Tier Architecture appeared first on Towards Data Science.

## Language Model Security Modules: Turning LLM Interpretability into Runtime Defense

DevFeed: [Language Model Security Modules: Turning LLM Interpretability into Runtime Defense](<https://devfeed.tech/articles/language-model-security-modules-turning-llm-interpretability-into-runtime-defense-143106.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.25697>)

Author: XiuYu Zhang, Bonan Ruan, Junfeng Fang, An Zhang, Tat-Seng Chua, Zhenkai Liang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [model-security](<https://devfeed.tech/tags/model-security.md>), [runtime](<https://devfeed.tech/tags/runtime.md>)

### AI overview

LMSM adds a shared mediation path to LLM serving so interpretability signals can enforce runtime policies without each detector needing its own integration. Pluggable backends provide evidence such as probe or autoencoder signals; policies specify required evidence and evaluations, and LMSM verifies they ran before allowing output. In vLLM tests on Qwen3-4B and Gemma-3-4B, including eager and compiled execution, each evaluated backend retained at least 96% of native throughput with default optimizations. On Qwen3-4B, a representative probe policy reduced HarmBench attack success from 39.4% to 3.3%, while XSTest false refusals increased from 0.8% to 3.1%.

### Source excerpt

arXiv:2608.25697v2 Announce Type: replace Abstract: Large language models (LLMs) are served behind layered defenses such as alignment and guardrail models, yet jailbreaks still get through. Interpretability research looks inside the model, where methods such as probes and sparse autoencoders expose signals of harmful content during generation. A signal alone, however, is not a defense. To become one, the serving system must tie each signal to the right request, apply the policy as declared, and release output only when the policy allows. Today, defenses built on such signals handle these duties on their own, so every new detector needs its own integration. Yet none of these duties depends on how a signal is computed. We present Language Model Security Modules (LMSM), which adapts the design of Linux Security Modules to LLM serving and carries out these duties once in a shared mediation path. The path hosts pluggable backends that turn model internals into evidence and policies that decide what that evidence allows. Each policy declares the evidence and evaluations it needs, and LMSM provides exactly these for each request and verifies that they took place, so a failure is never mistaken for permission. Protection therefore remains cheap and correct under common serving optimizations. We evaluate LMSM in vLLM, with both eager and compiled execution, on Qwen3-4B and Gemma-3-4B with fitted probes, a released transcoder, and a released sparse autoencoder. With vLLM's default optimizations on, every backend keeps at least 96% of native throughput, and all of them share the same mediation path. On Qwen3-4B, a representative probe-based policy cuts HarmBench attack success from 39.4% to 3.3% while XSTest false refusals rise from 0.8% to 3.1%. LMSM lets existing and future interpretability methods become runtime defenses without rebuilding the serving path.

## BatchGen: An Architecture for Scalable and Efficient Batch Inference

DevFeed: [BatchGen: An Architecture for Scalable and Efficient Batch Inference](<https://devfeed.tech/articles/batchgen-an-architecture-for-scalable-and-efficient-batch-inference-142960.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.21712>)

Author: Tairan Xu, Leyang Xue, Zhan Lu, Jinfu Deng, Hongyang Xiao, Yinsicheng Jiang, Congjie He, Matej Sandor, Le Xu, Luo Mai

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai](<https://devfeed.tech/tags/ai.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [batch](<https://devfeed.tech/tags/batch.md>), [batches](<https://devfeed.tech/tags/batches.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [completion](<https://devfeed.tech/tags/completion.md>), [compute](<https://devfeed.tech/tags/compute.md>), [coroutine](<https://devfeed.tech/tags/coroutine.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [devices](<https://devfeed.tech/tags/devices.md>), [engines](<https://devfeed.tech/tags/engines.md>), [event](<https://devfeed.tech/tags/event.md>), [event-driven](<https://devfeed.tech/tags/event-driven.md>), [execution](<https://devfeed.tech/tags/execution.md>), [existing](<https://devfeed.tech/tags/existing.md>), [fail](<https://devfeed.tech/tags/fail.md>), [inference](<https://devfeed.tech/tags/inference.md>)

### AI overview

BatchGen uses a sequence coroutine compute model to address runtime load variation and GPU utilization challenges in large-scale batch inference. It represents each sequence as a fine-grained, event-driven coroutine, allowing the runtime to form larger expert-level batches, mitigate stragglers, move work across devices, and sustain utilization on cost-effective or memory-constrained GPUs. The abstract reports up to 2.3x shorter batch completion time on a 128-GPU cluster and up to 9.6x better performance than the strongest offloading baseline on memory-constrained accelerators.

### Source excerpt

arXiv:2606.21712v2 Announce Type: replace Abstract: Batch inference has become a central mode of AI computation, yet existing inference engines still rely on execution models designed for interactive serving. When scaled to millions of sequences, batch workloads reveal two fundamental requirements: the ability to handle extreme inter- and intra-sequence load variation that emerges only at runtime, and the ability to sustain high utilization across large fleets of GPUs. Existing systems fail to meet these requirements, losing substantial fractions of achievable throughput. We introduce a new architectural foundation for batch inference: the sequence coroutine compute model, which represents each sequence as a fine-grained, event-driven coroutine. This model exposes expressive primitives that allow the runtime to reorganize work dynamically, enabling larger expert-level batches, mitigating stragglers, reallocating work across devices, and maintaining utilization even on cost-effective or memory-constrained GPUs. Building on this abstraction, we implement BatchGen, a production-ready system that uses the coroutine model at cluster scale. On a 128-GPU cluster, BatchGen reduces batch completion time by up to $2.3\times$, and on memory-constrained accelerators it outperforms the strongest offloading baseline by up to $9.6\times$. We will open-source BatchGen at https://github.com/batchgen-project/batchgen

## Quantifying the Intrusiveness of Retrofitting Retrieval-Augmented Generation onto Legacy Software Systems

DevFeed: [Quantifying the Intrusiveness of Retrofitting Retrieval-Augmented Generation onto Legacy Software Systems](<https://devfeed.tech/articles/quantifying-the-intrusiveness-of-retrofitting-retrieval-augmented-generation-onto-legacy-software-systems-142202.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23931>)

Author: K. M. Tahlil Mahfuz Faruk, Rakib Abdullah

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Legacy Modernization](<https://devfeed.tech/topics/legacy-modernization.md>), [legacy systems](<https://devfeed.tech/topics/legacy-systems.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [backends](<https://devfeed.tech/tags/backends.md>), [call](<https://devfeed.tech/tags/call.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [churn](<https://devfeed.tech/tags/churn.md>), [code](<https://devfeed.tech/tags/code.md>), [commits](<https://devfeed.tech/tags/commits.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [delay](<https://devfeed.tech/tags/delay.md>), [dependency](<https://devfeed.tech/tags/dependency.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [generation](<https://devfeed.tech/tags/generation.md>), [legacy](<https://devfeed.tech/tags/legacy.md>), [legacy-software](<https://devfeed.tech/tags/legacy-software.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [retrieval-augmented-generation](<https://devfeed.tech/tags/retrieval-augmented-generation.md>), [software](<https://devfeed.tech/tags/software.md>), [software-systems](<https://devfeed.tech/tags/software-systems.md>), [systems](<https://devfeed.tech/tags/systems.md>)

### AI overview

The paper proposes three ways to add retrieval-augmented generation (RAG) to legacy software--sidecar, SDK-embed and naive-invasive--and a Legacy Intrusion Score (LIS) for comparing code changes, extended by LIS-X to cover operational effects. Across five systems, the code-level ranking is sidecar < SDK-embed < naive-invasive, though the authors say this follows from the metrics' component-wise dominance and validates the score rather than testing a hypothesis. In a live study on one host and one system, sidecar showed no detectable endpoint latency increase; SDK-embed and naive-invasive added 569 ms and 475 ms, with no significant difference. Both passed a 2-second dependency delay through to legacy requests, and RAG failure could leave successful responses without the AI field. RAG load slowed the legacy service on a shared host, while sidecar data staleness averaged 911 ms at a 2-second polling interval. These operational findings are limited to that single setup.

### Source excerpt

arXiv:2611.23931v1 Announce Type: new Abstract: Enterprises increasingly add retrieval-augmented generation (RAG) to existing REST and monolithic systems, usually by writing an ad hoc HTTP call into a handler, and have no way to quantify how intrusive that choice is. We present (1) a taxonomy of three retrofit patterns, sidecar, SDK-embed and naive-invasive; (2) the Legacy Intrusion Score (LIS), three code sub-metrics (churn, breadth, coupling), and an extension, LIS-X, with three operational sub-metrics measured on a live system (run-time overhead, fault propagation and silent feature degradation); (3) a static evaluation on five legacy systems, two of them real open-source codebases fetched from their repositories at pinned commits; and (4) a live case study that deploys all three patterns as separate OS processes. The code-level ranking sidecar < SDK-embed < naive-invasive holds in all five systems and under 2,000 random weightings, but it follows from component-wise dominance of the sub-metrics, so it validates the metric rather than testing a hypothesis. In the live study, outcomes are not fixed by definition. The sidecar adds no detectable latency to the legacy endpoint (0.07 ms, 95% CI [-0.10, 0.24]), whereas SDK-embed and naive-invasive add 569 and 475 ms and are not distinguishable from each other (p=0.62). Both pass a 2 s dependency delay on to legacy requests, and when the RAG layer fails fast they keep returning successful responses that no longer contain the AI field. Load on the RAG layer slows the legacy service by 1.51-1.57x on a shared host, and sidecar staleness averages 911 ms with a 2 s polling interval. Retrieval hit@1 is 0.95; three hosted LLMs answer all 12 test questions correctly and a local model 9/12, and the four backends are swapped by changing one request field. All dynamic results come from one host and one system.

## Neural Memory Orchestration

DevFeed: [Neural Memory Orchestration](<https://devfeed.tech/articles/neural-memory-orchestration-142972.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23857>)

Author: Michael Zizov, Boris Morose

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Memory Heirarchy](<https://devfeed.tech/topics/memory-heirarchy.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-os](<https://devfeed.tech/tags/cs-os.md>), [ebpf](<https://devfeed.tech/tags/ebpf.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [memory](<https://devfeed.tech/tags/memory.md>), [neural](<https://devfeed.tech/tags/neural.md>), [prefetching](<https://devfeed.tech/tags/prefetching.md>), [reductions](<https://devfeed.tech/tags/reductions.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [sequences](<https://devfeed.tech/tags/sequences.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

Overseer applies learned page prefetching to live HPC workloads: an eBPF collector reports page faults, the OverNet transformer predicts future page addresses on network-attached inference hardware, and a kernel enforcer loads pages before they are needed. OverNet represents fault sequences using per-instance ordinal ranks over deltas, allowing one trained model to serve different target processes without retraining. In two page-fault-heavy workloads, the system reduced runtime by roughly 35% for Hilbert-curve traversal and by about an order of magnitude for dense matrix multiplication, with fewer major faults. These results are limited to the two workloads described.

### Source excerpt

arXiv:2611.23857v1 Announce Type: new Abstract: Modern HPC workloads increasingly stall on the memory hierarchy rather than on compute, and page faults from mmap'd files, swap, and tiered memory are a growing fraction of wall time. Heuristic prefetchers in the kernel handle stride-like and temporally clustered patterns well but degrade sharply on the irregular, data-dependent access streams typical of these workloads. Learned prefetchers have been shown to capture such patterns in simulation but have not been taken to a deployed pipeline running against live workloads on real hardware. We present Overseer, an end-to-end learned page-prefetching system. An eBPF collector observes page faults on the target host, a transformer predictor (OverNet) runs on network-attached inference hardware and emits future page addresses, and a kernel-resident enforcer materializes those pages into the target's address space before they are demanded. OverNet encodes fault sequences through per-instance ordinal ranks over deltas, which lets a single trained model serve any target process without retraining. On two page-fault-heavy workloads, Overseer reduces runtime by roughly 35% on Hilbert-curve traversal and by roughly an order of magnitude on dense matrix multiplication, with corresponding reductions in major-fault counts.

## How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI

DevFeed: [How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI](<https://devfeed.tech/articles/how-salesforce-is-advancing-global-scale-model-serving-with-cross-region-inference-routing-in-amazon-sagemaker-ai-141212.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/how-salesforce-is-advancing-global-scale-model-serving-with-cross-region-inference-routing-in-amazon-sagemaker-ai/>)

Author: Apoorva Chandra

Published: 2026-11-18T20:13:18Z

Content type: tutorial

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Advanced AI workflows](<https://devfeed.tech/topics/advanced-ai-workflows.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [amazon-sagemaker-ai](<https://devfeed.tech/tags/amazon-sagemaker-ai.md>), [call](<https://devfeed.tech/tags/call.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [cross](<https://devfeed.tech/tags/cross.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [data](<https://devfeed.tech/tags/data.md>), [data-residency](<https://devfeed.tech/tags/data-residency.md>), [global](<https://devfeed.tech/tags/global.md>), [inference](<https://devfeed.tech/tags/inference.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [model](<https://devfeed.tech/tags/model.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [region](<https://devfeed.tech/tags/region.md>), [regions](<https://devfeed.tech/tags/regions.md>), [routing](<https://devfeed.tech/tags/routing.md>), [salesforce](<https://devfeed.tech/tags/salesforce.md>), [scale](<https://devfeed.tech/tags/scale.md>), [serving](<https://devfeed.tech/tags/serving.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

Salesforce replaced its Internal Gateway's basic health checks, retries, and Region switching with an Amazon SageMaker AI inference router that sends calls across its existing endpoints and accounts. The router prefers local capacity, shifts traffic when request errors indicate an unhealthy or unavailable destination, and gradually returns traffic after recovery; high latency alone does not trigger failover. Residency-tagged requests use a target containing only the local endpoint. Endpoint owners authorize cross-account access with resource policies, and response fields identify the serving resource and routing reason for logs and CloudWatch metrics. In a saturation test, traffic stayed local during a latency rise, then shifted after container timeouts; about 78% reached the remote Region, with p50 latency returning to 0.2 seconds and under 100 ms of cross-Region overhead at p50. The router does not deploy models or add capacity, so registered Regions need matching model artifacts and sufficient capacity; a receiving endpoint can cap remote traffic. Salesforce plans further tests of sustained saturation, latency, and cascading failures.

### Source excerpt

Learn how Salesforce uses the cross-Region inference router in Amazon SageMaker AI to turn fragmented Regional capacity into reliable model serving. Salesforce adds capacity securely across Regions and accounts while keeping its data residency commitments, and sees per request where each call was served and why.

## Hammerspace enters the hardware market with new storage and intelligence capabilities

DevFeed: [Hammerspace enters the hardware market with new storage and intelligence capabilities](<https://devfeed.tech/articles/hammerspace-enters-the-hardware-market-with-new-storage-and-intelligence-capabilities-138530.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/17/hammerspace-enters-the-hardware-market-with-new-storage-and-intelligence-capabilities/>)

Author: Paul Gillin

Published: 2026-11-17T21:00:42Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI-Ready Data](<https://devfeed.tech/topics/ai-ready-data.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [systems](<https://devfeed.tech/topics/systems.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-data-platform](<https://devfeed.tech/tags/ai-data-platform.md>), [ai-training-and-inference](<https://devfeed.tech/tags/ai-training-and-inference.md>), [data-orchestration](<https://devfeed.tech/tags/data-orchestration.md>), [distributed-data](<https://devfeed.tech/tags/distributed-data.md>), [distributed-file-catalog](<https://devfeed.tech/tags/distributed-file-catalog.md>), [flash-storage](<https://devfeed.tech/tags/flash-storage.md>), [hammerspace](<https://devfeed.tech/tags/hammerspace.md>), [hammerspace-data-system](<https://devfeed.tech/tags/hammerspace-data-system.md>), [hammerspace-inc](<https://devfeed.tech/tags/hammerspace-inc.md>), [hammerspace-open-flash-platform](<https://devfeed.tech/tags/hammerspace-open-flash-platform.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [infra](<https://devfeed.tech/tags/infra.md>), [intelligence](<https://devfeed.tech/tags/intelligence.md>), [news](<https://devfeed.tech/tags/news.md>), [non-volatile-memory-express-storage](<https://devfeed.tech/tags/non-volatile-memory-express-storage.md>), [parallel-network-file-system](<https://devfeed.tech/tags/parallel-network-file-system.md>), [storage](<https://devfeed.tech/tags/storage.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [tier-0-storage](<https://devfeed.tech/tags/tier-0-storage.md>)

### AI overview

Hammerspace is adding turnkey hardware and data intelligence to its data orchestration software for AI workloads. Its Open Flash Platform uses data processing units to combine processing, networking, and SSDs; the company claims 350 GB/s per rack unit, while broader performance testing is still underway. Customers can combine it with Tier 0 NVMe storage and lower-cost disk or object storage in one namespace, or keep using third-party storage. The intelligence module catalogs distributed files, indexes metadata, scans for sensitive data, and supports natural-language search and policy creation; current customers can receive it through a software upgrade without changing their architecture. Systems are configured and supported for each customer, hardware is priced individually, and software pricing remains based on managed data capacity.

### Source excerpt

Hammerspace Inc., which provides high-speed access to distributed data, is expanding beyond its traditional role as a software provider to offer complete, integrated data systems for artificial intelligence workloads, adding its own high-performance flash storage hardware and new data intelligence capabilities to its existing data orchestration platform. The company said the Hammerspace Data System combines [...] The post Hammerspace enters the hardware market with new storage and intelligence capabilities appeared first on SiliconANGLE.

## Secure SI by Design Consortium Officially Approved by U.S. Government

DevFeed: [Secure SI by Design Consortium Officially Approved by U.S. Government](<https://devfeed.tech/articles/secure-si-by-design-consortium-officially-approved-by-u-s-government-144025.md>)

Original publisher: [Read original article](<https://www.paloaltonetworks.com/blog/2026/11/secure-si-by-design-consortium-officially-approved-by-u-s-government/>)

Author: Daniel Kroese and Simone Gammeri

Published: 2026-11-17T16:59:52Z

Content type: news

Language: en

Sources: [Palo Alto Networks](<https://devfeed.tech/sources/palo-alto-networks.md>)

Topics: [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [78](<https://devfeed.tech/tags/78.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [accelerate](<https://devfeed.tech/tags/accelerate.md>), [adoption](<https://devfeed.tech/tags/adoption.md>), [advocacy](<https://devfeed.tech/tags/advocacy.md>), [announcement](<https://devfeed.tech/tags/announcement.md>), [coalition](<https://devfeed.tech/tags/coalition.md>), [cognizant](<https://devfeed.tech/tags/cognizant.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [design](<https://devfeed.tech/tags/design.md>), [factory](<https://devfeed.tech/tags/factory.md>), [government](<https://devfeed.tech/tags/government.md>), [ibm](<https://devfeed.tech/tags/ibm.md>), [industry](<https://devfeed.tech/tags/industry.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [openai](<https://devfeed.tech/tags/openai.md>), [palo-alto-networks](<https://devfeed.tech/tags/palo-alto-networks.md>), [points-of-view](<https://devfeed.tech/tags/points-of-view.md>), [public-sector](<https://devfeed.tech/tags/public-sector.md>), [reflection](<https://devfeed.tech/tags/reflection.md>), [servicenow](<https://devfeed.tech/tags/servicenow.md>), [snowflake](<https://devfeed.tech/tags/snowflake.md>), [united-states](<https://devfeed.tech/tags/united-states.md>)

### AI overview

The U.S. government approved the Secure SI by Design Consortium under the Department of Commerce's American SI Exports Program. Led by Palo Alto Networks with partners including NVIDIA, IBM, OpenAI, Snowflake, Cognizant, Reflection, Factory, and ServiceNow, it brings together companies across infrastructure, data pipelines, models, cybersecurity, and applications to embed security throughout the SI stack. The article says approval provides streamlined government-to-government advocacy, priority export-control licensing reviews, and access to financing mechanisms through EXIM Bank and the U.S. International Development Finance Corporation. For agent governance, it points to automated registries, standardized risk assessments, and real-time monitoring. The text cites industry figures describing limited security measures and oversight despite widespread agent adoption, but provides no details on how the proposed tools will be implemented or evaluated.

### Source excerpt

The United States government has officially approved the Secure SI by Design Consortium, an industry coalition led by Palo Alto Networks alongside NVIDIA, IBM, OpenAI, Snowflake, Cognizant, Reflection, Factory, and ServiceNow. Established ... The post Secure SI by Design Consortium Officially Approved by U.S. Government appeared first on Palo Alto Networks Blog.

## What Production AI Demands from the Data Layer

DevFeed: [What Production AI Demands from the Data Layer](<https://devfeed.tech/articles/what-production-ai-demands-from-the-data-layer-139722.md>)

Original publisher: [Read original article](<https://blog.everpuredata.com/solutions/what-production-ai-demands-from-the-data-layer/>)

Author: Sabrinath Rao

Published: 2026-11-17T13:50:00Z

Content type: opinion

Language: en

Sources: [Everpure](<https://devfeed.tech/sources/everpure.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [scalable-architecture](<https://devfeed.tech/topics/scalable-architecture.md>), [AI Factory](<https://devfeed.tech/topics/ai-factory.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [IO](<https://devfeed.tech/topics/io.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-and-machine-learning](<https://devfeed.tech/tags/ai-and-machine-learning.md>), [ai-data](<https://devfeed.tech/tags/ai-data.md>), [ai-data-layer](<https://devfeed.tech/tags/ai-data-layer.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [ai-training-and-inference](<https://devfeed.tech/tags/ai-training-and-inference.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [cache](<https://devfeed.tech/tags/cache.md>), [data](<https://devfeed.tech/tags/data.md>), [data-layer](<https://devfeed.tech/tags/data-layer.md>), [flashblade-exa](<https://devfeed.tech/tags/flashblade-exa.md>), [flashblade-object-storage](<https://devfeed.tech/tags/flashblade-object-storage.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [inference](<https://devfeed.tech/tags/inference.md>), [production-ai](<https://devfeed.tech/tags/production-ai.md>), [solutions](<https://devfeed.tech/tags/solutions.md>)

### AI overview

The post argues that production AI benefits when storage resources scale independently and are measured by their effect on useful compute. FlashBlade//EXA separates metadata operations from payload traffic, allowing each path to scale on demand; STN reports growth from about 35 GB/s to as much as 100 GB/s. In submitted MLPerf Storage v3.0 results, it ranked first on the reported checkpointing and KV cache metrics, with checkpoint write bandwidth rising from 327.65 GiB/s on 10 data nodes to 877.52 GiB/s on 30; the author cautions these workloads are proxies and recommends testing real workload mixes. Purity//FB 4.8.4 adds S3 over RDMA through standard S3 calls, though the newer capability has early-release constraints. Pure KVA retains attention states and integrates with vLLM; company benchmarks report faster inference and improved time to first token. For multi-tenancy, file systems are assigned to separate data node groups to isolate drives, ports, and failures. The suggested evaluations include checkpoint stall time, goodput, concurrent time to first token, data copies, node-failure impact, and watts per sustained GB/s.

### Source excerpt

What Production AI Demands from the Data Layer by Everpure Blog Explore how a scalable AI data layer can accelerate checkpointing and inference while helping keep GPUs productive across AI workloads. The post What Production AI Demands from the Data Layer appeared first on Everpure Blog.

## How to Design MCP Servers That Leave Orchestration to the Host

DevFeed: [How to Design MCP Servers That Leave Orchestration to the Host](<https://devfeed.tech/articles/how-to-design-mcp-servers-that-leave-orchestration-to-the-host-137193.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/how-to-design-mcp-servers-that-leave-orchestration-to-the-host/>)

Author: Priyansh Bhardwaj

Published: 2026-11-17T12:00:00Z

Content type: article

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [API Governance](<https://devfeed.tech/topics/api-governance.md>), [Back end](<https://devfeed.tech/topics/backend.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [agent-architecture](<https://devfeed.tech/tags/agent-architecture.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [api](<https://devfeed.tech/tags/api.md>), [audit](<https://devfeed.tech/tags/audit.md>), [design](<https://devfeed.tech/tags/design.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [mcp](<https://devfeed.tech/tags/mcp.md>), [mcp-servers](<https://devfeed.tech/tags/mcp-servers.md>), [model-context-protocol](<https://devfeed.tech/tags/model-context-protocol.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [planning](<https://devfeed.tech/tags/planning.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [python](<https://devfeed.tech/tags/python.md>), [python-sdk](<https://devfeed.tech/tags/python-sdk.md>), [resource](<https://devfeed.tech/tags/resource.md>), [sdk](<https://devfeed.tech/tags/sdk.md>), [server](<https://devfeed.tech/tags/server.md>), [software-architecture](<https://devfeed.tech/tags/software-architecture.md>), [state](<https://devfeed.tech/tags/state.md>), [tools](<https://devfeed.tech/tags/tools.md>), [v2](<https://devfeed.tech/tags/v2.md>), [workflow](<https://devfeed.tech/tags/workflow.md>)

### AI overview

Treat MCP as an integration boundary: servers expose domain capabilities, while hosts own planning, user interaction, workflow state, identity, deadlines and consequential-action policy. The Python SDK v2 catalogue example uses two narrow read-only tools, one JSON resource and one bounded prompt, with no planner or model client. It returns structured fields and limits dependency lookup to direct dependencies. The host can call only what the request needs, making the same server usable across different applications. The excerpt says MCP 2026-07-28 removed session identifiers and the initialization handshake, making the protocol core stateless; durable business workflows still require explicit identifiers and an owned store. Remote deployments still need authentication, action- and tenant-specific authorization, timeout and consistency policies, and careful failure handling. Writes additionally call for explicit approval, idempotency where retries are possible, deadlines and audit records. A specialised server workflow can be justified when it has a clear owner and permission model.

### Source excerpt

A Python SDK v2 catalogue server with two narrow tools, one resource, and one prompt shows how to keep planning, workflow state, and authorisation host-side. The post How to Design MCP Servers That Leave Orchestration to the Host appeared first on Towards Data Science.

## Disaggregated LLM Inference in Space: A Communication Feasibility Study

DevFeed: [Disaggregated LLM Inference in Space: A Communication Feasibility Study](<https://devfeed.tech/articles/disaggregated-llm-inference-in-space-a-communication-feasibility-study-137034.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20893>)

Author: Arno Troch, Roger Sanchez-Vital, Seyyidahmed Lahmer, Bartek Kozicki, Jeroen Famaey, Michael Peeters

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [communications](<https://devfeed.tech/topics/communications.md>), [Latency](<https://devfeed.tech/topics/latency.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Data transmission](<https://devfeed.tech/topics/data-transmission.md>)

Tags: [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [cache](<https://devfeed.tech/tags/cache.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [decode](<https://devfeed.tech/tags/decode.md>), [delay](<https://devfeed.tech/tags/delay.md>), [disaggregation](<https://devfeed.tech/tags/disaggregation.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fast](<https://devfeed.tech/tags/fast.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [space](<https://devfeed.tech/tags/space.md>)

### AI overview

The abstract finds that orbital disaggregated LLM inference depends on the serving method's communication pattern. Prefill-decode disaggregation (PDD) transfers a large KV cache once per request, so link capacity is its main constraint; with fast enough links, it can tolerate satellite separations up to 1,000 km. Attention-FFN disaggregation (AFD) exchanges small activations at every mixture-of-experts layer for every generated token, so propagation delay accumulates layer by layer and requires attention and expert instances to remain within about 1 km. The study proposes a hierarchical orbital inference architecture and network design guidelines that also account for ground access latency.

### Source excerpt

arXiv:2611.20893v1 Announce Type: new Abstract: Orbital data centers could enable large-scale AI inference, but efficiently serving a frontier language model to many users may require several compute satellites to cooperate over optical links. While disaggregated inference, which splits large language model (LLM) serving across machines, is well established in terrestrial data centers, its viability under the unique constraints of orbital deployments remains an open question. This article investigates how link capacity and satellite separation affect two types of disaggregated LLM serving, in both compact satellite formations and distributed constellations. Prefill-decode disaggregation (PDD) migrates a large KV cache once per request, while attention-FFN disaggregation (AFD) exchanges small activations at every mixture-of-experts layer for every generated token. The two respond to distance in different ways. PDD is limited mainly by link capacity and, with fast enough links, tolerates separations of up to a thousand kilometers. AFD is limited mainly by propagation delay, which accumulates layer by layer and forces attention and expert instances within about a kilometer of each other. Based on these results, we propose a hierarchical orbital inference architecture and provide network design guidelines for future orbital data centers, including the role of ground access latency.

## RESILAGENT: A Three-Pillar Architecture for Cyber-Physical Resilience in Electrical Power and Energy Systems

DevFeed: [RESILAGENT: A Three-Pillar Architecture for Cyber-Physical Resilience in Electrical Power and Energy Systems](<https://devfeed.tech/articles/resilagent-a-three-pillar-architecture-for-cyber-physical-resilience-in-electrical-power-and-energy-systems-137706.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21480>)

Author: Nikolaos Kekatos, Georgios Koutidis, Konstantinos Antonakopoulos, Stylianos Basagiannis, Sokratis Katsikas, Georgios Kavallieratos, Alexios Lekidis, Tom Nianios, Elpiniki Papageorgiou

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Resilience](<https://devfeed.tech/topics/resilience.md>), [cyber resiliency](<https://devfeed.tech/topics/cyber-resiliency.md>), [AI in cybersecurity](<https://devfeed.tech/topics/ai-in-cybersecurity.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Electrical engineering and computer science (EECS)](<https://devfeed.tech/topics/electrical-engineering-and-computer-science-eecs.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-artificial-intelligence](<https://devfeed.tech/tags/agentic-artificial-intelligence.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [attack](<https://devfeed.tech/tags/attack.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [blockchain](<https://devfeed.tech/tags/blockchain.md>), [bus](<https://devfeed.tech/tags/bus.md>), [climate](<https://devfeed.tech/tags/climate.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-based](<https://devfeed.tech/tags/cloud-based.md>), [compromise](<https://devfeed.tech/tags/compromise.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [cyber](<https://devfeed.tech/tags/cyber.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [energy](<https://devfeed.tech/tags/energy.md>), [power](<https://devfeed.tech/tags/power.md>), [power-and-energy](<https://devfeed.tech/tags/power-and-energy.md>), [resilience](<https://devfeed.tech/tags/resilience.md>), [systems](<https://devfeed.tech/tags/systems.md>)

### AI overview

RESILAGENT proposes a three-pillar architecture for power-system resilience: Sensing & Data Intelligence, Resilience Analytics & Reporting, and Incident Response & Recovery. A Cloud-based Orchestrator connects them in a Detect-Simulate-Decide-Sign loop. The design combines agentic AI and multimodal language models for cross-modal detection, real-time digital twins for what-if simulation, and a permissioned blockchain anchor for tamper-evident threat intelligence and CER/NIS2-aligned evidence. It is instantiated in two scenarios: a transmission operator facing coordinated cyber compromise, sabotage, and heatwave, and a distribution operator facing a supply-chain compromise during a storm. An IEEE-bus micro-benchmark measures false-data-injection detection, cyber-physical fusion lift, N-1 contingency screening, and anchor cost; the supplied excerpt gives no benchmark results.

### Source excerpt

arXiv:2611.21480v1 Announce Type: cross Abstract: Electrical Power and Energy Systems face hybrid threats that combine cyber intrusions, physical sabotage and climate-driven stress, and routinely outpace today's siloed monitoring stacks. We present RESILAGENT, a concept architecture organised around three operational pillars, Sensing & Data Intelligence, Resilience Analytics & Reporting, and Incident Response & Recovery, mediated by a Cloud-based Orchestrator and exercised through a closed Detect-Simulate-Decide-Sign loop. The framework fuses agentic artificial intelligence and multimodal large language models for cross-modal detection, real-time digital twins for what-if simulation, and a permissioned blockchain anchor for tamper-evident threat intelligence and CER/NIS2-aligned evidence. We instantiate it on two scenarios, a transmission operator under coordinated cyber compromise, sabotage and heatwave, and a distribution operator under a supply-chain compromise during a storm, and report an IEEE-bus micro-benchmark that quantifies each pillar through false-data-injection attack detection, cyber-physical fusion lift, N-1 contingency screening and anchor cost.

## AdaModalX: Synchronization-Free MFM Inference for Distributed Multimodal Mobile Sensing via Progressive Data Orchestration

DevFeed: [AdaModalX: Synchronization-Free MFM Inference for Distributed Multimodal Mobile Sensing via Progressive Data Orchestration](<https://devfeed.tech/articles/adamodalx-synchronization-free-mfm-inference-for-distributed-multimodal-mobile-sensing-via-progressive-data-orchestration-139057.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20464>)

Author: Xiangrui Xu, Sicong Liu, Zhiwen Yu, Zimu Zhou, Bin Guo

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [applications](<https://devfeed.tech/tags/applications.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [completeness](<https://devfeed.tech/tags/completeness.md>), [contribution](<https://devfeed.tech/tags/contribution.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-orchestration](<https://devfeed.tech/tags/data-orchestration.md>), [device](<https://devfeed.tech/tags/device.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [edge](<https://devfeed.tech/tags/edge.md>), [execution](<https://devfeed.tech/tags/execution.md>), [exit](<https://devfeed.tech/tags/exit.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [features](<https://devfeed.tech/tags/features.md>), [inference](<https://devfeed.tech/tags/inference.md>), [mobile](<https://devfeed.tech/tags/mobile.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [orchestration](<https://devfeed.tech/tags/orchestration.md>), [sensing](<https://devfeed.tech/tags/sensing.md>), [synchronization](<https://devfeed.tech/tags/synchronization.md>)

### AI overview

AdaModalX aims to reduce latency in distributed mobile sensing without waiting for every modality to arrive. A lightweight device-side probe lets reliable samples exit locally while uncertain ones continue uploading. At the edge, the system forms candidates from received data and, when needed, imputed features for delayed inputs; a contribution assessor using frozen MFM encoders estimates their utility and triggers the MFM when a candidate meets its threshold. Across four mobile sensing applications, the authors report up to 26.97% higher accuracy and up to 64.75% lower end-to-end latency. These are reported maximum gains; the excerpt does not specify baselines or per-application results.

### Source excerpt

arXiv:2611.20464v1 Announce Type: new Abstract: Multimodal foundation models (MFMs) support generalizable inference in distributed mobile sensing applications by integrating heterogeneous data streams. However, existing MFM inference pipelines typically require complete and synchronized multimodal inputs, whereas mobile sensing streams arrive asynchronously due to network variability and heterogeneous data volumes across modalities. Waiting for all modalities increases end-to-end latency, while inference with incomplete inputs can reduce accuracy. We argue that multimodal inference should depend on the utility of available information rather than modality completeness. Based on this principle, we propose AdaModalX, a progressive multimodal inference framework that decouples inference execution from modality synchronization through runtime utility-aware orchestration. At each device, importance-aware data uploading uses a lightweight probe distilled from the edge-side MFM to allow reliable samples to exit locally while uncertain samples continue uploading. At the edge, on-demand MFM activation constructs candidates from fully received samples and, when needed, imputed features derived from partially received delayed inputs. A lightweight contribution assessor built on frozen MFM encoders estimates candidate utility and activates the MFM once the candidate meets its corresponding utility threshold. Experiments on four mobile sensing applications show that AdaModalX improves the accuracy-latency trade-off, achieving up to 26.97% accuracy improvement and up to 64.75% end-to-end latency reduction.

## Taxonomy of multi-agent design patterns, anti-patterns, and lessons from manufacturing

DevFeed: [Taxonomy of multi-agent design patterns, anti-patterns, and lessons from manufacturing](<https://devfeed.tech/articles/taxonomy-of-multi-agent-design-patterns-anti-patterns-and-lessons-from-manufacturing-136364.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/taxonomy-of-multi-agent-design-patterns-anti-patterns-and-lessons-from-manufacturing/>)

Author: Malini Sethi

Published: 2026-11-16T21:53:46Z

Content type: article

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [software-architecture](<https://devfeed.tech/topics/software-architecture.md>), [agent to agent communication](<https://devfeed.tech/topics/agent-to-agent-communication.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [supervisor agent pattern](<https://devfeed.tech/topics/supervisor-agent-pattern.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [hierarchical agent systems](<https://devfeed.tech/topics/hierarchical-agent-systems.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [advanced-300](<https://devfeed.tech/tags/advanced-300.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-loops](<https://devfeed.tech/tags/agent-loops.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amazon-bedrock](<https://devfeed.tech/tags/amazon-bedrock.md>), [amazon-bedrock-guardrails](<https://devfeed.tech/tags/amazon-bedrock-guardrails.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [automotive](<https://devfeed.tech/tags/automotive.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [aws](<https://devfeed.tech/tags/aws.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [cost](<https://devfeed.tech/tags/cost.md>), [design](<https://devfeed.tech/tags/design.md>), [design-patterns](<https://devfeed.tech/tags/design-patterns.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-systems](<https://devfeed.tech/tags/distributed-systems.md>), [framework](<https://devfeed.tech/tags/framework.md>), [manufacturing](<https://devfeed.tech/tags/manufacturing.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [taxonomy](<https://devfeed.tech/tags/taxonomy.md>)

### AI overview

The post recommends starting with deterministic Tier 1 workflows and using autonomous multi-agent coordination only when subtasks emerge during the work, since flexibility adds token cost, latency, and debugging burden. It distinguishes pipelines, conditional workflows, routers, and parallel workflows from plan-based orchestration, handoffs, shared conversations, and debate. It advises hard retry and token limits, schema-based handoffs, bounded agents, and approval gates for actions affecting customers or equipment. For manufacturing, it emphasizes p99 latency budgets and deterministic fallbacks, keeping PLC/SCADA/MES control deterministic, validating recommendations against current OT state before approved actuation, processing time-sensitive sensor data at the edge, and structuring agent memory and handoffs around shifts. AWS IoT Greengrass is named for edge monitoring; the supplied text does not detail the promised broader service mapping.

### Source excerpt

Multi-agent coordination is a distributed systems problem, not a model problem. This post gives a field-tested taxonomy of multi-agent design patterns and anti-patterns, a framework for deciding when multiple agents are warranted, and how to build cost-efficient, reliable systems on AWS.

## Dell Exascale Adds 32-Drive Liquid-Cooled Storage for AI Racks, and a 512-Core Air-Cooled PowerEdge M8825

DevFeed: [Dell Exascale Adds 32-Drive Liquid-Cooled Storage for AI Racks, and a 512-Core Air-Cooled PowerEdge M8825](<https://devfeed.tech/articles/dell-exascale-adds-32-drive-liquid-cooled-storage-for-ai-racks-and-a-512-core-air-cooled-poweredge-m8825-136078.md>)

Original publisher: [Read original article](<https://www.storagereview.com/news/dell-exascale-liquid-cooled-storage-poweredge-m8825-openmanage-rack-scale-sc26>)

Author: Harold Fritts

Published: 2026-11-16T15:40:37Z

Content type: news

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [dell](<https://devfeed.tech/topics/dell.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [virtual machines](<https://devfeed.tech/topics/virtual-machines.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-factory](<https://devfeed.tech/tags/ai-factory.md>), [air](<https://devfeed.tech/tags/air.md>), [core](<https://devfeed.tech/tags/core.md>), [data](<https://devfeed.tech/tags/data.md>), [dell](<https://devfeed.tech/tags/dell.md>), [dell-ai-factory](<https://devfeed.tech/tags/dell-ai-factory.md>), [dell-technologies](<https://devfeed.tech/tags/dell-technologies.md>), [drive](<https://devfeed.tech/tags/drive.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-storage](<https://devfeed.tech/tags/enterprise-storage.md>), [gateway](<https://devfeed.tech/tags/gateway.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [liquid](<https://devfeed.tech/tags/liquid.md>), [news](<https://devfeed.tech/tags/news.md>), [nvme](<https://devfeed.tech/tags/nvme.md>), [platform](<https://devfeed.tech/tags/platform.md>), [poweredge](<https://devfeed.tech/tags/poweredge.md>), [software](<https://devfeed.tech/tags/software.md>), [software-defined-storage](<https://devfeed.tech/tags/software-defined-storage.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

Dell's SC26 updates add two Dell Exascale storage options for AI racks: a fanless 2-OU direct-liquid-cooled system with up to 32 NVMe drives, and a 1U air-cooled system with up to 20. Dell expects both in early 2027; block support for the liquid-cooled platform is planned for the first half of 2027. The 1U system's claimed 67% drive-density gain compares with prior 2U Exascale hardware. Other announcements include an Agentic Data Gateway that centrally governs agent access to data, tools, and models inside the customer environment; OpenManage Rack Scale, which manages up to 100,000 devices; and Fabric Manager for network-fabric automation. Dell's performance and efficiency figures are vendor or lab results. The air-cooled, two-socket PowerEdge M8825 offers up to 512 cores, has preliminary specifications, and is targeted for the first half of 2027. The excerpt also describes AMD GPU configurations, validated AI solutions, and services, with availability varying by offering.

### Source excerpt

Dell Technologies has announced a round of Dell AI Factory updates at SC26, led by two new hardware options for Dell Exascale, its software-defined storage for AI racks: a 2-OU direct-liquid-cooled platform with up to 32 NVMe drives and a 1U air-cooled platform with up to 20. The news also covers an Agentic Data Gateway The post Dell Exascale Adds 32-Drive Liquid-Cooled Storage for AI Racks, and a 512-Core Air-Cooled PowerEdge M8825 appeared first on StorageReview.com.

## Tabular Population Priors as Structured Retrieval for EHR Medication Set Prediction

DevFeed: [Tabular Population Priors as Structured Retrieval for EHR Medication Set Prediction](<https://devfeed.tech/articles/tabular-population-priors-as-structured-retrieval-for-ehr-medication-set-prediction-135057.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17341>)

Author: Animesh Agarwal, Meysam Ghaffari, Nina Fatehi, Carlos Morato

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [Structured-data](<https://devfeed.tech/topics/structured-data.md>), [Amazon Bedrock Knowledge Bases](<https://devfeed.tech/topics/amazon-bedrock-knowledge-bases.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [aggregate](<https://devfeed.tech/tags/aggregate.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [set](<https://devfeed.tech/tags/set.md>), [structured](<https://devfeed.tech/tags/structured.md>)

### AI overview

For fixed-vocabulary EHR medication prediction, a population prior built from sparse condition-medication co-occurrences may provide more task-aligned retrieval than retrieving similar patient records. In a PACERAG pipeline, replacing only the patient retriever with ranked medication-class candidates raised F1 from 0.303 to 0.338 and Jaccard from 0.218 to 0.237 across 1,004 held-out MIMIC-IV admissions. The swap required no additional LLM calls or inference-time patient index. A non-LLM top-k baseline reached F1 0.248. These results are limited to the reported evaluation.

### Source excerpt

arXiv:2611.17341v1 Announce Type: new Abstract: Medication prediction from electronic health records is a fixed vocabulary structured prediction problem, yet recent clinical RAG systems often treat it like open ended case based reasoning: retrieve similar patient admissions and ask an LLM to infer a medication set. This retrieval object is indirect the model receives retrieved patient records while the target is a discrete set of medication class labels. We propose a more task aligned retrieval object: a tabular population prior ranked medication class candidates from a sparse condition medication co occurrence table estimated from training admissions. Using PACERAG as a fixed downstream pipeline and replacing only its patient retriever, this drop in swap improves F1 from 0.303 to 0.338 (+11.5% relative) and Jaccard from 0.218 to 0.237 on 1,004 heldout MIMIC IV admissions, with no additional LLM calls and no inference time patient index. A non LLM top k baseline reaches F1=0.248, confirming the prior is predictive before LLM integration. Aggregate population statistics can provide a more task aligned, efficient, and auditable retrieval source than patient level nearest neighbors for fixed vocabulary EHR set prediction.

## How to Make Enterprise RAG Systems More Reliable

DevFeed: [How to Make Enterprise RAG Systems More Reliable](<https://devfeed.tech/articles/how-to-make-enterprise-rag-systems-more-reliable-134596.md>)

Original publisher: [Read original article](<https://hackernoon.com/how-to-make-enterprise-rag-systems-more-reliable?source=rss>)

Author: Vinuraj Devaraj

Published: 2026-11-15T21:09:32Z

Content type: opinion

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Access Control](<https://devfeed.tech/topics/access-control.md>), [enterprise data protection](<https://devfeed.tech/topics/enterprise-data-protection.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [Digital Trust](<https://devfeed.tech/topics/digital-trust.md>)

Tags: [access-control](<https://devfeed.tech/tags/access-control.md>), [ai-security](<https://devfeed.tech/tags/ai-security.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data-freshness](<https://devfeed.tech/tags/data-freshness.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-ai](<https://devfeed.tech/tags/enterprise-ai.md>), [enterprise-rag](<https://devfeed.tech/tags/enterprise-rag.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [graph-rag](<https://devfeed.tech/tags/graph-rag.md>), [permission-aware-retrieval](<https://devfeed.tech/tags/permission-aware-retrieval.md>), [rag](<https://devfeed.tech/tags/rag.md>), [rag-architecture](<https://devfeed.tech/tags/rag-architecture.md>), [rag-observability](<https://devfeed.tech/tags/rag-observability.md>), [rag-systems](<https://devfeed.tech/tags/rag-systems.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [vector-search-security](<https://devfeed.tech/tags/vector-search-security.md>)

### AI overview

Enterprise RAG reliability depends on operating the document collection well: enforce permissions during retrieval, track index freshness, evaluate live traffic continuously, and measure cost by pipeline stage. Post-retrieval permission filters can leak ranking signals and waste compute; pre-filtering may reduce recall for selective queries, while separate tenant indexes add operational overhead and limit cross-domain search. Index upserts, change-data-capture, and incremental re-embedding are available, but shared consistency models and benchmarks for continuously changing corpora are lacking. RAGAS, ARES, RAGTruth, TruLens, LangSmith, and Arize Phoenix provide evaluation capabilities, though continuous scoring remains uncommon. GraphRAG, LightRAG, and HippoRAG support graph-oriented questions, but graph construction can be far more costly than vector indexing, and large-scale update costs and quality-versus-cost tradeoffs remain poorly measured. The author recommends piloting GraphRAG for global, multi-hop questions with an explicit budget.

### Source excerpt

Enterprise RAG needs more than accurate retrieval. Explore four critical gaps in access control, data freshness, continuous evaluation, and cost.

## What If UI Components Were Built Like Modular Phones?

DevFeed: [What If UI Components Were Built Like Modular Phones?](<https://devfeed.tech/articles/what-if-ui-components-were-built-like-modular-phones-134643.md>)

Original publisher: [Read original article](<https://www.amitmerchant.com/what-if-ui-components-were-built-like-modular-phones/>)

Author: Amit Merchant

Published: 2026-11-15T00:00:00Z

Content type: tutorial

Language: en

Sources: [Amit Merchant](<https://devfeed.tech/sources/amit-merchant.md>)

Topics: [Code Modularity](<https://devfeed.tech/topics/code-modularity.md>), [User interface design](<https://devfeed.tech/topics/ui-design.md>), [React](<https://devfeed.tech/topics/react.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [abstractions](<https://devfeed.tech/tags/abstractions.md>), [actions](<https://devfeed.tech/tags/actions.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [array](<https://devfeed.tech/tags/array.md>), [await](<https://devfeed.tech/tags/await.md>), [battery](<https://devfeed.tech/tags/battery.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [behaviors](<https://devfeed.tech/tags/behaviors.md>), [behaviour](<https://devfeed.tech/tags/behaviour.md>), [components](<https://devfeed.tech/tags/components.md>), [modular](<https://devfeed.tech/tags/modular.md>), [modules](<https://devfeed.tech/tags/modules.md>), [phone](<https://devfeed.tech/tags/phone.md>), [phones](<https://devfeed.tech/tags/phones.md>), [ui](<https://devfeed.tech/tags/ui.md>), [ui-components](<https://devfeed.tech/tags/ui-components.md>)

### AI overview

Vercel's UI SDK builds custom UI components by composing reusable interaction behaviours such as popup positioning, focus management, keyboard navigation, and selection, rather than relying only on predefined widgets. Primitives can be mapped to framework elements and reused; the article's example adds typeahead by composing another behaviour. The SDK also offers unstyled Select, Dialog, Tabs, Combobox, and Date Picker components. Its Agent Bridge exposes interactive components' state and available actions in a JSON snapshot, then lets an agent invoke those actions through `act()`, giving agents an explicit interface beyond interpreting the DOM. The excerpt presents this as a way to retain control over component implementation while making interactions accessible to agents.

### Source excerpt

Remember the idea of modular phones? Instead of buying a phone with everything fixed into it, you could assemble one using individual modules.

## Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot

DevFeed: [Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot](<https://devfeed.tech/articles/cost-optimizing-assemblyai-speech-to-text-inference-on-aws-with-amazon-ec2-spot-132652.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/machine-learning/cost-optimizing-assemblyai-speech-to-text-inference-on-aws-with-amazon-ec2-spot/>)

Author: Belair Albahagwi

Published: 2026-11-13T19:35:35Z

Content type: news

Language: en

Sources: [Artificial Intelligence](<https://devfeed.tech/sources/artificial-intelligence.md>)

Topics: [asr](<https://devfeed.tech/topics/asr.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Amazon EC2](<https://devfeed.tech/topics/amazon-ec2.md>), [cloud cost management](<https://devfeed.tech/topics/cloud-cost-management.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Speech Processing](<https://devfeed.tech/topics/speech-processing.md>), [AI Architecture](<https://devfeed.tech/topics/ai-architecture.md>)

Tags: [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [amazon-sqs](<https://devfeed.tech/tags/amazon-sqs.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [assemblyai](<https://devfeed.tech/tags/assemblyai.md>), [aws](<https://devfeed.tech/tags/aws.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-savings](<https://devfeed.tech/tags/cost-savings.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [ec2](<https://devfeed.tech/tags/ec2.md>), [ecs](<https://devfeed.tech/tags/ecs.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [regions](<https://devfeed.tech/tags/regions.md>), [retry](<https://devfeed.tech/tags/retry.md>), [scale](<https://devfeed.tech/tags/scale.md>), [speech](<https://devfeed.tech/tags/speech.md>), [speech-to-text](<https://devfeed.tech/tags/speech-to-text.md>), [spot](<https://devfeed.tech/tags/spot.md>), [sqs](<https://devfeed.tech/tags/sqs.md>)

### AI overview

AssemblyAI reports 70% lower inference compute costs than On-Demand pricing using GPU Spot Instances, while maintaining SLA commitments. Its Amazon ECS clusters combine an always-on On-Demand baseline with diversified Spot capacity across g4dn, g5, and g6 GPU families and Availability Zones. Amazon SQS decouples job submission from execution: tasks finish work within Spot's two-minute interruption notice or return unfinished jobs to the queue for another worker to retry. Custom images with model artifacts and containers take about two minutes to create, reducing recovery time. Independent deployments in each AWS Region keep audio within required geographic boundaries. The article says compute demand grew nearly fourfold year over year; interruptions may add processing time, and exhausted retries can be routed to a dead-letter queue for investigation.

### Source excerpt

Learn how AssemblyAI runs cost-optimized speech-to-text inference at scale on AWS, using Amazon ECS, Amazon EC2 Spot GPU instances, and an Amazon SQS-based retry architecture to achieve 70% cost savings while maintaining SLA commitments across European Regions.

[Next page](<https://devfeed.tech/topics/ai-architecture.md?cursor=WyIyMDI2LTExLTEzVDE5OjM1OjM1KzAwOjAwIiwgImNkNWI3OTQ3LWQ1MjQtNGE1OC05N2Y0LTE2MTc4ZDliMjEwMSJd>)