# Modal

We share some insights about serverless computing, and the problems that we solve along the way.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## How Abundant designed efficient rollout architecture with Modal Sandboxes

DevFeed: [How Abundant designed efficient rollout architecture with Modal Sandboxes](<https://devfeed.tech/articles/how-abundant-designed-efficient-rollout-architecture-with-modal-sandboxes-145277.md>)

Original publisher: [Read original article](<https://modal.com/blog/abundant-efficient-rollout-architecture-with-modal-sandboxes>)

Author: Modal

Published: 2026-11-19T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [anthropic](<https://devfeed.tech/topics/anthropic.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-labs](<https://devfeed.tech/tags/ai-labs.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [api](<https://devfeed.tech/tags/api.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [awareness](<https://devfeed.tech/tags/awareness.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [environments](<https://devfeed.tech/tags/environments.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [labs](<https://devfeed.tech/tags/labs.md>), [modal](<https://devfeed.tech/tags/modal.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sandboxes](<https://devfeed.tech/tags/sandboxes.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Abundant runs reinforcement-learning task environments in Modal Sandboxes, with Modal Functions handling scheduling and cleanup. Its rollout monitor runs inside each Sandbox, sending a heartbeat every 30 seconds along with live events and the final result; after three failed connection attempts, the control plane terminates a quiet Sandbox. Modal's CPU and memory bursting lets Abundant request resources closer to average use while accommodating peaks when machine capacity allows, reducing idle compute and easing per-task sizing. To limit evaluation awareness, Abundant removes task instructions before work begins and uses a reviewer agent to flag suspicious traces. The excerpt says the architecture took about two weeks to implement and, as of November 2026, supported over two million Sandboxes monthly. Modal also powered more than 1,300 hours of agent research across nearly 500 trials for the initial Frontier RSI results.

### Source excerpt

Abundant builds RL environments that run on Modal Sandboxes, powering training for frontier labs.

## Building new research tools for Maya epigraphy on Modal

DevFeed: [Building new research tools for Maya epigraphy on Modal](<https://devfeed.tech/articles/building-new-research-tools-for-maya-epigraphy-on-modal-142067.md>)

Original publisher: [Read original article](<https://modal.com/blog/scaling-intelligence-asya>)

Author: Peri Langlois

Published: 2026-11-18T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Advanced AI workflows](<https://devfeed.tech/topics/advanced-ai-workflows.md>), [3D](<https://devfeed.tech/topics/3d.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [3d-model](<https://devfeed.tech/tags/3d-model.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-compute](<https://devfeed.tech/tags/cloud-compute.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [collective](<https://devfeed.tech/tags/collective.md>), [culture](<https://devfeed.tech/tags/culture.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [drawing](<https://devfeed.tech/tags/drawing.md>), [maya](<https://devfeed.tech/tags/maya.md>), [modal](<https://devfeed.tech/tags/modal.md>), [research](<https://devfeed.tech/tags/research.md>), [stanford](<https://devfeed.tech/tags/stanford.md>), [tools](<https://devfeed.tech/tags/tools.md>), [writing](<https://devfeed.tech/tags/writing.md>)

### AI overview

Miguel Novelo combines family and scholarly knowledge with multiple AI models to investigate Maya glyphs, checking model-generated connections against sources because general-purpose models alone lack the cultural and linguistic understanding he needs. Using Modal, he built a pipeline that turns 2D glyph images into 3D models from roughly 100 syllabic signs; this also lets epigraphers generate, rotate, and relight carving models from photographs, reducing work that once took about a week to a fraction of that time. His installation invites visitors to draw in sand, transforming their marks into glyphs in a growing projected codex. He plans to refine the process with collected drawings and ultimately digitize carvings in 3D to study whether carving depth and style reveal patterns that flat images miss. The excerpt also notes concerns about applying AI to Indigenous writing systems; Novelo frames his work around transmission and collaboration while acknowledging the need for respect and repair.

### Source excerpt

Miguel Novelo grew up surrounded by Maya glyphs carved into stone across Mexico's Yucatán peninsula, printed in books, and painted on ceramics. For centuries, Maya scribes used this writing system to record histories, trace the movements of the heavens, and keep calendars. For most of his life, he saw them only as drawings. "I never thought I could understand them," he says. Novelo is an artist and lecturer at Stanford, where he teaches a course called World Building: Video, Sound, and Space.

## Scaling Intelligence: A landmark commission and exhibition

DevFeed: [Scaling Intelligence: A landmark commission and exhibition](<https://devfeed.tech/articles/scaling-intelligence-a-landmark-commission-and-exhibition-133572.md>)

Original publisher: [Read original article](<https://modal.com/blog/scaling-intelligence>)

Author: Peri Langlois

Published: 2026-11-13T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine Intelligence](<https://devfeed.tech/topics/machine-intelligence.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-generated-images](<https://devfeed.tech/tags/ai-generated-images.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [arm](<https://devfeed.tech/tags/arm.md>), [art](<https://devfeed.tech/tags/art.md>), [budget](<https://devfeed.tech/tags/budget.md>), [cats](<https://devfeed.tech/tags/cats.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [compute](<https://devfeed.tech/tags/compute.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [exhibition](<https://devfeed.tech/tags/exhibition.md>), [intersection](<https://devfeed.tech/tags/intersection.md>), [machine-intelligence](<https://devfeed.tech/tags/machine-intelligence.md>), [nonprofit](<https://devfeed.tech/tags/nonprofit.md>), [san-francisco](<https://devfeed.tech/tags/san-francisco.md>), [technology](<https://devfeed.tech/tags/technology.md>)

### AI overview

Scaling Intelligence presents 11 artworks that use AI models, agents, sensors, and cloud compute as creative materials, often making their limits and effects part of the work. A satellite-image loop searches endlessly for a nonexistent Arctic island; creek sensors feed a model that builds a changing state for an LED installation; and a choir of language models learns from visitors but falls silent when its compute budget runs out. Other works let visitors create glyphs with a model trained on ancient Maya writing, retrain a conversational agent nightly on public interactions, or observe groups of agents forming rules and institutions. Across the exhibition, AI translates otherwise imperceptible signals, responds to participation, or runs open-ended processes, while several pieces foreground dependence on people, data, or finite computation.

### Source excerpt

Last week, Modal and Gray Area, the San Francisco nonprofit that has supported artists working at the intersection of art and technology for nearly two decades, opened *Scaling Intelligence*, an exhibition of 11 new artworks made with machine intelligence and cloud-scale compute. Over the past few months, we've worked closely with the artists, supporting them with engineering help and compute credits as they built everything from scratch.

## Modal becomes a gVisor maintainer

DevFeed: [Modal becomes a gVisor maintainer](<https://devfeed.tech/articles/modal-becomes-a-gvisor-maintainer-129819.md>)

Original publisher: [Read original article](<https://modal.com/blog/modal-gvisor-maintainer>)

Author: Luis Capelo

Published: 2026-11-11T12:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [virtualization](<https://devfeed.tech/topics/virtualization.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [cgroups](<https://devfeed.tech/topics/cgroups.md>), [GPU rendering](<https://devfeed.tech/topics/gpu-rendering.md>), [Cloud Gaming](<https://devfeed.tech/topics/cloud-gaming.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding-agents](<https://devfeed.tech/tags/ai-coding-agents.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [aws](<https://devfeed.tech/tags/aws.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [blackwell-gpu](<https://devfeed.tech/tags/blackwell-gpu.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-gaming](<https://devfeed.tech/tags/cloud-gaming.md>), [cloud-native-computing-foundation](<https://devfeed.tech/tags/cloud-native-computing-foundation.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [containers](<https://devfeed.tech/tags/containers.md>), [gvisor](<https://devfeed.tech/tags/gvisor.md>), [modal](<https://devfeed.tech/tags/modal.md>)

### AI overview

Modal is joining gVisor's maintainers as the project moves to CNCF governance, building on its use of gVisor for more than 40 million containers a day. gVisor places a Go-based application kernel (the Sentry) between container workloads and the host kernel, then layers Linux isolation controls around it; unlike hardware virtualization, it can run on Linux machines without nested virtualization. Modal cites multi-tenant isolation, GPU access through nvproxy, roughly 25 ms startup with 12 MB memory overhead, checkpoint and restore, and portability across clouds and bare-metal systems as reasons for choosing it. Its upstream work includes GPU driver and workload support, an RDMA proxy for multi-node training, snapshots, and filesystem and shared-memory performance improvements. The article reports about 99% of native bus bandwidth in NCCL all-reduce benchmarks for RDMA and describes multi-GPU snapshots and GPUDirect Storage as ongoing work. Modal's vision is broader accelerator support, portable state, and faster I/O under neutral, multi-vendor governance.

### Source excerpt

Modal runs upwards of 40 million gVisor containers every day. Here's why we bet on it, what we've built into it, and where we want to take it next.

## How Snorkel runs realistic agent simulations with Modal

DevFeed: [How Snorkel runs realistic agent simulations with Modal](<https://devfeed.tech/articles/how-snorkel-runs-realistic-agent-simulations-with-modal-121806.md>)

Original publisher: [Read original article](<https://modal.com/blog/how-snorkel-runs-realistic-agent-simulations-with-modal>)

Author: Modal

Published: 2026-11-06T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-data](<https://devfeed.tech/tags/ai-data.md>), [ai-labs](<https://devfeed.tech/tags/ai-labs.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [modal](<https://devfeed.tech/tags/modal.md>), [models](<https://devfeed.tech/tags/models.md>), [open-weight-models](<https://devfeed.tech/tags/open-weight-models.md>), [sandboxes](<https://devfeed.tech/tags/sandboxes.md>), [simulations](<https://devfeed.tech/tags/simulations.md>), [snorkel](<https://devfeed.tech/tags/snorkel.md>)

### AI overview

Snorkel uses Modal to run millions of agent simulations monthly, with GPU and CPU sandboxes that reproduce varied benchmark environments, including multi-service systems and GPU workloads. Snorkel says Modal was the only sandbox provider it found able to meet its GPU needs at scale; using one provider for CPU and GPU simplified infrastructure and cost tracking. High concurrency supports time-sensitive evaluations that customers may want within a day. Modal's CPU and memory bursting lets sandboxes expand resources during spikes; Snorkel reports fewer resource-limit failures and about 20% lower costs, though bursting is still used by only 10-15% of its sandboxes. For inference, Snorkel uses Modal Shared and Dedicated Endpoints to serve off-the-shelf and customer-fine-tuned open-weight models. It plans to evaluate Modal for training as well.

### Source excerpt

Modal powers inference for the open-weight models that Snorkel is evaluating as well as GPU and CPU sandboxes for millions of simulations per month.

## Launching a trillion-parameter model at Sakana AI

DevFeed: [Launching a trillion-parameter model at Sakana AI](<https://devfeed.tech/articles/launching-a-trillion-parameter-model-at-sakana-ai-105637.md>)

Original publisher: [Read original article](<https://modal.com/blog/sakana-case-study>)

Author: Peri Langlois; Saatwik Nagpal

Published: 2026-10-23T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [function calling](<https://devfeed.tech/topics/function-calling.md>)

Tags: [a-b-testing](<https://devfeed.tech/tags/a-b-testing.md>), [acceptance-rate](<https://devfeed.tech/tags/acceptance-rate.md>), [ai](<https://devfeed.tech/tags/ai.md>), [api](<https://devfeed.tech/tags/api.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language](<https://devfeed.tech/tags/language.md>), [memory](<https://devfeed.tech/tags/memory.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [model](<https://devfeed.tech/tags/model.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [precision](<https://devfeed.tech/tags/precision.md>), [quality](<https://devfeed.tech/tags/quality.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [sakana](<https://devfeed.tech/tags/sakana.md>), [tool](<https://devfeed.tech/tags/tool.md>), [tool-calling](<https://devfeed.tech/tags/tool-calling.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Modal and Sakana AI launched Sakana Namazu, a roughly 1T-parameter Japanese business model, using a serving stack tuned for quality, speed, and elastic capacity. They quantized routed experts to NVFP4 while keeping attention, routing, and shared experts in BF16, and calibrated on Japanese, domain-representative conversations; the text reports Japanese instruction following matched Sakana's production checkpoint within benchmark noise. A custom speculative draft model trained on raw production token IDs improved token acceptance and reduced decode latency; a later DFlash2 drafter trained on about 1M real requests raised average accepted draft length from 3.51 to 5.01 and lowered p99 latency from 68 to 52 seconds in a held-out replay. Further work removed CPU-GPU synchronization stalls, accelerated expert kernels, cached request setup, fixed a rare cross-GPU verification disagreement, and enabled speculative JSON-schema generation. A KV-aware router reduced queueing and tail latency. Modal's on-demand GPU pool let the API scale for launch without a long-term capacity reservation.

### Source excerpt

Deploying a \~1-trillion parameter Mixture-of-Experts (MoE) model for API inference is a classic systems challenge. At full FP16 or BF16 precision, the memory footprint requires very large clusters. Standard post-training quantization (PTQ) recipes shrink that footprint but can degrade model quality, especially for non-English models, where the usual quantization calibration datasets don't capture language register or tool-calling behavior.

## Introducing our Training API

DevFeed: [Introducing our Training API](<https://devfeed.tech/articles/introducing-our-training-api-96406.md>)

Original publisher: [Read original article](<https://modal.com/blog/introducing-training-api>)

Author: Micah Yong

Published: 2026-10-22T12:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [lora](<https://devfeed.tech/topics/lora.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [agents](<https://devfeed.tech/tags/agents.md>), [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [async](<https://devfeed.tech/tags/async.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [backend](<https://devfeed.tech/tags/backend.md>), [code](<https://devfeed.tech/tags/code.md>), [command](<https://devfeed.tech/tags/command.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [models](<https://devfeed.tech/tags/models.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [run](<https://devfeed.tech/tags/run.md>), [server](<https://devfeed.tech/tags/server.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Spindle is an open-source, Tinker-compatible training server for Modal that lets teams run their own Python training loops and pay for GPU time rather than tokens. It sits between Modal Dojo, which manages the loop, and raw Modal cluster primitives, which give teams control down to the cluster. Spindle handles distributed training and supports full-parameter training and multi-LoRA across open models. The authors report matching Tinker reward results within 0.01 in a search-agent rerun and describe a long-context run that resumed automatically on preemptible GPUs. Multi-LoRA lets clients share trainer GPUs: in one 27B-model example, four clients kept the trainer about 88% busy and reduced per-run cost, while adding more than four caused thrashing and lower per-client throughput. Cost and performance depend on keeping GPUs busy; the article says Spindle is still in beta.

### Source excerpt

Spindle is an open-source API for training and fine-tuning open models -- deploy a server with a single command and run any Tinker-compatible training code against it.

## Get started with LLM training on Modal

DevFeed: [Get started with LLM training on Modal](<https://devfeed.tech/articles/get-started-with-llm-training-on-modal-93939.md>)

Original publisher: [Read original article](<https://modal.com/blog/get-started-with-llm-training-on-modal>)

Author: Andrew Hinh

Published: 2026-10-21T12:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [agents](<https://devfeed.tech/tags/agents.md>), [api](<https://devfeed.tech/tags/api.md>), [axiom](<https://devfeed.tech/tags/axiom.md>), [cli](<https://devfeed.tech/tags/cli.md>), [config](<https://devfeed.tech/tags/config.md>), [custom](<https://devfeed.tech/tags/custom.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [debug](<https://devfeed.tech/tags/debug.md>), [dojo](<https://devfeed.tech/tags/dojo.md>), [environment](<https://devfeed.tech/tags/environment.md>), [library](<https://devfeed.tech/tags/library.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-training](<https://devfeed.tech/tags/llm-training.md>), [modal](<https://devfeed.tech/tags/modal.md>), [observability](<https://devfeed.tech/tags/observability.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Modal Dojo is a library for post-training LLMs on Modal, built on open-source frameworks Miles and Slime. The article describes a workflow: choose and evaluate a supported or custom model, load training data from Hugging Face, a Harbor environment, or production harness traces, then launch a model-specific recipe that sets hardware, parallelism, rollout size, environment, and other parameters. Modal provides auto-optimized recipes that users can adjust. Its dashboard exposes loss and reward curves, framework metrics, rollout trajectories, substep profiling, and container logs for monitoring and debugging. The CLI provides run status, metadata, logs, and traces for agents. The authors say the library has been validated across model sizes from half a billion to multiple trillions of parameters; the provided example uses Qwen3 and a custom GSM8K reward function. The text also reports customer use at Axiom Bio and Afterquery.

### Source excerpt

Reinforcement learning for LLMs is hard, so we built Modal Dojo -- a library that gives you the building blocks, tuned defaults, and observability to get started with production-grade LLM training.

## Serverless routing made sticky

DevFeed: [Serverless routing made sticky](<https://devfeed.tech/articles/serverless-routing-made-sticky-91153.md>)

Original publisher: [Read original article](<https://modal.com/blog/serverless-routing-made-sticky>)

Author: Siwon Park; Kartik Sinha; Kathir Meyyappan; Daniel Shaar

Published: 2026-10-20T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Serverless](<https://devfeed.tech/topics/serverless.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>)

Tags: [affinity](<https://devfeed.tech/tags/affinity.md>), [agents](<https://devfeed.tech/tags/agents.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [games](<https://devfeed.tech/tags/games.md>), [modal](<https://devfeed.tech/tags/modal.md>), [multiplayer](<https://devfeed.tech/tags/multiplayer.md>), [routing](<https://devfeed.tech/tags/routing.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [servers](<https://devfeed.tech/tags/servers.md>), [sessions](<https://devfeed.tech/tags/sessions.md>), [sticky](<https://devfeed.tech/tags/sticky.md>), [sticky-sessions](<https://devfeed.tech/tags/sticky-sessions.md>), [voice](<https://devfeed.tech/tags/voice.md>)

### AI overview

Modal Sticky Sessions keep each session's requests on one container while allowing the fleet to autoscale with active sessions. A session starts through the SDK or HTTP API and returns a signed token clients can share to access the same state. Unlike WebSocket connections tied to individual Function inputs, which may reconnect to another container, or best-effort affinity routing, sessions provide stronger routing guarantees. Modal's proxies validate token-encoded routing metadata; load balancers probe containers for available concurrency, and sessions queue when a container reaches its configured session capacity. The autoscaler uses active sessions and queued demand to scale. The article describes voice agents, shared chat rooms, and multiplayer games as use cases, while noting that sessions keep their state on the assigned container.

### Source excerpt

Voice agents, multiplayer games, and other workloads can now build on Sticky Sessions. Achieve strong affinity without losing autoscaling. Here's how we built this on Modal Servers.

## NVIDIA Vera CPUs double throughput for Modal Sandboxes

DevFeed: [NVIDIA Vera CPUs double throughput for Modal Sandboxes](<https://devfeed.tech/articles/nvidia-vera-cpus-double-throughput-for-modal-sandboxes-90282.md>)

Original publisher: [Read original article](<https://modal.com/blog/nvidia-vera-sandboxes>)

Author: Erik Dunteman

Published: 2026-10-19T14:01:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [NVIDIA Vera](<https://devfeed.tech/topics/nvidia-vera.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [concurrently](<https://devfeed.tech/tags/concurrently.md>), [container](<https://devfeed.tech/tags/container.md>), [cpu-throttling](<https://devfeed.tech/tags/cpu-throttling.md>), [cpu-time](<https://devfeed.tech/tags/cpu-time.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [docker](<https://devfeed.tech/tags/docker.md>), [docker-containers](<https://devfeed.tech/tags/docker-containers.md>), [emulation](<https://devfeed.tech/tags/emulation.md>), [experience](<https://devfeed.tech/tags/experience.md>), [latency](<https://devfeed.tech/tags/latency.md>), [modal](<https://devfeed.tech/tags/modal.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-vera](<https://devfeed.tech/tags/nvidia-vera.md>)

### AI overview

Modal benchmarked NVIDIA Vera CPUs on agentic sandbox workloads and reports up to twice the task throughput of an equivalent x86 server under load, while maintaining comparable or lower task latency. The article attributes the results to Vera's ability to sustain performance under contention and discusses implications for container density and sandbox workloads.

### Source excerpt

We benchmarked NVIDIA's new Arm-based Vera CPU on agentic Sandbox workloads. Under heavy load, it completes far more tasks than the x86 and Graviton alternatives, without sacrificing latency.

## How Legora runs 2,000 parallel agent evals on Modal Sandboxes

DevFeed: [How Legora runs 2,000 parallel agent evals on Modal Sandboxes](<https://devfeed.tech/articles/how-legora-runs-2-000-parallel-agent-evals-on-modal-sandboxes-100766.md>)

Original publisher: [Read original article](<https://modal.com/blog/how-legora-runs-thousand-scale-parallel-agent-evals-on-modal-sandboxes>)

Author: Modal

Published: 2026-10-19T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [out-of-process](<https://devfeed.tech/topics/out-of-process.md>), [long-running](<https://devfeed.tech/topics/long-running.md>), [dynamic test splitting](<https://devfeed.tech/topics/dynamic-test-splitting.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-evals](<https://devfeed.tech/tags/agent-evals.md>), [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [application](<https://devfeed.tech/tags/application.md>), [commit](<https://devfeed.tech/tags/commit.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evals](<https://devfeed.tech/tags/evals.md>), [inside](<https://devfeed.tech/tags/inside.md>), [legal](<https://devfeed.tech/tags/legal.md>), [legora](<https://devfeed.tech/tags/legora.md>), [modal](<https://devfeed.tech/tags/modal.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [pinned](<https://devfeed.tech/tags/pinned.md>), [run](<https://devfeed.tech/tags/run.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>), [sandboxes](<https://devfeed.tech/tags/sandboxes.md>), [separate](<https://devfeed.tech/tags/separate.md>), [test](<https://devfeed.tech/tags/test.md>), [works](<https://devfeed.tech/tags/works.md>)

### AI overview

Legora runs 2,000 or more concurrent evaluations by booting its full application at a pinned commit in a separate Modal Sandbox for each run, so its autonomous legal agent can be tested in realistic projects with thousands of documents. Modal was selected after evaluating other cloud-managed providers for scalability, low startup latency, and support for complex environments. Modal Volumes provide shared document collections and snapshots of documents, database state, permissions, and blobs, allowing repeated runs to start consistently while deduplicating setup. An hourly image-build and Sandbox prewarming pipeline reduced reset-to-commit work from rebuilding the full application to applying only the latest delta. Legora also moved from gVisor to VM Sandboxes, where a full Linux kernel enabled Docker to work as on a normal host and simplified setup. The platform supports BAR benchmarking across model quality, cost, and latency; Legora reports a 30% agent-cost reduction from smarter compaction and custom tool search.

### Source excerpt

To test a legal agent that works for hours across thousands of documents, Legora boots its entire application, pinned to a specific commit, inside a separate Modal Sandbox for every eval run.

## Building fast filesystems for VMs

DevFeed: [Building fast filesystems for VMs](<https://devfeed.tech/articles/building-fast-filesystems-for-vms-88774.md>)

Original publisher: [Read original article](<https://modal.com/blog/vm-sandbox-filesystem>)

Author: Amit Prasad

Published: 2026-10-16T12:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Ext4](<https://devfeed.tech/topics/ext4.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>)

Tags: [block](<https://devfeed.tech/tags/block.md>), [containers](<https://devfeed.tech/tags/containers.md>), [copy-on-write](<https://devfeed.tech/tags/copy-on-write.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [docker](<https://devfeed.tech/tags/docker.md>), [filesystems](<https://devfeed.tech/tags/filesystems.md>), [vm](<https://devfeed.tech/tags/vm.md>)

### AI overview

Modal describes a VM sandbox filesystem architecture that uses virtio-blk and guest-side ext4 metadata to avoid the round trips associated with virtio-fs. ImageFS-blk serves image blocks lazily, preserves sub-second cold starts, and supports block-level copy-on-write. The article reports a bun install benchmark improving from 12.5 seconds to 1.07 seconds.

### Source excerpt

How we made VM Sandbox filesystems fast by speaking in blocks instead of files.

## Augment Your LiveKit Agent with Modal GPUs

DevFeed: [Augment Your LiveKit Agent with Modal GPUs](<https://devfeed.tech/articles/augment-your-livekit-agent-with-modal-gpus-97784.md>)

Original publisher: [Read original article](<https://modal.com/blog/livekit-modal>)

Author: Andrew Hinh

Published: 2026-10-15T12:00:00Z

Content type: tutorial

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [livekit](<https://devfeed.tech/topics/livekit.md>), [sessions](<https://devfeed.tech/topics/sessions.md>)

Tags: [affinity](<https://devfeed.tech/tags/affinity.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-sdk](<https://devfeed.tech/tags/agent-sdk.md>), [agents-sdk](<https://devfeed.tech/tags/agents-sdk.md>), [apis](<https://devfeed.tech/tags/apis.md>), [applications](<https://devfeed.tech/tags/applications.md>), [audio](<https://devfeed.tech/tags/audio.md>), [avatar](<https://devfeed.tech/tags/avatar.md>), [container](<https://devfeed.tech/tags/container.md>), [custom](<https://devfeed.tech/tags/custom.md>), [docs](<https://devfeed.tech/tags/docs.md>), [expression](<https://devfeed.tech/tags/expression.md>), [frameworks-and-tools](<https://devfeed.tech/tags/frameworks-and-tools.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [livekit](<https://devfeed.tech/tags/livekit.md>), [modal](<https://devfeed.tech/tags/modal.md>), [models](<https://devfeed.tech/tags/models.md>), [open-weights](<https://devfeed.tech/tags/open-weights.md>), [sessions](<https://devfeed.tech/tags/sessions.md>), [sticky-sessions](<https://devfeed.tech/tags/sticky-sessions.md>)

### AI overview

The excerpt describes running BitHuman's open-weights video avatar with Qwen3-TTS through LiveKit Agents on Modal GPUs. LiveKit's BitHuman integration uses a self-hosted GPU container with a `/launch` endpoint that starts the avatar and joins a LiveKit room. Because the endpoint returns as soon as the avatar joins, the article uses Modal Sticky Sessions to keep the container available across client-side session start and end, and let WebSocket connections reconnect to the same container. The agent starts a session when a participant joins, passes its token to `/launch`, and ends the session when the call completes. GPU snapshots hide cold starts that would otherwise require reloading gigabytes of model weights. The supplied text is an excerpt and does not include the referenced full code.

### Source excerpt

How to run a LiveKit video avatar with open-weights models on Modal GPUs, using Sticky Sessions for container affinity.

## Muse Spark Open is now available on Modal

DevFeed: [Muse Spark Open is now available on Modal](<https://devfeed.tech/articles/muse-spark-open-is-now-available-on-modal-88078.md>)

Original publisher: [Read original article](<https://modal.com/blog/meta-muse-spark-open-now-available-on-modal>)

Author: Modal

Published: 2026-10-15T00:00:00Z

Content type: news

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [flux](<https://devfeed.tech/topics/flux.md>)

Tags: [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [deepswe](<https://devfeed.tech/tags/deepswe.md>), [deepswe-1-1](<https://devfeed.tech/tags/deepswe-1-1.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [harness](<https://devfeed.tech/tags/harness.md>), [muse](<https://devfeed.tech/tags/muse.md>), [muse-spark](<https://devfeed.tech/tags/muse-spark.md>), [open](<https://devfeed.tech/tags/open.md>), [terminal-bench](<https://devfeed.tech/tags/terminal-bench.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

Meta released weights for Muse Spark Open, a multimodal model designed for coding and long-horizon tasks, and made it available on Modal Auto Endpoints. It supports code writing and debugging, planning, tool use, and failure recovery. The document reports evaluation scores of 76.4 on Terminal-Bench 2.1 and 60 on DeepSWE 1.1, and says the model runs as a Shared Endpoint on Modal with per-token billing.

### Source excerpt

Muse Spark Open, a coding-optimized multimodal model from Meta, is now available on Modal Auto Endpoints.

## Modal crosses $1B in Annualized Run Rate and raises $750M Series D

DevFeed: [Modal crosses $1B in Annualized Run Rate and raises $750M Series D](<https://devfeed.tech/articles/modal-crosses-1b-in-annualized-run-rate-and-raises-750m-series-d-87127.md>)

Original publisher: [Read original article](<https://modal.com/blog/1-billion-annualized-run-rate-series-d>)

Author: Erik Bernhardsson

Published: 2026-10-14T00:00:00Z

Content type: news

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Holoscan](<https://devfeed.tech/topics/holoscan.md>)

Tags: [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [ai-platform](<https://devfeed.tech/tags/ai-platform.md>), [amplify-partners](<https://devfeed.tech/tags/amplify-partners.md>), [bain](<https://devfeed.tech/tags/bain.md>), [build](<https://devfeed.tech/tags/build.md>), [building-software](<https://devfeed.tech/tags/building-software.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

Modal says usage has tripled since its Series C announcement five months earlier and that it crossed $1 billion in annualized run rate. The company describes its AI infrastructure, including a custom file system, container runtime, scheduler and networking across more than 20 clouds. It also reports raising a $750 million Series D at a $15.75 billion post-money valuation.

### Source excerpt

Modal has crossed $1B in annualized run rate and raised a $750M Series D led by Accel to build new infrastructure for AI applications. We can't think of a better time to be building software.

## Toward RL on mixed-precision, mixed-accelerator samplers

DevFeed: [Toward RL on mixed-precision, mixed-accelerator samplers](<https://devfeed.tech/articles/toward-rl-on-mixed-precision-mixed-accelerator-samplers-86229.md>)

Original publisher: [Read original article](<https://modal.com/blog/toward-rl-on-mixed-precision-mixed-accelerator-samplers>)

Author: Nan Jiang

Published: 2026-10-13T12:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [batch](<https://devfeed.tech/tags/batch.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [precision](<https://devfeed.tech/tags/precision.md>), [rl](<https://devfeed.tech/tags/rl.md>)

### AI overview

The article reports experiments using fully asynchronous reinforcement learning with samplers running on mixed accelerators and precisions. Off-policy correction kept training stable and produced performance on par with homogeneous samplers, while mixed samplers cost about 15% less per update for most of training.

### Source excerpt

We trained a model on agentic coding tasks using fully asynchronous RL and mixed-precision, mixed-accelerator samplers. With off-policy correction, training stayed stable through convergence, on par with homogeneous samplers, at a lower cost.

## Endpoint Candidates: evaluate and promote inference endpoints

DevFeed: [Endpoint Candidates: evaluate and promote inference endpoints](<https://devfeed.tech/articles/endpoint-candidates-evaluate-and-promote-inference-endpoints-85472.md>)

Original publisher: [Read original article](<https://modal.com/blog/endpoint-candidates>)

Author: Amit Lohe; Deven Navani; Garrett O'Brien

Published: 2026-10-12T00:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cost](<https://devfeed.tech/tags/cost.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-endpoints](<https://devfeed.tech/tags/inference-endpoints.md>), [inference-performance](<https://devfeed.tech/tags/inference-performance.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [latency](<https://devfeed.tech/tags/latency.md>), [network](<https://devfeed.tech/tags/network.md>)

### AI overview

Modal announces the general availability of Endpoint Candidates, which let teams stage alternative inference deployments and mirror a sample of production traffic to compare quality, latency, throughput, and cost before promotion. The proxy bounds per-candidate buffers and total retained mirror data, canceling overloaded mirrors while production requests continue.

### Source excerpt

Test and promote inference endpoints to evaluate quality, latency, throughput, and cost.

## TypeSafe AI scales Jev to trillions of tokens in three days on Modal

DevFeed: [TypeSafe AI scales Jev to trillions of tokens in three days on Modal](<https://devfeed.tech/articles/typesafe-ai-scales-jev-to-trillions-of-tokens-in-three-days-on-modal-79956.md>)

Original publisher: [Read original article](<https://modal.com/blog/typesafeai-case-study>)

Author: Modal

Published: 2026-10-05T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Serverless](<https://devfeed.tech/topics/serverless.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [baseten](<https://devfeed.tech/topics/baseten.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [compute](<https://devfeed.tech/tags/compute.md>), [inference](<https://devfeed.tech/tags/inference.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [latency](<https://devfeed.tech/tags/latency.md>), [modal](<https://devfeed.tech/tags/modal.md>)

### AI overview

TypeSafe AI built Jev, a model designed to produce fast, structured decisions rather than generated text. Modal helped train and deploy it, and TypeSafe used Modal's serverless infrastructure to handle bursty demand and scale inference while maintaining low latency.

### Source excerpt

TypeSafe AI launched Jev, a new class of AI model built for automation, and scaled to trillions of tokens over a weekend on Modal.

## Modal Clusters are generally available

DevFeed: [Modal Clusters are generally available](<https://devfeed.tech/articles/modal-clusters-are-generally-available-79886.md>)

Original publisher: [Read original article](<https://modal.com/blog/modal-clusters-generally-available>)

Author: Peyton Walters

Published: 2026-10-01T12:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [containers](<https://devfeed.tech/tags/containers.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [rdma](<https://devfeed.tech/tags/rdma.md>)

### AI overview

Modal announces general availability of Modal Clusters, a multi-node GPU compute offering scheduled from shared capacity and billed by the second. It describes gang scheduling, RDMA support, GPU health remediation, and use cases for AI training and inference.

### Source excerpt

Multi-node GPU clusters with RDMA, gang scheduled from Modal's shared capacity pool and billed by the second, behind a single decorator.

## Runtime Roundup: VM Sandboxes, Multi-node clusters, and more

DevFeed: [Runtime Roundup: VM Sandboxes, Multi-node clusters, and more](<https://devfeed.tech/articles/runtime-roundup-vm-sandboxes-multi-node-clusters-and-more-79934.md>)

Original publisher: [Read original article](<https://modal.com/blog/runtime-product-update-sandbox-endpoints>)

Author: Olivia Johnston

Published: 2026-10-01T00:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Docker Sandboxes](<https://devfeed.tech/topics/docker-sandboxes.md>), [clustering](<https://devfeed.tech/topics/clustering.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [cost](<https://devfeed.tech/tags/cost.md>), [inference](<https://devfeed.tech/tags/inference.md>), [sandboxes](<https://devfeed.tech/tags/sandboxes.md>)

### AI overview

Modal recaps announcements from its Runtime conference, including VM Sandboxes for agents, sandbox sidecars, multi-node clusters, sticky sessions, and endpoint candidates for comparing inference servers.

### Source excerpt

Everything announced live at Modal's inaugural conference, Runtime.

## VM Sandboxes: Full computers for agents

DevFeed: [VM Sandboxes: Full computers for agents](<https://devfeed.tech/articles/vm-sandboxes-full-computers-for-agents-79958.md>)

Original publisher: [Read original article](<https://modal.com/blog/vm-sandboxes-agent-computers>)

Author: Amit Prasad

Published: 2026-10-01T00:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Docker Sandboxes](<https://devfeed.tech/topics/docker-sandboxes.md>), [DeepSeek Harness](<https://devfeed.tech/topics/dsh.md>), [cgroups](<https://devfeed.tech/topics/cgroups.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [cgroups](<https://devfeed.tech/tags/cgroups.md>), [container](<https://devfeed.tech/tags/container.md>), [containers](<https://devfeed.tech/tags/containers.md>), [sandboxes](<https://devfeed.tech/tags/sandboxes.md>), [vm](<https://devfeed.tech/tags/vm.md>)

### AI overview

Modal has made VM Sandboxes generally available, offering full Linux virtual machines for agents that need capabilities such as Docker, local databases, development servers, graphical environments, or Linux kernel features. The VM runtime shares APIs and image support with Modal's existing gVisor sandboxes, while existing workloads can continue using gVisor by default. Early customers use the VMs for coding sessions, legal agent evaluations, and agent simulations.

### Source excerpt

VM Sandboxes are built for those who need to give their agents the power of a full computer.

## Sidecars: A low-latency trust boundary for Sandboxes

DevFeed: [Sidecars: A low-latency trust boundary for Sandboxes](<https://devfeed.tech/articles/sidecars-a-low-latency-trust-boundary-for-sandboxes-79873.md>)

Original publisher: [Read original article](<https://modal.com/blog/introducing-sandbox-sidecars-trust-boundary>)

Author: Olivia Johnston; Rebecka Storm; Adam Azzam

Published: 2026-10-01T00:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Docker Sandboxes](<https://devfeed.tech/topics/docker-sandboxes.md>), [AI Sandbox](<https://devfeed.tech/topics/ai-sandbox.md>), [Cloud Architecture](<https://devfeed.tech/topics/cloud-architecture.md>)

Tags: [agent-infrastructure](<https://devfeed.tech/tags/agent-infrastructure.md>), [control-plane](<https://devfeed.tech/tags/control-plane.md>), [data-exfiltration](<https://devfeed.tech/tags/data-exfiltration.md>), [data-exposure](<https://devfeed.tech/tags/data-exposure.md>), [egress](<https://devfeed.tech/tags/egress.md>), [low-latency](<https://devfeed.tech/tags/low-latency.md>), [sandbox](<https://devfeed.tech/tags/sandbox.md>), [sandboxes](<https://devfeed.tech/tags/sandboxes.md>), [sidecars](<https://devfeed.tech/tags/sidecars.md>)

### AI overview

Modal introduces Sidecars, isolated containers that run alongside a Sandbox on the same host to separate trusted services from untrusted code, including agent-generated code. The company says Sidecars provide a trust boundary with local communication, dynamic creation, independent outbound network policies, and lower latency than using separate Sandboxes. Ramp describes using them to run a custom egress proxy that monitors and controls a coding agent's outbound traffic.

### Source excerpt

Sidecars are isolated containers that run alongside your main Sandbox on the same host.

## Quail: Speeding up AI-SQL by jointly optimizing query planner and inference engine

DevFeed: [Quail: Speeding up AI-SQL by jointly optimizing query planner and inference engine](<https://devfeed.tech/articles/quail-speeding-up-ai-sql-by-jointly-optimizing-query-planner-and-inference-engine-79924.md>)

Original publisher: [Read original article](<https://modal.com/blog/quail-billion-tpm>)

Author: Charles Frye; Shreya Shankar

Published: 2026-09-24T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [AI for SQL query generation](<https://devfeed.tech/topics/ai-for-sql-query-generation.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [SQL](<https://devfeed.tech/topics/sql.md>), [JOIN](<https://devfeed.tech/topics/join.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [DuckDB](<https://devfeed.tech/topics/duckdb.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [business-intelligence](<https://devfeed.tech/tags/business-intelligence.md>), [caching](<https://devfeed.tech/tags/caching.md>), [perf](<https://devfeed.tech/tags/perf.md>), [planner](<https://devfeed.tech/tags/planner.md>), [sql](<https://devfeed.tech/tags/sql.md>)

### AI overview

The article presents Quail, an inference engine that jointly optimizes SQL query plans and LLM inference for AI-SQL workloads. It describes query planning, KV-cache reuse and kernel optimizations, and reports over one billion tokens per minute per H100 GPU on one multi-join query, with 1.84x faster performance than a vLLM baseline averaged across the authors' benchmark tasks.

### Source excerpt

Maximizing perf on AI-SQL queries with the KV-optimal left-deep join

## How to serve trillions of tokens for trillion-parameter coding agents

DevFeed: [How to serve trillions of tokens for trillion-parameter coding agents](<https://devfeed.tech/articles/how-to-serve-trillions-of-tokens-for-trillion-parameter-coding-agents-79952.md>)

Original publisher: [Read original article](<https://modal.com/blog/trillion-tokens-trillion-parameters>)

Author: Janelle Cai; Charles Frye; James Liu; Timothy Feng; Richard Gong

Published: 2026-09-23T00:00:00Z

Content type: tutorial

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>), [database-optimization](<https://devfeed.tech/topics/database-optimization.md>), [Chinchilla scaling law](<https://devfeed.tech/topics/chinchilla-scaling-law.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [agentic-coding](<https://devfeed.tech/tags/agentic-coding.md>), [ai](<https://devfeed.tech/tags/ai.md>), [coding](<https://devfeed.tech/tags/coding.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

The article explains how Modal optimized inference for Moonshot AI's Kimi K2.6 coding-agent workload, then scaled the service. It covers speculative decoding, tensor parallelism, KV-cache capacity and routing, and reports improved performance and throughput across replicas.

### Source excerpt

Learn how we optimized performance and efficiency serving the workload that is changing software engineering forever -- and you can too.

[Next page](<https://devfeed.tech/sources/modal.md?cursor=WyIyMDI2LTA5LTIzVDAwOjAwOjAwKzAwOjAwIiwgImI5ZmI0NmIxLTkzMmItNDU0YS05NzgxLTg0OGE5NGM3MDA4ZSJd>)