# CPU Cache

A hardware cache used by a computer's CPU to reduce the time or energy needed to access data from main memory.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Traffic mix matters

DevFeed: [Traffic mix matters](<https://devfeed.tech/articles/traffic-mix-matters-140983.md>)

Original publisher: [Read original article](<http://brooker.co.za/blog/2026/11/18/workload-mix.html>)

Author: Marc Brooker

Published: 2026-11-18T00:00:00Z

Content type: article

Language: en

Sources: [Marc Brooker's](<https://devfeed.tech/sources/marc-brooker-s-blog.md>)

Topics: [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [thread safe data sharing](<https://devfeed.tech/topics/thread-safe-data-sharing.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [database](<https://devfeed.tech/tags/database.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [fast](<https://devfeed.tech/tags/fast.md>), [fifo](<https://devfeed.tech/tags/fifo.md>), [limit](<https://devfeed.tech/tags/limit.md>), [load](<https://devfeed.tech/tags/load.md>), [load-testing](<https://devfeed.tech/tags/load-testing.md>), [local](<https://devfeed.tech/tags/local.md>), [lock](<https://devfeed.tech/tags/lock.md>), [locks](<https://devfeed.tech/tags/locks.md>), [m365](<https://devfeed.tech/tags/m365.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [postgres](<https://devfeed.tech/tags/postgres.md>), [queue](<https://devfeed.tech/tags/queue.md>), [store](<https://devfeed.tech/tags/store.md>), [system](<https://devfeed.tech/tags/system.md>), [traffic](<https://devfeed.tech/tags/traffic.md>)

### AI overview

Workload mix can change throughput by orders of magnitude, so load tests and benchmarks need traffic that resembles production. In a 16-worker lock-table simulation, keys follow a Zipf distribution: greater skew concentrates contention on a few locks, lowering throughput and leaving other resources underused. In a second model, requests use an LRU cache backed by slower storage; greater skew improves cache hits and throughput across a broad range. The opposing results show that key skew can affect systems in different ways. The excerpt also notes a preference for optimistic concurrency control and a single cost/latency mode, and says Aurora DSQL sends reads to one fast storage layer without workers waiting on locks.

### Source excerpt

Traffic mix matters Wanna listen to my mixtape? The preliminary report on last week's M365 outage contains one statement that really caught my eye: Our normal load testing did not detect the latent issue because [of] the traffic mix and SKUs used in our labs. There's a lot going on in this report (as there is in any report for a complex system), but I wanted to zoom in on this one issue: traffic mix. It's important, because its the one thing that makes the difference between a great load test and a complete waste of time. The performance of computer systems is extremely sensitive to workload mix. Let's start with a simple simulator, looking at a system with a table of locks. Requests arrive with a key, hold the lock for their key for some period of time, and then free the lock and complete. This isn't an unreasonable mental model of a relational database, where each transaction takes a lock during its execution. arrivals backlog 16 worker threads lock table (one lock per key) done Poisson FIFO, never empty at saturation holder FIFO waiters hold ~Exp(mean 1 ms) key 1 key 2 key 3 ⋮ key N holding its key's lock blocked behind the holder (thread still occupied) Keys (N): Simulate Each request takes one of 16 worker threads, waits in a FIFO queue for the lock on its key, holds the lock for an exponentially distributed time (mean 1 ms), then releases it and frees its thread. Keys are drawn from a Zipf distribution over N keys; a coefficient of 0 is uniform. As we may expect, the more skewed the distribution, the less throughput you get out of the system. A small number of keys are highly contended, leading to under utilization of the rest of the resources of the machine, driven by the global concurrency limit. Now, let's consider a different mental model of a relational database (at least a single-system one like Postgres). A fast, local, in-memory cache containing some of the accessed data backed by a slower store containing the rest of the data. arrivals backlog 16 work

## Intel Diamond Rapids Xeon Gets Up to 1.6TB/s of Memory Bandwidth and 1.28GB of Cache Across 256 P-Cores

DevFeed: [Intel Diamond Rapids Xeon Gets Up to 1.6TB/s of Memory Bandwidth and 1.28GB of Cache Across 256 P-Cores](<https://devfeed.tech/articles/intel-diamond-rapids-xeon-gets-up-to-1-6tb-s-of-memory-bandwidth-and-1-28gb-of-cache-across-256-p-cores-136080.md>)

Original publisher: [Read original article](<https://www.storagereview.com/news/intel-diamond-rapids-1-6tb-s-memory-bandwidth-1-28gb-cache-gpu-d270-sc26>)

Author: Harold Fritts

Published: 2026-11-16T15:39:46Z

Content type: news

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [18a](<https://devfeed.tech/tags/18a.md>), [acceleration](<https://devfeed.tech/tags/acceleration.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-frameworks](<https://devfeed.tech/tags/ai-frameworks.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [cache](<https://devfeed.tech/tags/cache.md>), [computing](<https://devfeed.tech/tags/computing.md>), [diamond-rapids](<https://devfeed.tech/tags/diamond-rapids.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [high-performance-computing](<https://devfeed.tech/tags/high-performance-computing.md>), [intel](<https://devfeed.tech/tags/intel.md>), [last-level-cache](<https://devfeed.tech/tags/last-level-cache.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-bandwidth](<https://devfeed.tech/tags/memory-bandwidth.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [scale](<https://devfeed.tech/tags/scale.md>), [server](<https://devfeed.tech/tags/server.md>), [xeon](<https://devfeed.tech/tags/xeon.md>)

### AI overview

Intel says its planned Diamond Rapids Xeon will scale to 256 performance cores, up to 1.6TB/s of memory bandwidth and 1.28GB of last-level cache, with APX, AVX 10.2 and enhanced AMX. The bandwidth figure's memory configuration and whether it is per socket remain unspecified, and Intel gave no launch date. The article also describes the air-cooled PCIe GPU D270, with 160GB of LPDDR5X and 1.5TB/s bandwidth, plus partner designs up to 480GB. Intel's performance comparisons against NVIDIA are vendor results on pre-production hardware; they were not independently run or audited, and the financial comparison's per-card normalization is unclear. Intel also outlines continued investment in HPC software, silicon photonics and a proposed Classiq acquisition, pending regulatory approval.

### Source excerpt

Intel laid out its high-performance computing plans for SC26, with Diamond Rapids, its next-generation Xeon server architecture, at the center. Intel says the platform will scale to 256 performance cores with up to 1.6TB/s of memory bandwidth and up to 1.28GB of last-level cache. The outline came from Jeff McVeigh, vice president and general manager The post Intel Diamond Rapids Xeon Gets Up to 1.6TB/s of Memory Bandwidth and 1.28GB of Cache Across 256 P-Cores appeared first on StorageReview.com.

## Anatomy of a CPU: How Every Part of a Modern CPU Works Together

DevFeed: [Anatomy of a CPU: How Every Part of a Modern CPU Works Together](<https://devfeed.tech/articles/anatomy-of-a-cpu-how-every-part-of-a-modern-cpu-works-together-134713.md>)

Original publisher: [Read original article](<https://www.techspot.com/article/2000-anatomy-cpu/>)

Author: Julio Franco

Published: 2026-11-16T06:18:00Z

Content type: article

Language: en

Sources: [TechSpot](<https://devfeed.tech/sources/techspot.md>)

Topics: [cpu](<https://devfeed.tech/topics/cpu.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [Cache Coherence](<https://devfeed.tech/topics/cache-coherence.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-accelerators](<https://devfeed.tech/tags/ai-accelerators.md>), [arm](<https://devfeed.tech/tags/arm.md>), [branch-prediction](<https://devfeed.tech/tags/branch-prediction.md>), [chiplets](<https://devfeed.tech/tags/chiplets.md>), [cpu](<https://devfeed.tech/tags/cpu.md>)

### AI overview

A modern CPU overlaps instruction work and predicts what it will need to keep its cores busy. It fetches instructions in blocks, decodes them into operations, and uses registers, execution units, and a cache hierarchy to process data. Branch prediction, pipelining, out-of-order execution, and prefetching improve throughput, while speculative execution can leave cache traces that create security risks. Modern processors also integrate GPUs, NPUs, and media or vector accelerators. Multiple cores communicate through an on-chip interconnect and coordinate shared data; hybrid cores and chiplets balance performance, efficiency, and manufacturing cost with scheduling and communication tradeoffs. The article presents these as general features, since manufacturers keep circuit details proprietary, and illustrates them with AMD Zen 5.

### Source excerpt

What actually happens inside a CPU? We break down the key parts of a modern processor, from fetching and decoding instructions to caches, branch prediction, chiplets and AI accelerators. Read Entire Article

## Probabilistic data structures in Go

DevFeed: [Probabilistic data structures in Go](<https://devfeed.tech/articles/probabilistic-data-structures-in-go-132644.md>)

Original publisher: [Read original article](<https://antonz.org/probabilistic-data-structures/>)

Author: Anton Zhiyanov

Published: 2026-11-13T15:00:00Z

Content type: article

Language: en

Sources: [Anton Zhiyanov](<https://devfeed.tech/sources/anton-zhiyanov.md>)

Topics: [Data structures](<https://devfeed.tech/topics/data-structures.md>), [bloom-filter](<https://devfeed.tech/topics/bloom-filter.md>), [hashing](<https://devfeed.tech/topics/hashing.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [rate-limiting](<https://devfeed.tech/topics/rate-limiting.md>)

Tags: [bloom-filter](<https://devfeed.tech/tags/bloom-filter.md>), [count-min-sketch](<https://devfeed.tech/tags/count-min-sketch.md>), [data-structures](<https://devfeed.tech/tags/data-structures.md>), [go](<https://devfeed.tech/tags/go.md>), [hyperloglog](<https://devfeed.tech/tags/hyperloglog.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [probabilistic-data-structures](<https://devfeed.tech/tags/probabilistic-data-structures.md>), [t-digest](<https://devfeed.tech/tags/t-digest.md>), [top-k](<https://devfeed.tech/tags/top-k.md>)

### AI overview

The Go package `nalgeon/unsure` implements five approximate structures that trade exact answers for lower memory use: Bloom filters test possible set membership (false positives, no deletion); count-min sketches estimate frequencies with possible overcounts; HyperLogLog estimates distinct-item counts; t-digests estimate quantiles and CDFs; and top-k sketches report frequent items with approximate counts. The article describes `unsure`'s cache-blocked Bloom filter, conservative count-min updates, histogram-based HyperLogLog estimator, merging t-digest, and two-row HeavyKeeper-style top-k design. It gives examples of workloads and limits, including that t-digest merges lose accuracy, top-k works best on skewed streams, and HyperLogLog's sparse-count comparison favors `axiom`. Reported benchmarks show `unsure` often faster or smaller than named Go implementations, though the advantage varies by operation and data distribution; these are excerpts from full benchmarks.

### Source excerpt

Bloom filter, count-min sketch, HyperLogLog, t-digest, and top-k sketch.

## LazyWeights: Decode-on-Demand Edge LLMs

DevFeed: [LazyWeights: Decode-on-Demand Edge LLMs](<https://devfeed.tech/articles/lazyweights-decode-on-demand-edge-llms-133173.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17108>)

Author: Arnab Sanyal, Gourav Datta, Sandeep P. Chinchali

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Jetson](<https://devfeed.tech/topics/jetson.md>), [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [communications](<https://devfeed.tech/topics/communications.md>), [Tensor Cores](<https://devfeed.tech/topics/tensor-cores.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [amd](<https://devfeed.tech/tags/amd.md>), [area](<https://devfeed.tech/tags/area.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asic](<https://devfeed.tech/tags/asic.md>), [awq](<https://devfeed.tech/tags/awq.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [beats](<https://devfeed.tech/tags/beats.md>), [best](<https://devfeed.tech/tags/best.md>), [bits](<https://devfeed.tech/tags/bits.md>), [but](<https://devfeed.tech/tags/but.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [code](<https://devfeed.tech/tags/code.md>), [compress](<https://devfeed.tech/tags/compress.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-it](<https://devfeed.tech/tags/cs-it.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decode](<https://devfeed.tech/tags/decode.md>), [demand](<https://devfeed.tech/tags/demand.md>), [edge](<https://devfeed.tech/tags/edge.md>), [llms](<https://devfeed.tech/tags/llms.md>), [math-it](<https://devfeed.tech/tags/math-it.md>)

### AI overview

LazyWeights reduces edge LLM memory traffic by tuning GPTQ codes for compression, Huffman-coding them losslessly in interleaved lanes, and decoding on demand between DRAM and L2 while leaving GPU kernels unchanged. At stated 1% and 3.5% perplexity budgets, three tested models were 5.7-31.4% smaller than the best compared llama.cpp formats. The abstract reports up to 36% lower latency and 38% lower energy per token on Jetson Orin Nano Super, with larger reductions using prompt-lookup speculation; on Jetson Nano it reports 58-64% less time and 53-57% less energy than CPU llama.cpp. A streamed Mistral-7B ran 6.6x faster at 3.89 bits per weight. The decoder was evaluated as a small ASIC unit and shown bit-exact on a Zynq UltraScale+ FPGA. These are abstract-level reported results; experimental details are not included.

### Source excerpt

arXiv:2611.17108v1 Announce Type: new Abstract: Memory decides which LLMs an edge device runs, and how fast: capacity on a $4$ GB Jetson Nano, bandwidth on an $8$ GB Jetson Orin Nano Super. Deployed formats (GPTQ, AWQ, $\texttt{llama.cpp}$'s K- and I-quants) spend equal bits on every integer code of a matrix, although the codes are far from uniform. $\texttt{LazyWeights}$ keeps these formats but tunes GPTQ so the codes compress, Huffman-codes them losslessly in interleaved lanes, and decodes them on demand in a small unit between DRAM and L2, so GPU kernels run unchanged and DRAM moves only coded bytes. At $1\%$ and $3.5\%$ perplexity budgets, coded Phi-3-mini, Mistral-7B and SmolLM-1.7B are $5.7-31.4\%$ smaller than the best $\texttt{llama.cpp}$ format. A bit-exact $0.21$ mm$^2$ ASIC decoder unit ($1$ GHz, $7$ nm) cuts Orin latency and energy per token by up to $36\%$ and $38\%$ ($54\%$ and $44\%$ with prompt-lookup speculation). On the Nano, $\texttt{LazyWeights}$ needs $58-64\%$ less time and $53-57\%$ less energy per token than CPU $\texttt{llama.cpp}$, and runs a streamed Mistral-7B 6.6$\times$ faster at $3.89$ bits per weight. A Huffman decoder beats an rANS decoder in rate and area at equal throughput; the ASIC decoder unit runs bit-exact on a Zynq UltraScale+ FPGA (AMD ZCU102).

## Google Cloud Warns Three AMD Turin CPU Flaws Could Leak Data Across Processes in VMs

DevFeed: [Google Cloud Warns Three AMD Turin CPU Flaws Could Leak Data Across Processes in VMs](<https://devfeed.tech/articles/google-cloud-warns-three-amd-turin-cpu-flaws-could-leak-data-across-processes-in-vms-130463.md>)

Original publisher: [Read original article](<https://thehackernews.com/2026/11/google-cloud-warns-three-amd-turin-cpu.html>)

Author: The Hacker News

Published: 2026-11-12T12:09:14Z

Content type: news

Language: en

Sources: [Cyber Newsroom](<https://devfeed.tech/sources/cyber-newsroom.md>)

Topics: [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Security research](<https://devfeed.tech/topics/security-research.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [Gke](<https://devfeed.tech/topics/gke.md>)

Tags: [2025](<https://devfeed.tech/tags/2025.md>), [after](<https://devfeed.tech/tags/after.md>), [all](<https://devfeed.tech/tags/all.md>), [amd](<https://devfeed.tech/tags/amd.md>), [channel](<https://devfeed.tech/tags/channel.md>), [chips](<https://devfeed.tech/tags/chips.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-security](<https://devfeed.tech/tags/cloud-security.md>), [company](<https://devfeed.tech/tags/company.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [data](<https://devfeed.tech/tags/data.md>), [google](<https://devfeed.tech/tags/google.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [hardware-security](<https://devfeed.tech/tags/hardware-security.md>), [linux-security](<https://devfeed.tech/tags/linux-security.md>), [need](<https://devfeed.tech/tags/need.md>), [os](<https://devfeed.tech/tags/os.md>), [patch](<https://devfeed.tech/tags/patch.md>), [processes](<https://devfeed.tech/tags/processes.md>), [side-channel](<https://devfeed.tech/tags/side-channel.md>), [stop](<https://devfeed.tech/tags/stop.md>), [this](<https://devfeed.tech/tags/this.md>), [turin](<https://devfeed.tech/tags/turin.md>), [virtualization](<https://devfeed.tech/tags/virtualization.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

Google Cloud says three Transient Scheduler Attacks on AMD Turin CPUs could let code running on the same processor core infer data from other VM processes or the kernel. Google has mitigated cross-VM and VM-to-host attacks; the reported risk is within a VM. For affected VMs, customers need an OS vendor fix, then must stop and start the VM after Google's CPU information update reaches it; a guest OS reboot alone is insufficient. Bare metal C4D and Z4D users need vendor fixes, while Turin GKE node images were not yet listed as available and Cloud Run requires no action. The supplied text says Linux kernel and Xen mitigations exist, but Debian had fixed the new flaw only in unstable. For Turin servers on their own hardware, AMD says firmware and OS updates are both needed; mitigations may affect performance. Google rates the issue Medium, and the text reports no known exploitation.

### Source excerpt

Google Cloud Warns Three AMD Turin CPU Flaws Could Leak Data Across Processes in VMs Google Cloud VMs on AMD Turin chips need a guest OS patch and a stop-and-start to protect against three CPU side-channel flaws, Google said this week. The company has already applied mitigations again ... Read more Published Date: Nov 12, 2026 (3 hours, 10 minutes ago) Vulnerabilities has been mentioned in this article.

## Rethinking Memory For AI Inference

DevFeed: [Rethinking Memory For AI Inference](<https://devfeed.tech/articles/rethinking-memory-for-ai-inference-130168.md>)

Original publisher: [Read original article](<https://semiengineering.com/rethinking-memory-for-ai-inference/>)

Author: Liz Allan

Published: 2026-11-12T08:11:46Z

Content type: article

Language: en

Sources: [Semiconductor Engineering](<https://devfeed.tech/sources/semiconductor-engineering.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [about](<https://devfeed.tech/tags/about.md>), [abstraction](<https://devfeed.tech/tags/abstraction.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [arteris](<https://devfeed.tech/tags/arteris.md>), [ayar-labs](<https://devfeed.tech/tags/ayar-labs.md>), [baya-systems](<https://devfeed.tech/tags/baya-systems.md>), [boost](<https://devfeed.tech/tags/boost.md>), [but](<https://devfeed.tech/tags/but.md>), [cadence](<https://devfeed.tech/tags/cadence.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [compute-in-memory](<https://devfeed.tech/tags/compute-in-memory.md>), [data](<https://devfeed.tech/tags/data.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [expedera](<https://devfeed.tech/tags/expedera.md>), [in-memory-compute](<https://devfeed.tech/tags/in-memory-compute.md>), [inference](<https://devfeed.tech/tags/inference.md>), [keysight-eda](<https://devfeed.tech/tags/keysight-eda.md>), [low-power-high-performance](<https://devfeed.tech/tags/low-power-high-performance.md>), [lpddr](<https://devfeed.tech/tags/lpddr.md>), [memory](<https://devfeed.tech/tags/memory.md>), [near-memory](<https://devfeed.tech/tags/near-memory.md>), [performance](<https://devfeed.tech/tags/performance.md>), [power](<https://devfeed.tech/tags/power.md>), [processing](<https://devfeed.tech/tags/processing.md>), [processing-in-memory](<https://devfeed.tech/tags/processing-in-memory.md>), [processing-near-memory](<https://devfeed.tech/tags/processing-near-memory.md>), [quadric](<https://devfeed.tech/tags/quadric.md>), [rambus](<https://devfeed.tech/tags/rambus.md>), [reduce](<https://devfeed.tech/tags/reduce.md>), [semiconductor](<https://devfeed.tech/tags/semiconductor.md>), [semiconductor-engineering](<https://devfeed.tech/tags/semiconductor-engineering.md>), [sk-hynix](<https://devfeed.tech/tags/sk-hynix.md>), [software](<https://devfeed.tech/tags/software.md>), [thermal](<https://devfeed.tech/tags/thermal.md>), [top-stories](<https://devfeed.tech/tags/top-stories.md>), [vinci](<https://devfeed.tech/tags/vinci.md>)

### AI overview

Moving AI inference compute closer to memory can reduce data movement, latency, and energy use, which matters for edge devices with limited power. The best fit depends on the workload: PIM and near-memory approaches can help low-batch decoding, while large-batch decode and prefill may become activation-communication bottlenecks. LPDDR6-PIM targets low-power applications with simple MAC operations; custom HBM logic base dies allow broader functions but face thermal limits. Adoption also requires compilers, operating systems, memory controllers, and system protocols to support offloading and scheduling. Hardware and software complexity, manufacturing cost, workload matching, and heat remain constraints, and broad commercial use is still early.

### Source excerpt

Processing data closer to memory could reduce power and boost performance, but software, thermal, and architectural challenges remain. The post Rethinking Memory For AI Inference appeared first on Semiconductor Engineering.

## The Memory Challenge Of Physical AI

DevFeed: [The Memory Challenge Of Physical AI](<https://devfeed.tech/articles/the-memory-challenge-of-physical-ai-130170.md>)

Original publisher: [Read original article](<https://semiengineering.com/the-memory-challenge-of-physical-ai/>)

Author: Joe Chen

Published: 2026-11-12T08:04:39Z

Content type: article

Language: en

Sources: [Semiconductor Engineering](<https://devfeed.tech/sources/semiconductor-engineering.md>)

Topics: [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Memory Heirarchy](<https://devfeed.tech/topics/memory-heirarchy.md>), [Chip design](<https://devfeed.tech/topics/chip-design.md>), [Robotics](<https://devfeed.tech/topics/robotics.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [act](<https://devfeed.tech/tags/act.md>), [actuator](<https://devfeed.tech/tags/actuator.md>), [adas](<https://devfeed.tech/tags/adas.md>), [ai](<https://devfeed.tech/tags/ai.md>), [automotive](<https://devfeed.tech/tags/automotive.md>), [blogs-lphp](<https://devfeed.tech/tags/blogs-lphp.md>), [cadence](<https://devfeed.tech/tags/cadence.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [data](<https://devfeed.tech/tags/data.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [device](<https://devfeed.tech/tags/device.md>), [everything-low-power](<https://devfeed.tech/tags/everything-low-power.md>), [gddr](<https://devfeed.tech/tags/gddr.md>), [hierarchy](<https://devfeed.tech/tags/hierarchy.md>), [latency](<https://devfeed.tech/tags/latency.md>), [low-power-high-performance](<https://devfeed.tech/tags/low-power-high-performance.md>), [lpddr](<https://devfeed.tech/tags/lpddr.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-hierarchy](<https://devfeed.tech/tags/memory-hierarchy.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [physical-ai](<https://devfeed.tech/tags/physical-ai.md>), [requirements](<https://devfeed.tech/tags/requirements.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [right](<https://devfeed.tech/tags/right.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [sensor-streaming](<https://devfeed.tech/tags/sensor-streaming.md>), [sensors](<https://devfeed.tech/tags/sensors.md>), [size](<https://devfeed.tech/tags/size.md>), [streams](<https://devfeed.tech/tags/streams.md>)

### AI overview

Physical AI performance depends on predictable data movement across compute, on-chip SRAM and cache, and external memory. Keeping frequently reused or latency-sensitive data close to processing engines can reduce transfers, energy use, bandwidth demand, and stalls; limited on-chip capacity means less-used or larger data remains in external memory. Architects should size the full hierarchy--including capacity, bandwidth, buffering, power, and integration--for each device's streams, reuse patterns, latency needs, and operating limits. The text describes LPDDR as a fit when bandwidth per watt and capacity matter, while GDDR may suit continuous image or sensor streams needing high bandwidth with fewer DRAM devices. Humanoids, drones, cameras, and vehicles impose different combinations of power, thermal, throughput, and response-predictability demands. Cadence is cited as offering LPDDR6/LPDDR5X PHY and Controller IP up to 14.4 Gbps and GDDR7 PHY IP up to 36 Gbps.

### Source excerpt

Right-size the memory hierarchy around the device's data streams, reuse patterns, latency requirements, and operating constraints. The post The Memory Challenge Of Physical AI appeared first on Semiconductor Engineering.

## CACHEFORGE: LLM-Guided End-to-End Generative Cache Replacement Policy for Performance and Hardware Efficiency

DevFeed: [CACHEFORGE: LLM-Guided End-to-End Generative Cache Replacement Policy for Performance and Hardware Efficiency](<https://devfeed.tech/articles/cacheforge-llm-guided-end-to-end-generative-cache-replacement-policy-for-performance-and-hardware-efficiency-128150.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.07668>)

Author: Kaushal Mhapsekar, Bita Aslrousta, Brijesh Kumar Bhayana, Paula Contreras, Azam Ghanbari, Ethan Goodman, Anna Andriiko, Samira Mirbagher Ajorpaz

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [generative](<https://devfeed.tech/tags/generative.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [ipc](<https://devfeed.tech/tags/ipc.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [llm](<https://devfeed.tech/tags/llm.md>), [memory](<https://devfeed.tech/tags/memory.md>), [mutation](<https://devfeed.tech/tags/mutation.md>), [performance](<https://devfeed.tech/tags/performance.md>), [policy](<https://devfeed.tech/tags/policy.md>), [prefetching](<https://devfeed.tech/tags/prefetching.md>), [replacement](<https://devfeed.tech/tags/replacement.md>), [reward](<https://devfeed.tech/tags/reward.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [simulator](<https://devfeed.tech/tags/simulator.md>), [spec](<https://devfeed.tech/tags/spec.md>), [temperature](<https://devfeed.tech/tags/temperature.md>)

### AI overview

CACHEFORGE uses an LLM-guided hardware-aware loop to generate and evolve cache replacement policies: it proposes C++ logic, evaluates it in the trace-based CRC-2 ChampSim simulator, and applies reward shaping, structural checks, mutation, temperature scheduling, and policy crossover to enforce feasibility and explore alternatives. Across SPEC CPU2006, the abstract reports total hit-rate gains over CRC-2 baselines ranging from 5.73% over LRU to 27.36% over MPPPB. On memory-intensive workloads, it reports IPC gains ranging from 2.71% over Hawkeye to 10.15% over LRU. These results are reported for the stated benchmarks and comparisons.

### Source excerpt

arXiv:2610.07668v2 Announce Type: replace-cross Abstract: Modern cache replacement designs saturate because they operate within fixed representational structures, hand-crafted and heuristic based feature-engineered predictors, or offline imitation models that cannot generate new decision logic on their own. At the same time, replacement is shaped by the causal interaction of prefetching, thrashing, spatial locality, and access-type behavior, producing an enormous design space that is difficult to traverse manually. Prior approaches typically rely on heuristics, parameter tuning, or imitation of an offline optimal policy, capturing correlations rather than synthesizing new mechanisms. As a result, their performance gains often plateau and they overfit under dynamic workload conditions. CACHEFORGE is the first framework to evolve cache-replacement policies end-to-end by embedding a large language model inside a governed hardware-aware loop. In each iteration, the LLM proposes new C++ replacement logic, the policy is evaluated under a trace-based CRC-2 ChampSim simulator, and the framework enforces feasibility through reward shaping, structural checks, dynamic mutation, temperature scheduling, and cross-policy crossover. This closed-loop generation-evolution loop specifically designed for cache replacement policy enables the discovery of compact policies that satisfy hardware constraints while exploring algorithmic transformations beyond fixed predictor structures. Across SPEC CPU2006, CACHEFORGE outperforms all CRC-2 baselines. It improves the total hit rate by 27.36%, 19.69%, 13.72%, 13.15%, 11.83%, and 5.73% over MPPPB, ReD, Hawk-eye, SHiP++, LIME, and LRU, respectively. On memory-intensive workloads, it increases IPC by 10.15%, 7.89%, 6.34%, 3.64%, 3.12%, and 2.71% over LRU, MPPPB, LIME, ReD, SHiP++, and Hawkeye.

## Randomized Caches in HW: Overheads And Limitations (RUB, Max Planck, Georgia Tech)

DevFeed: [Randomized Caches in HW: Overheads And Limitations (RUB, Max Planck, Georgia Tech)](<https://devfeed.tech/articles/randomized-caches-in-hw-overheads-and-limitations-rub-max-planck-georgia-tech-115787.md>)

Original publisher: [Read original article](<https://semiengineering.com/randomized-caches-in-hw-overheads-and-limitations-rub-max-planck-georgia-tech/>)

Author: Technical Paper Link

Published: 2026-11-03T18:01:10Z

Content type: news

Language: en

Sources: [Semiconductor Engineering](<https://devfeed.tech/sources/semiconductor-engineering.md>)

Topics: [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [side channel](<https://devfeed.tech/topics/side-channel.md>), [Chip design](<https://devfeed.tech/topics/chip-design.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asic](<https://devfeed.tech/tags/asic.md>), [cache-security](<https://devfeed.tech/tags/cache-security.md>), [fpga](<https://devfeed.tech/tags/fpga.md>), [georgia-tech](<https://devfeed.tech/tags/georgia-tech.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hardware-security](<https://devfeed.tech/tags/hardware-security.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [last-level-cache](<https://devfeed.tech/tags/last-level-cache.md>), [max-planck-institute-for-security-and-privacy](<https://devfeed.tech/tags/max-planck-institute-for-security-and-privacy.md>), [microarchitecture](<https://devfeed.tech/tags/microarchitecture.md>), [processor-architecture](<https://devfeed.tech/tags/processor-architecture.md>), [randomized-caches](<https://devfeed.tech/tags/randomized-caches.md>), [risc-v](<https://devfeed.tech/tags/risc-v.md>), [ruhr-university-bochum](<https://devfeed.tech/tags/ruhr-university-bochum.md>), [security](<https://devfeed.tech/tags/security.md>), [side-channel-attacks](<https://devfeed.tech/tags/side-channel-attacks.md>), [soc](<https://devfeed.tech/tags/soc.md>), [spec](<https://devfeed.tech/tags/spec.md>), [technical-papers](<https://devfeed.tech/tags/technical-papers.md>)

### AI overview

The researchers implemented CEASER-S, ScatterCache, SassCache, and MIRAGE as last-level caches in a multicore, out-of-order RISC-V system-on-chip to examine practical hardware challenges that architectural descriptions leave unexplored. They report common and design-specific implementation challenges and practical solutions. Their complete FPGA implementations were evaluated by running SPEC CPU 2017 rate and GAPBS workloads to completion, providing, to their knowledge, the first comparative evaluation of these designs on a shared full-system hardware platform. The excerpt does not give the designs' specific overheads, limitations, or comparative results.

### Source excerpt

Researchers at Ruhr University Bochum, the Max Planck Institute for Security and Privacy, and Georgia Tech published "Randomized Caches in Hardware: Challenges and Trade-offs." Abstract Excerpt: "Randomized caches protect against side-channel attacks by obfuscating the mapping of memory addresses to cache locations. Existing proposals typically describe architectural behavior, leaving practical hardware implementation challenges largely unexplored.... " read more The post Randomized Caches in HW: Overheads And Limitations (RUB, Max Planck, Georgia Tech) appeared first on Semiconductor Engineering.

## A cheap FPGA board becomes retro-style computer

DevFeed: [A cheap FPGA board becomes retro-style computer](<https://devfeed.tech/articles/a-cheap-fpga-board-becomes-retro-style-computer-114866.md>)

Original publisher: [Read original article](<https://blog.adafruit.com/2026/11/03/a-cheap-fpga-board-becomes-retro-style-computer/>)

Author: Anne Barela

Published: 2026-11-03T17:12:59Z

Content type: news

Language: en

Sources: [Adafruit Industries](<https://devfeed.tech/sources/adafruit-industries-makers-hackers-artists-designers-and-engineers.md>)

Topics: [cpu](<https://devfeed.tech/topics/cpu.md>), [Computing](<https://devfeed.tech/topics/computing.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [board](<https://devfeed.tech/tags/board.md>), [cache](<https://devfeed.tech/tags/cache.md>), [chip](<https://devfeed.tech/tags/chip.md>), [clocks](<https://devfeed.tech/tags/clocks.md>), [computer](<https://devfeed.tech/tags/computer.md>), [core](<https://devfeed.tech/tags/core.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [fpga](<https://devfeed.tech/tags/fpga.md>), [gaming](<https://devfeed.tech/tags/gaming.md>), [github](<https://devfeed.tech/tags/github.md>), [gowin](<https://devfeed.tech/tags/gowin.md>), [hdmi](<https://devfeed.tech/tags/hdmi.md>), [history](<https://devfeed.tech/tags/history.md>), [inside](<https://devfeed.tech/tags/inside.md>), [logic](<https://devfeed.tech/tags/logic.md>), [psram](<https://devfeed.tech/tags/psram.md>), [retro](<https://devfeed.tech/tags/retro.md>), [risc-v](<https://devfeed.tech/tags/risc-v.md>), [sd](<https://devfeed.tech/tags/sd.md>), [sipeed](<https://devfeed.tech/tags/sipeed.md>), [tang-nano-9k](<https://devfeed.tech/tags/tang-nano-9k.md>), [vintage-computing](<https://devfeed.tech/tags/vintage-computing.md>)

### AI overview

The m9b project fits a retro-style computer into a $15 Sipeed Tang Nano 9K by building its 36 MHz RISC-V CPU and peripherals in the Gowin FPGA's logic, which has 8,640 lookup tables and 52 KB of block RAM but no hard CPU core. The system adds 24 KB of cache, 8 MB of PSRAM, HDMI output at 720x480 and 60 Hz, 48 kHz stereo audio, two USB ports, and SD storage. Its HDMI encoder, audio path, and USB host are also implemented in logic, with all clocks derived from the board's 27 MHz crystal. The author reports porting Doom, Descent, Tyrian, System Shock, and Ultima VII. Place and route uses 7,560 of 8,640 logic cells, 97% of slices, and 25 of 26 block RAMs. This is part one of a six-part write-up, focused on the FPGA, system around the CPU, and clocking.

### Source excerpt

The Parts Bin has built a complete retro-style computer inside a $15 Sipeed Tang Nano 9K board, and posted it at the-parts-bin.github.io. The Gowin FPGA on the board has 8,640 lookup tables and 52 KB of block RAM. It doesn't have a hard CPU core, so the processor had to be built from the same [...]

## Randomized Caches in Hardware: Challenges and Trade-offs

DevFeed: [Randomized Caches in Hardware: Challenges and Trade-offs](<https://devfeed.tech/articles/randomized-caches-in-hardware-challenges-and-trade-offs-114019.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.00282>)

Author: Xinrui Wang, Pascal Kosak, Christof Paar, Daniel Genkin, Yuval Yarom, Anirban Chakraborty

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [RISC-V](<https://devfeed.tech/topics/riscv.md>), [implementation](<https://devfeed.tech/topics/implementation.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>)

Tags: [2017](<https://devfeed.tech/tags/2017.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [added](<https://devfeed.tech/tags/added.md>), [area](<https://devfeed.tech/tags/area.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [asic](<https://devfeed.tech/tags/asic.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [cache](<https://devfeed.tech/tags/cache.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [channel](<https://devfeed.tech/tags/channel.md>), [chip](<https://devfeed.tech/tags/chip.md>), [complete](<https://devfeed.tech/tags/complete.md>), [completion](<https://devfeed.tech/tags/completion.md>), [components](<https://devfeed.tech/tags/components.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [core](<https://devfeed.tech/tags/core.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [randomized-caches](<https://devfeed.tech/tags/randomized-caches.md>)

### AI overview

The study implements CEASER-S, ScatterCache, SassCache, and MIRAGE as LLCs in a multi-core out-of-order RISC-V SoC and evaluates full FPGA systems on SPEC CPU 2017 rate and GAPBS workloads. Pipelining the index derivation function alone leaves other critical paths as frequency bottlenecks in three variants, lowering frequency by up to 35% versus a baseline set-associative cache. Redesigning those paths while retaining mappings, rekeying, and victim-selection policies brings the affected variants to within 10% of baseline frequency, with added latency or reduced concurrency. Increased LLC access latency, more than additional misses, drives IPC losses of up to 5% on SPEC and 11% on GAPBS. Reported overheads vary by design and technology: FPGA lookup-table use rises 10-35%, while ASIC area rises 1-14%.

### Source excerpt

arXiv:2611.00282v1 Announce Type: cross Abstract: Randomized caches protect against side-channel attacks by obfuscating the mapping of memory addresses to cache locations. Existing proposals typically describe architectural behavior, leaving practical hardware implementation challenges largely unexplored. We implement four prominent designs, CEASER-S, ScatterCache, SassCache, and MIRAGE, and integrate them as the last-level cache (LLC) of a multi-core out-of-order RISC-V system-on-chip (SoC). We identify common and design-specific implementation challenges and develop practical solutions. We evaluate complete FPGA implementations by running SPEC CPU 2017 rate and GAPBS workloads to completion, providing, to our knowledge, the first comparative evaluation of these designs on a common full-system hardware platform. Pipelining the index derivation function (IDF) alone is insufficient: in three of the four variants, other components become the frequency bottleneck, reducing frequency by up to 35% relative to the baseline set-associative cache. Redesigning the critical paths while preserving mappings, rekeying, and victim-selection policies brings the three affected variants to no more than 10% below the baseline frequency, at the cost of added latency or reduced concurrency. Increased LLC access latency, rather than additional misses, dominates IPC degradation, reducing instructions per cycle by up to 5% for SPEC and 11% for GAPBS. Hardware overhead depends on the design and implementation technology: FPGA lookup-table usage increases by 10-35%, while ASIC area increases by 1-14%.

## High-Throughput Private Information Retrieval with on-CPU Accelerators

DevFeed: [High-Throughput Private Information Retrieval with on-CPU Accelerators](<https://devfeed.tech/articles/high-throughput-private-information-retrieval-with-on-cpu-accelerators-113652.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02376>)

Author: Gyuyeong Kim

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [batching](<https://devfeed.tech/tags/batching.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cs-os](<https://devfeed.tech/tags/cs-os.md>), [information-retrieval](<https://devfeed.tech/tags/information-retrieval.md>), [private](<https://devfeed.tech/tags/private.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

AltoPIR uses on-CPU matrix accelerators such as Intel AMX to improve batched private information retrieval. It keeps the accelerator fed from CPU caches, reads each database byte once per batch, and avoids writing intermediate products to memory. The authors report up to 50.1x the throughput of existing CPU-based solutions and support for databases larger than a GPU's memory, without requiring a discrete accelerator.

### Source excerpt

arXiv:2611.02376v1 Announce Type: cross Abstract: Private information retrieval (PIR) hides which record a client retrieves by scanning the entire database, and batching is an effective technique to improve throughput by sharing this scan across queries. However, existing CPU-based solutions gain little from batching because each query still requires its own multiplications, and vector instructions perform only a few at a time. GPUs and other discrete accelerators can accelerate these computations, but require extra hardware and can constrain database size due to their limited memory capacity. In this paper, we present AltoPIR, the first PIR system that leverages on-CPU matrix accelerators such as Intel AMX. AltoPIR keeps the accelerator fed from the CPU caches, reading each database byte from memory once for all queries in a batch and never writing intermediate products to memory. This enables high-performance PIR through efficient batching without requiring discrete accelerators. Our evaluation shows that AltoPIR achieves up to 50.1x higher throughput than existing CPU-based solutions and serves databases larger than a GPU's memory.

## Causal Attention on CPU: A Cache-Aware and Load-Balanced Scheduling Kernel for Prefill

DevFeed: [Causal Attention on CPU: A Cache-Aware and Load-Balanced Scheduling Kernel for Prefill](<https://devfeed.tech/articles/causal-attention-on-cpu-a-cache-aware-and-load-balanced-scheduling-kernel-for-prefill-108728.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38809>)

Author: Kwun Fung Lau, Frost Mitchell, Kanya Mo, Emani Ashok

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [scheduling](<https://devfeed.tech/topics/scheduling.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [attention](<https://devfeed.tech/tags/attention.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [prefill](<https://devfeed.tech/tags/prefill.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>)

### AI overview

CACPU targets CPU execution of causal full-sequence attention during prefilling. It combines tiered cache blocking to suit multi-level caches with in-kernel task dispatching to address thread scheduling. The abstract reports up to 2.01x speedup over PyTorch SDPA, FlexAttention, and SGLang attention kernels; when integrated into SGLang, it reports up to 1.74x lower Time to First Token for Llama-3.1-8B while preserving model accuracy. These are reported peak results; the excerpt does not provide workload details or broader performance comparisons.

### Source excerpt

arXiv:2610.38809v1 Announce Type: new Abstract: Agentic AI drives significant productivity gains across diverse domains, but concurrent long-context prefills from multiple agents can saturate GPU compute resources. Although CPUs offer a viable alternative, existing CPU attention kernels fall short in adapting to multi-tier cache hierarchies and lack effective thread scheduling. To address this, we introduce CACPU, a high-performance attention kernel featuring tiered cache blocking and an in-kernel task dispatching mechanism designed for causal full-sequence prefilling. CACPU achieves up to a 2.01x speedup over PyTorch SDPA, FlexAttention, and SGLang attention kernels. Integrated into SGLang, it reduces Time to First Token (TTFT) of Llama-3.1-8B by up to 1.74x while preserving model accuracy.

## A report from the LPC sched\_ext microconference

DevFeed: [A report from the LPC sched\_ext microconference](<https://devfeed.tech/articles/a-report-from-the-lpc-sched-ext-microconference-105707.md>)

Original publisher: [Read original article](<https://lwn.net/Articles/1103426/>)

Author: corbet

Published: 2026-10-28T22:11:43Z

Content type: news

Language: en

Sources: [LWN.net](<https://devfeed.tech/sources/lwn-net.md>)

Topics: [scheduling](<https://devfeed.tech/topics/scheduling.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [blog](<https://devfeed.tech/tags/blog.md>), [cache](<https://devfeed.tech/tags/cache.md>), [conference](<https://devfeed.tech/tags/conference.md>), [device](<https://devfeed.tech/tags/device.md>), [discussions](<https://devfeed.tech/tags/discussions.md>), [engine](<https://devfeed.tech/tags/engine.md>), [execution](<https://devfeed.tech/tags/execution.md>), [extension](<https://devfeed.tech/tags/extension.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [intel](<https://devfeed.tech/tags/intel.md>), [linux](<https://devfeed.tech/tags/linux.md>), [locality](<https://devfeed.tech/tags/locality.md>), [lock](<https://devfeed.tech/tags/lock.md>), [main-topics](<https://devfeed.tech/tags/main-topics.md>), [policy](<https://devfeed.tech/tags/policy.md>), [priority](<https://devfeed.tech/tags/priority.md>), [priority-inversion](<https://devfeed.tech/tags/priority-inversion.md>), [report](<https://devfeed.tech/tags/report.md>), [sched-ext](<https://devfeed.tech/tags/sched-ext.md>)

### AI overview

The sched_ext microconference report highlights three discussion areas: addressing priority inversion through lock holder and waiter preemption, time slice extension, and proxy execution with BPF schedulers; coordinating host and guest schedulers across virtual machine boundaries, with sched_ext as the host's policy engine; and improving task placement using Intel HFI feedback, device and accelerator locality, and cache topology.

### Source excerpt

The Linux Plumbers Conference blog has a report from the sched_ext microconference held in Prague. Three main topics dominated the discussions Priority inversion and how to handle it in sched_ext: lock holder and lock waiter preemption, time slice extension, and making proxy execution work together with BPF schedulers. Paravirtualized scheduling: letting the host and guest schedulers cooperate across the VM boundary, with sched_ext as the policy engine on the host. Device and topology-aware scheduling: using hardware feedback (Intel HFI), device/accelerator locality and cache topology to drive task placement.

## SecDMP: Provably Secure Data Memory-Dependent Prefetching (Extended Version)

DevFeed: [SecDMP: Provably Secure Data Memory-Dependent Prefetching (Extended Version)](<https://devfeed.tech/articles/secdmp-provably-secure-data-memory-dependent-prefetching-extended-version-103381.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37454>)

Author: Jens Sprengers, Marton Bognar, Lesly-Ann Daniel, Frank Piessens

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Cryptography](<https://devfeed.tech/topics/cryptography.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [attacks](<https://devfeed.tech/tags/attacks.md>), [classification](<https://devfeed.tech/tags/classification.md>), [complete](<https://devfeed.tech/tags/complete.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [memory](<https://devfeed.tech/tags/memory.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [performance](<https://devfeed.tech/tags/performance.md>), [risc-v](<https://devfeed.tech/tags/risc-v.md>), [secrets](<https://devfeed.tech/tags/secrets.md>), [security](<https://devfeed.tech/tags/security.md>), [semantics](<https://devfeed.tech/tags/semantics.md>), [side-channel](<https://devfeed.tech/tags/side-channel.md>), [spectre](<https://devfeed.tech/tags/spectre.md>)

### AI overview

DMPs can leak secrets through side channels even in constant-time code hardened against Spectre. The paper classifies DMP leakage and reports that its RISC-V experiments found the existing defense designed to preserve performance leaves some leakage classes unprotected. SecDMP requires secrets to be marked in memory and prevents DMPs from encoding them in microarchitectural state, while preserving performance for public data. The authors formally prove it adds no leakage beyond a baseline without DMPs; combined with speculative-execution defenses, it restores complete microarchitectural security for constant-time code. Their evaluation reports that SecDMP prevents all DMP leakage with minimal performance overhead and hardware cost.

### Source excerpt

arXiv:2610.37454v1 Announce Type: new Abstract: Data memory-dependent prefetchers (DMPs) are a microarchitectural optimization recently introduced in mainstream desktop and server CPUs. Although DMPs offer performance improvements, they have been shown to enable side-channel leakage of secrets, even from constant-time code that has been hardened against Spectre attacks. Unfortunately, existing defenses either incur significant performance overhead or, as we show, fail to prevent all DMP leakage. In this work, we address this gap by designing a defense that is both secure and efficient. First, we propose a classification of DMP leakage, categorize existing and novel attacks, and analyze which leakage classes are mitigated by existing defenses. We validate this analysis empirically by implementing the first open-source DMPs on a RISC-V processor, showing that the only existing defense that aims to preserve DMP performance fails to protect against entire leakage classes. Next, we propose SecDMP, a provably secure and efficient hardware-based defense. SecDMP requires secrets to be marked in memory and ensures that DMPs never encode these secrets in the microarchitecture---without compromising performance for public data. We formally prove that SecDMP introduces no leakage beyond a baseline semantics without DMPs. Notably, when combined with speculative-execution defenses, SecDMP restores complete microarchitectural security for constant-time code. Finally, our empirical evaluation shows that SecDMP prevents all DMP leakage while incurring minimal performance overhead and hardware cost.

## Linux Fixes AMD Ryzen AI 9 HX 370 Regression Due To Cache Aware Scheduling

DevFeed: [Linux Fixes AMD Ryzen AI 9 HX 370 Regression Due To Cache Aware Scheduling](<https://devfeed.tech/articles/linux-fixes-amd-ryzen-ai-9-hx-370-regression-due-to-cache-aware-scheduling-100409.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/Linux-CAS-Regression-AMD-HX-370>)

Author: Michael Larabel

Published: 2026-10-26T10:22:28Z

Content type: news

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [scheduling](<https://devfeed.tech/topics/scheduling.md>), [Linux Kernel](<https://devfeed.tech/topics/linux-kernel.md>), [Linux](<https://devfeed.tech/topics/linux.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [bugfixes](<https://devfeed.tech/topics/bugfixes.md>)

Tags: [3ghz](<https://devfeed.tech/tags/3ghz.md>), [7-4](<https://devfeed.tech/tags/7-4.md>), [amd](<https://devfeed.tech/tags/amd.md>), [amd-ryzen](<https://devfeed.tech/tags/amd-ryzen.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [bug](<https://devfeed.tech/tags/bug.md>), [builds](<https://devfeed.tech/tags/builds.md>), [cache](<https://devfeed.tech/tags/cache.md>), [clang](<https://devfeed.tech/tags/clang.md>), [code](<https://devfeed.tech/tags/code.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [core](<https://devfeed.tech/tags/core.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [linux](<https://devfeed.tech/tags/linux.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [porting](<https://devfeed.tech/tags/porting.md>), [regression](<https://devfeed.tech/tags/regression.md>), [ryzen-ai](<https://devfeed.tech/tags/ryzen-ai.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [soc](<https://devfeed.tech/tags/soc.md>), [stable](<https://devfeed.tech/tags/stable.md>), [strix-point](<https://devfeed.tech/tags/strix-point.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>)

### AI overview

On the AMD Ryzen AI 9 HX 370, Cache Aware Scheduling regressed full-LTO Clang builds by keeping cache-intensive tasks on slower Zen 5C cores. Those cores reach 3.3 GHz and have 8 MB of L3 cache, versus up to about 5.1 GHz and 16 MB on the faster Zen 5 cores. The Linux scheduler fix prioritizes asymmetric packing when it can move a task to a higher-priority idle CPU, overriding LLC co-location during regular and active load balancing. The patch was merged for Linux 7.4 and marked for back-porting to 7.2 and 7.3; it may also help other Ryzen chips with mixed Zen 5/5C cores.

### Source excerpt

Merged for Linux 7.4 Git and marked for back-porting to Linux 7.2/7.3 stable is a regression fix affecting at least the AMD Ryzen AI 9 HX 370 "Strix Point" laptop SoC when making use of Cache Aware Scheduling...

## CAFS: A Cache-Aware Frequency Sort for Low-Cardinality Integer Data on x86-64

DevFeed: [CAFS: A Cache-Aware Frequency Sort for Low-Cardinality Integer Data on x86-64](<https://devfeed.tech/articles/cafs-a-cache-aware-frequency-sort-for-low-cardinality-integer-data-on-x86-64-93870.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.25040>)

Author: Vasiliy S. Shlyk

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [vectorization techniques](<https://devfeed.tech/topics/vectorization-techniques.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [x86](<https://devfeed.tech/topics/x86.md>), [olap](<https://devfeed.tech/topics/olap.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [array](<https://devfeed.tech/tags/array.md>), [avx2](<https://devfeed.tech/tags/avx2.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cardinality](<https://devfeed.tech/tags/cardinality.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cs-ds](<https://devfeed.tech/tags/cs-ds.md>), [olap](<https://devfeed.tech/tags/olap.md>), [open-addressing](<https://devfeed.tech/tags/open-addressing.md>), [operational](<https://devfeed.tech/tags/operational.md>), [pdqsort](<https://devfeed.tech/tags/pdqsort.md>), [radix-sort](<https://devfeed.tech/tags/radix-sort.md>), [simd](<https://devfeed.tech/tags/simd.md>), [ska-sort](<https://devfeed.tech/tags/ska-sort.md>), [sort](<https://devfeed.tech/tags/sort.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [x86-64](<https://devfeed.tech/tags/x86-64.md>)

### AI overview

CAFS targets integer arrays with many values but low cardinality, common in OLAP workloads. On x86-64 with AVX2, it combines a cache-line-sized SIMD bucket, a Chao1 estimator using 1,024 strided samples, and an adaptive dispatcher with a spill safety guard; its branchless hot loop uses cmpeq, movemask, and tzcnt. Across 592,770 benchmark runs, CAFS reached 1.7-3.1x pdqsort, 1.7-3.5x IPS4o, and 1.2-2.3x vqsort throughput when K was much smaller than N. The reported crossover is about K=130,000 for pdqsort and 670,000 for vqsort; only vqsort overtook CAFS beyond its crossover, making that threshold the stated limit of CAFS's operational range. The evidence is an abstract-level summary of benchmarks, not full paper coverage.

### Source excerpt

arXiv:2605.25040v2 Announce Type: replace-cross Abstract: Integer sorts in OLAP engines often run on columns whose cardinality $K$ is much smaller than the array length $N$. After a group-by stage the intermediate key column has $K$ bounded by the number of distinct group keys, and even a column-store scan typically operates on dictionary-encoded categorical fields where $K$ never exceeds a few thousand. A comparison sort on such a column still pays $\Theta(N \log N)$ comparisons, and a radix sort still pays $\Theta(N \cdot B/b)$ byte passes, irrespective of $K$. This paper describes CAFS, an integer sort that does exploit it on x86-64 with AVX2. The algorithm combines a SIMD bucket sized to one cache line, a Chao1 cardinality estimator over 1024 strided samples (kept in a heap-allocated 40 KB open-addressing table), and an adaptive dispatcher backed by a spill safety guard. The hot loop is branchless and uses AVX2 cmpeq together with movemask and tzcnt to locate the matching lane. We benchmarked CAFS on a full-factorial grid of 58 array sizes $N$ from $10^3$ to $3 \cdot 10^7$ with dense $K$ schedules per $N$, producing 592770 timed runs against pdqsort, IPS4o, vqsort, ska_sort, and std::sort. In the $K \ll N$ band the throughput is 1.7 to 3.1x that of pdqsort, 1.7 to 3.5x IPS4o, and 1.2 to 2.3x vqsort. The operational crossover against pdqsort is at $K \approx 1.3 \cdot 10^5$; against ska_sort, $K \approx 8.14 \cdot 10^5$; against vqsort, $K \approx 6.7 \cdot 10^5$; and against IPS4o the curves only converge near $K = N$. Of the five baselines, only vqsort actually overtakes CAFS once the crossover is passed, which makes the vqsort threshold at $K \approx 6.7 \cdot 10^5$ the binding constraint on the operational range of CAFS.

## Weight Placement Is Not Enough: Synchronization in Multi-Node CPU Decoding of Ternary LLMs

DevFeed: [Weight Placement Is Not Enough: Synchronization in Multi-Node CPU Decoding of Ternary LLMs](<https://devfeed.tech/articles/weight-placement-is-not-enough-synchronization-in-multi-node-cpu-decoding-of-ternary-llms-93757.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.28620>)

Author: Runguo Li, Dezhen Lu

Published: 2026-10-21T04:00:00Z

Content type: news

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [synchronization](<https://devfeed.tech/topics/synchronization.md>), [NUMA](<https://devfeed.tech/topics/numa.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [column](<https://devfeed.tech/tags/column.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [dram](<https://devfeed.tech/tags/dram.md>), [energy](<https://devfeed.tech/tags/energy.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [l3-cache](<https://devfeed.tech/tags/l3-cache.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llms](<https://devfeed.tech/tags/llms.md>), [local](<https://devfeed.tech/tags/local.md>), [megatron](<https://devfeed.tech/tags/megatron.md>), [model](<https://devfeed.tech/tags/model.md>), [numa](<https://devfeed.tech/tags/numa.md>), [split](<https://devfeed.tech/tags/split.md>), [synchronization](<https://devfeed.tech/tags/synchronization.md>), [thread](<https://devfeed.tech/tags/thread.md>), [token](<https://devfeed.tech/tags/token.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

On dual-socket, four-NUMA-node Xeon servers, placing ternary LLM weights with the threads that read them does not prevent substantial decode-time synchronization: threads wait at synchronization points for 20-48% of each token. Traces attribute most of that time to slower threads within each node, while barrier latency and serial per-node work contribute less. With identical kernels, a Megatron-style column split outperformed a row-owned split without cross-node reductions in every measured configuration; most of its gain came from reduced waiting, rather than fewer barriers. The resulting engine was 1.5-2.1x faster than ik_llama.cpp using all cores and 1.5-1.9x faster than its fastest thread-count setting for DRAM-bound decoding, with the same accuracy and less energy per token. A small L3-resident model saw larger speedups. Findings are limited to the measured configurations and two Xeon generations.

### Source excerpt

arXiv:2610.28620v1 Announce Type: new Abstract: Ternary LLMs read only a few gigabytes per decoded token, yet on dual-socket servers with four NUMA nodes the fastest ternary runtime we found uses less than half of the memory bandwidth. Placing each weight on the node of the thread that reads it is necessary but not enough: in a decode engine with node-local weights that reuses ik_llama.cpp's kernels, threads on two generations of Xeon servers still spend 20-48% of each token waiting at synchronization points. Per-thread barrier traces show that most of the wait is for the slowest threads of each node; the barriers' own latency (microseconds each, but up to 12% of the token for the smallest model) and serial per-node work are smaller parts. On identical kernels, a Megatron-style column split beats a row-owned split without cross-node reductions in every configuration we measured, and isolating the number of barriers shows that most of its gain comes from its threads waiting less for each other and for their node leaders, and only a minority from having fewer barriers. Where decoding is DRAM-bound, the resulting engine is 1.5-2.1x faster than ik_llama.cpp with all cores and 1.5-1.9x faster than ik_llama.cpp at its fastest thread count (3.2x and 2.8x for a small model that runs from the L3 cache), at the same accuracy and with less energy per token.

## Linux removes unnecessary cache flushing for Intel TDX

DevFeed: [Linux removes unnecessary cache flushing for Intel TDX](<https://devfeed.tech/articles/it-turns-out-intel-tdx-doesn-t-need-so-much-cache-flushing-89743.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/Intel-TDX-Less-Cache-Flushing>)

Author: Michael Larabel

Published: 2026-10-18T13:53:24Z

Content type: news

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [Linux](<https://devfeed.tech/topics/linux.md>), [Linux Kernel](<https://devfeed.tech/topics/linux-kernel.md>), [Nitro Isolation Engine](<https://devfeed.tech/topics/nitro-isolation-engine.md>), [Xeon 600](<https://devfeed.tech/topics/xeon-600.md>), [Intel Core](<https://devfeed.tech/topics/intel-core.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [IPython](<https://devfeed.tech/topics/ipython.md>), [NVS Encryption](<https://devfeed.tech/topics/nvs-encryption.md>), [cross-origin-storage](<https://devfeed.tech/topics/cross-origin-storage.md>), [cos](<https://devfeed.tech/topics/cos.md>), [NUMA](<https://devfeed.tech/topics/numa.md>)

Tags: [computing](<https://devfeed.tech/tags/computing.md>), [confidential-computing](<https://devfeed.tech/tags/confidential-computing.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [encryption](<https://devfeed.tech/tags/encryption.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [intel-xeon](<https://devfeed.tech/tags/intel-xeon.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [linux](<https://devfeed.tech/tags/linux.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-kernel](<https://devfeed.tech/tags/linux-kernel.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [processor](<https://devfeed.tech/tags/processor.md>), [software](<https://devfeed.tech/tags/software.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>), [xeon](<https://devfeed.tech/tags/xeon.md>)

### AI overview

Linux kernel patches remove software-driven cache flushing around Intel TDX memory transitions because finalized Xeon hardware handles the issue without software assistance. The change simplifies kernel code; the patch messages do not state an expected performance impact.

### Source excerpt

Intel Trust Domain Extensions (TDX) for confidential computing with VMs has assumed the need to perform cache flushing when transitioning memory between KeyIDs used by the total memory encryption (MKTME) for unique AES encryption keys. But it's turned out to be unneeded and the Linux kernel is working now to reduce the excessive cache flushing...

## How Modern Computer Memory Uses Caches, Virtual Memory, and Compressed RAM

DevFeed: [How Modern Computer Memory Uses Caches, Virtual Memory, and Compressed RAM](<https://devfeed.tech/articles/your-computer-memory-is-a-lie-85335.md>)

Original publisher: [Read original article](<https://hackaday.com/2026/10/12/your-computer-memory-is-a-lie/>)

Author: Al Williams

Published: 2026-10-12T14:00:11Z

Content type: article

Language: en

Sources: [Hackaday](<https://devfeed.tech/sources/hackaday.md>)

Topics: [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [NUMA](<https://devfeed.tech/topics/numa.md>), [DRAM price increase](<https://devfeed.tech/topics/dram-price-increase.md>), [upgrade](<https://devfeed.tech/topics/upgrade.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [buffer](<https://devfeed.tech/tags/buffer.md>), [cache](<https://devfeed.tech/tags/cache.md>), [compression](<https://devfeed.tech/tags/compression.md>), [computing](<https://devfeed.tech/tags/computing.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cram](<https://devfeed.tech/tags/cram.md>), [cxl](<https://devfeed.tech/tags/cxl.md>), [dram](<https://devfeed.tech/tags/dram.md>), [featured](<https://devfeed.tech/tags/featured.md>), [interest](<https://devfeed.tech/tags/interest.md>), [memory](<https://devfeed.tech/tags/memory.md>), [mmu](<https://devfeed.tech/tags/mmu.md>), [rants](<https://devfeed.tech/tags/rants.md>), [swapfile](<https://devfeed.tech/tags/swapfile.md>), [zram](<https://devfeed.tech/tags/zram.md>), [zswap](<https://devfeed.tech/tags/zswap.md>)

### AI overview

Modern memory systems hide a hierarchy of CPU caches, virtual address translation, NUMA, filesystem caching, and persistent storage behind a simple byte-addressed model. The article also describes Linux zram and zswap, then introduces CRAM, a developmental project that uses hardware-assisted compression exposed as a special NUMA node. Reported benchmarks show promising results for selected workloads, while the project remains unsuitable for everyday use.

### Source excerpt

When you first learn to program, memory seems pretty straightforward. You have a bunch of bytes, each with an address. If you want to read or write something, you specify ...read more

## \[$\] Evolving the LAVD scheduler from gaming to servers

DevFeed: [\[$\] Evolving the LAVD scheduler from gaming to servers](<https://devfeed.tech/articles/evolving-the-lavd-scheduler-from-gaming-to-servers-66591.md>)

Original publisher: [Read original article](<https://lwn.net/Articles/1097209/>)

Author: corbet

Published: 2026-10-07T13:33:48Z

Content type: news

Language: en

Sources: [LWN.net](<https://devfeed.tech/sources/lwn-net.md>)

Topics: [scheduling](<https://devfeed.tech/topics/scheduling.md>), [Linux Kernel](<https://devfeed.tech/topics/linux-kernel.md>), [Kernel](<https://devfeed.tech/topics/kernel.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [applications](<https://devfeed.tech/tags/applications.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cpus](<https://devfeed.tech/tags/cpus.md>), [data-center](<https://devfeed.tech/tags/data-center.md>), [eevdf](<https://devfeed.tech/tags/eevdf.md>), [energy](<https://devfeed.tech/tags/energy.md>), [gaming](<https://devfeed.tech/tags/gaming.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [scheduler](<https://devfeed.tech/tags/scheduler.md>), [servers](<https://devfeed.tech/tags/servers.md>)

### AI overview

LAVD, a BPF-based scheduler that began as a way to reduce gaming stutter on Steam Deck, prioritizes tail latency by estimating each task's urgency from how often it waits for and wakes other tasks and its average run time. It adjusts virtual deadlines by criticality, favors less critical work for preemption, and uses futex activity to avoid preempting lock holders. CPU placement considers cache domains, task criticality, CPU load and interrupt activity; cache warmth helps weigh migration costs. To limit power use, LAVD uses the kernel energy model to select how many CPUs to keep active, while assigning performance-critical tasks to faster CPUs. The scheduler has since expanded toward server workloads. Control-group support is in development using eventual, longer-term usage enforcement; AI workload support and upstreaming ideas to EEVDF remain future work.

### Source excerpt

The extensible scheduler class, which enables the creation of custom CPU schedulers with BPF, has led to a burst of innovation in this area; the LAVD scheduler has, perhaps, been one of the most noteworthy schedulers to emerge. Though it was originally designed for gaming applications, the LAVD scheduler has since grown to serve other types of workloads as well. At the 2026 edition of Kernel Recipes, Changwoo Min and Gavin Guo presented an overview of this scheduler and how it has evolved over time.

## Programmable Memory Controller Eases Adoption of New DRAM Techniques (ETH Zürich, CISPA, NYU)

DevFeed: [Programmable Memory Controller Eases Adoption of New DRAM Techniques (ETH Zürich, CISPA, NYU)](<https://devfeed.tech/articles/programmable-memory-controller-eases-adoption-of-new-dram-techniques-eth-zurich-cispa-nyu-66450.md>)

Original publisher: [Read original article](<https://semiengineering.com/programmable-memory-controller-eases-adoption-of-new-dram-techniques-eth-zurich-cispa-nyu/>)

Author: Technical Paper Link

Published: 2026-10-06T20:58:05Z

Content type: news

Language: en

Sources: [Semiconductor Engineering](<https://devfeed.tech/sources/semiconductor-engineering.md>)

Topics: [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>)

Tags: [architectures](<https://devfeed.tech/tags/architectures.md>), [cispa](<https://devfeed.tech/tags/cispa.md>), [ddr5](<https://devfeed.tech/tags/ddr5.md>), [dram](<https://devfeed.tech/tags/dram.md>), [dram-reliability](<https://devfeed.tech/tags/dram-reliability.md>), [eth-zurich](<https://devfeed.tech/tags/eth-zurich.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-architecture](<https://devfeed.tech/tags/memory-architecture.md>), [memory-controllers](<https://devfeed.tech/tags/memory-controllers.md>), [memory-interface](<https://devfeed.tech/tags/memory-interface.md>), [memory-latency](<https://devfeed.tech/tags/memory-latency.md>), [memory-maintenance](<https://devfeed.tech/tags/memory-maintenance.md>), [new-york-university](<https://devfeed.tech/tags/new-york-university.md>), [performance](<https://devfeed.tech/tags/performance.md>), [power-and-performance](<https://devfeed.tech/tags/power-and-performance.md>), [power-performance](<https://devfeed.tech/tags/power-performance.md>), [processing-in-memory](<https://devfeed.tech/tags/processing-in-memory.md>), [processing-using-dram](<https://devfeed.tech/tags/processing-using-dram.md>), [programmable-memory-controller](<https://devfeed.tech/tags/programmable-memory-controller.md>), [semiconductor](<https://devfeed.tech/tags/semiconductor.md>), [subarray-parallelism](<https://devfeed.tech/tags/subarray-parallelism.md>), [systems](<https://devfeed.tech/tags/systems.md>), [terracotta](<https://devfeed.tech/tags/terracotta.md>), [volatile-memory](<https://devfeed.tech/tags/volatile-memory.md>)

### AI overview

Researchers at ETH Zürich, CISPA, and New York University present Terracotta, a flexible DRAM interface and programmable memory controller designed to make new DRAM techniques easier to deploy. The framework uses configurable command extensions and post-silicon controller programming to reduce repeated hardware modifications.

### Source excerpt

Researchers at ETH Zürich, CISPA, and New York University published a technical paper titled "Terracotta: Enabling the Adoption of New DRAM Techniques via a Flexible DRAM Interface and Memory Controller." Abstract Excerpt: "DRAM continues to limit the performance, energy efficiency, and robustness of modern systems. Many prior works propose DRAM techniques that support in-DRAM computation,... " read more The post Programmable Memory Controller Eases Adoption of New DRAM Techniques (ETH Zürich, CISPA, NYU) appeared first on Semiconductor Engineering.

## Cartridges: A Cache You Train, Not a Context You Fill

DevFeed: [Cartridges: A Cache You Train, Not a Context You Fill](<https://devfeed.tech/articles/cartridges-a-cache-you-train-not-a-context-you-fill-78496.md>)

Original publisher: [Read original article](<https://blog.everpuredata.com/perspectives/cartidges-a-cache-you-train-not-a-context-you-fill/>)

Author: Noah Kasmanoff

Published: 2026-10-02T13:00:00Z

Content type: article

Language: en

Sources: [Everpure](<https://devfeed.tech/sources/everpure.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [Cache Coherence](<https://devfeed.tech/topics/cache-coherence.md>), [CPU Cache](<https://devfeed.tech/topics/cpu-cache.md>), [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>)

Tags: [ai-and-machine-learning](<https://devfeed.tech/tags/ai-and-machine-learning.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [costs](<https://devfeed.tech/tags/costs.md>), [perspectives](<https://devfeed.tech/tags/perspectives.md>)

### AI overview

The article explains cartridges, a research approach that trains compact key-value caches offline and reuses them to support long-context AI inference with less GPU memory. It compares cartridges with in-context learning, retrieval, and fine-tuning, and discusses self-study training, reported benchmark results, and limits including model-specific training and static documents. The article says cartridges remain a research result and are not ready for enterprise-scale use.

### Source excerpt

Cartridges: A Cache You Train, Not a Context You Fill by Everpure Blog This article explores what cartridges are, how they use compact KV caches to streamline long-context inference, and how they can help reduce resource demands. The post Cartridges: A Cache You Train, Not a Context You Fill appeared first on Everpure Blog.

[Next page](<https://devfeed.tech/topics/cpu-cache.md?cursor=WyIyMDI2LTEwLTAyVDEzOjAwOjAwKzAwOjAwIiwgImFlZDQzYjVkLTc0YTQtNDgwNy1hYzNjLTlmY2JjZGExNTZkZSJd>)