# l3 cache Published articles for l3 cache. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Weight Placement Is Not Enough: Synchronization in Multi-Node CPU Decoding of Ternary LLMs DevFeed: [Weight Placement Is Not Enough: Synchronization in Multi-Node CPU Decoding of Ternary LLMs]() Original publisher: [Read original article]() Author: Runguo Li, Dezhen Lu Published: 2026-10-21T04:00:00Z Content type: news Language: en Sources: [arXiv: Distributed Computing]() Topics: [synchronization](), [NUMA](), [CPU Cache]() Tags: [bandwidth](), [column](), [cpu](), [cross](), [cs-dc](), [decoding](), [dram](), [energy](), [kernels](), [l3-cache](), [latency](), [llms](), [local](), [megatron](), [model](), [numa](), [split](), [synchronization](), [thread](), [token](), [weight]() ### AI overview On dual-socket, four-NUMA-node Xeon servers, placing ternary LLM weights with the threads that read them does not prevent substantial decode-time synchronization: threads wait at synchronization points for 20-48% of each token. Traces attribute most of that time to slower threads within each node, while barrier latency and serial per-node work contribute less. With identical kernels, a Megatron-style column split outperformed a row-owned split without cross-node reductions in every measured configuration; most of its gain came from reduced waiting, rather than fewer barriers. The resulting engine was 1.5-2.1x faster than ik_llama.cpp using all cores and 1.5-1.9x faster than its fastest thread-count setting for DRAM-bound decoding, with the same accuracy and less energy per token. A small L3-resident model saw larger speedups. Findings are limited to the measured configurations and two Xeon generations. ### Source excerpt arXiv:2610.28620v1 Announce Type: new Abstract: Ternary LLMs read only a few gigabytes per decoded token, yet on dual-socket servers with four NUMA nodes the fastest ternary runtime we found uses less than half of the memory bandwidth. Placing each weight on the node of the thread that reads it is necessary but not enough: in a decode engine with node-local weights that reuses ik_llama.cpp's kernels, threads on two generations of Xeon servers still spend 20-48% of each token waiting at synchronization points. Per-thread barrier traces show that most of the wait is for the slowest threads of each node; the barriers' own latency (microseconds each, but up to 12% of the token for the smallest model) and serial per-node work are smaller parts. On identical kernels, a Megatron-style column split beats a row-owned split without cross-node reductions in every configuration we measured, and isolating the number of barriers shows that most of its gain comes from its threads waiting less for each other and for their node leaders, and only a minority from having fewer barriers. Where decoding is DRAM-bound, the resulting engine is 1.5-2.1x faster than ik_llama.cpp with all cores and 1.5-1.9x faster than ik_llama.cpp at its fastest thread count (3.2x and 2.8x for a small model that runs from the L3 cache), at the same accuracy and with less energy per token. ## A Renter's Guide to Sole Tenancy in GCE and GKE DevFeed: [A Renter's Guide to Sole Tenancy in GCE and GKE]() Original publisher: [Read original article]() Author: Fed Mestrone Published: 2026-10-19T14:35:25Z Content type: tutorial Language: en Sources: [Google Developer Experts]() Topics: [Gke](), [Google Cloud Platform (GCP)](), [cloud-infrastructure](), [virtual machines](), [virtualization](), [performance-engineering]() Tags: [attack](), [bare-metal](), [compliance](), [cpu](), [critical](), [crypto](), [crypto-mining](), [database](), [gaming](), [gce](), [gke](), [google](), [google-cloud](), [google-cloud-platform](), [guide](), [hardware](), [high-frequency-trading](), [infrastructure](), [isolation](), [l3-cache](), [latency](), [tenancy]() ### AI overview Google Cloud sole tenancy dedicates physical hosts to workloads on GCE or GKE Standard, addressing physical-isolation requirements, core-based BYOL licensing, and possible noisy-neighbor performance effects. GCE uses node templates and zone-specific node groups; VMs attach through node affinity, with maintenance options to migrate within the group, restart in place, or move to shared hosts. GKE Standard can place node pools on a node group or use sole-tenant affinity. Autopilot does not support sole tenancy, though the text notes GKE Sandbox as an alternative isolation method. With GKE, Cluster Autoscaling can add worker VMs and the Compute Engine Node Group Autoscaler can add physical hosts, but provisioning may be delayed or fail if capacity, limits, or quotas prevent expansion; node auto-provisioning is unsupported. Hosts are charged in full regardless of utilization, and fragmentation can occur both between VMs on hosts and within VMs among Pods, making resource requests, sizing, scheduling, and consolidation important. ### Source excerpt GCE, GKE and Sole TenancyWhat do property renters and Google Cloud users have in common? The solution to disturbing neighbours: rent the whole building! You have probably been there before, or know someone who has: you finally move into your new dream flat, everything is perfect, you saw so many places before, until this one finally resonated with you on all chords, and now you're living in it... But you're hardly a month into your tenancy and it's clear that your upstairs neighbour is a heavy-metal drummer (who loves to practice at 2:00 AM), the neighbour to your left must be running an industrial crypto-mining rig (that warms your living room wall to a crisp 27°C), and the kids downstairs provide louder entertainment than your own TV. Welcome to standard public cloud infrastructure! Sharing physical bare-metal hardware with third-party companies (technically known as multi-tenancy) is somewhat similar to renting a flat in a building where strangers live in lots of other flats... And ok, I admit it, I may have been stretching the scenario a bit: most people's neighbours are lovely and reasonable -- in the same way, running VMs on shared hardware is for the most part not a problem. However, there may be cases even in cloud computing where sometimes you just wish you did not have anyone else around you. Enter Sole Tenancy on Google Cloud. Why Do We Need Sole Tenancy? In the situation described earlier on, the neighbours' behaviours were such that they'd make anyone wish they had the entire building to themselves. So what would be the triggers in a cloud environment? The "Noisy Neighbour" Syndrome: High-throughput, latency-critical workloads (think high-frequency trading, real-time gaming engines, or massive database engines) don't like L3 cache contention or CPU stealing. Security Risks: several possible attack classes may affect VMs in shared hardware (side-channel attacks, memory remanence, resource exhaustion, ...). Google uses multiple layers of isolation, authenticatio ## How did AMD Ryzen get 50% faster in two years? DevFeed: [How did AMD Ryzen get 50% faster in two years?]() Original publisher: [Read original article]() Author: Daniel Lemire Published: 2026-09-18T17:26:04Z Content type: article Language: en Sources: [Daniel Lemire]() Topics: [cpu](), [Hardware](), [Parallelism](), [Cache]() Tags: [2022](), [3d](), [amd](), [core](), [cpu](), [desktop](), [epyc](), [geekbench](), [hardware](), [l2-cache](), [l3-cache](), [performance](), [processor](), [processors](), [ryzen](), [science-and-technology](), [september-2026](), [simd](), [venice]() ### AI overview The article compares AMD Ryzen 7 processors from 2022 to 2024 and explains how architectural changes produced roughly 50% higher Geekbench 6 performance. It highlights wider instruction dispatch, larger caches, more arithmetic units, a larger reorder buffer, and wider SIMD units, then briefly discusses AMD's planned 256-core Epyc Venice processor. ### Source excerpt People still tell me that CPUs are boring. That nothing much happens anymore. Let us look at AMD Ryzen 7 processors from 2022 to 2024: the 5800X3D (Zen 3), the 7800X3D (Zen 4) and the 9800X3D (Zen 5). They are comparable 8-core chips with 3D V-Cache. I have written about them before, in How stagnant ... Continue reading How did AMD Ryzen get 50% faster in two years? ## How To Smash The Memory Wall Plaguing High Performance Systems DevFeed: [How To Smash The Memory Wall Plaguing High Performance Systems]() Original publisher: [Read original article]() Author: Cristian Vasile Published: 2026-09-16T13:37:11Z Content type: tutorial Language: en Sources: [The Next Platform: In-depth coverage of high end computing]() Topics: [systems](), [cpu](), [data](), [Cache](), [Computing](), [parquet](), [Algorithms](), [computational complexity](), [The Chapel Parallel Programming Language]() Tags: [architecture](), [cache](), [cpu](), [distributed](), [high-performance](), [l2-cache](), [l3-cache](), [latency](), [memory](), [parallel](), [parquet](), [performance](), [store]() ### AI overview The article examines memory and cache-performance bottlenecks in high-performance systems processing large analytical datasets. It proposes exploring Z-Order (Morton Layout) memory addressing as an alternative to linear RAM addressing and uses AMD's Turin Epyc 9005 architecture as a theoretical baseline. ### Source excerpt In my day-to-day work, I process massive data volumes, wrestling with large-scale .parquet files, co ... ## L3 cache mapping on Sandy Bridge CPUs DevFeed: [L3 cache mapping on Sandy Bridge CPUs]() Original publisher: [Read original article]() Author: Mark Seaborn (noreply@blogger.com) Published: 2015-04-27T21:59:00Z Content type: article Language: en Sources: [Mark Seaborn]() Topics: [CPU Cache](), [Cache](), [Cache Coherence]() Tags: [2013](), [aslr](), [attacks](), [bits](), [bus](), [cache](), [cpus](), [distributed](), [dram](), [intel](), [kernel](), [l3-cache](), [map](), [mapping](), [memory](), [ring](), [rowhammer](), [side-channel](), [side-channel-attacks](), [size](), [slices]() ### AI overview On Sandy Bridge, physical address bits 6-16 select one of 2,048 L3 cache sets within a slice, bits 17-31 are used to hash the address to a slice, and bits 0-5 give the offset within a 64-byte cache line; bits 32 and above are unused. The cache has one slice per core, connected by a ring bus, and accessing another core's slice adds one or two hops. Slices are 1.5 MB with 12-way associativity on lower-end CPUs and 2 MB with 16-way associativity on higher-end CPUs. Hund, Willems, and Holz published the mapping for four-core CPUs; the source author reports deriving it for two-core CPUs. The excerpt gives the beginning of the four-core slice hash equations but cuts off before completing them. ### Source excerpt In 2013, some researchers reverse-engineered how Intel Sandy Bridge CPUs map physical addresses to cache sets in the L3 cache (the last-level cache). They were interested in the cache mapping because it can be used to defeat kernel ASLR. I'm interested because the cache mapping can be used to test whether cached memory accesses can do row hammering (which can cause exploitable bit flips in some DRAM devices). The researchers published the details in the paper "Practical Timing Side Channel Attacks Against Kernel Space ASLR" (Ralf Hund, Carsten Willems and Thorsten Holz). They only published the mapping for 4-core CPUs, but I have figured out the mapping for 2-core CPUs as well. Some background: On Sandy Bridge CPUs, the L3 cache is divided into slices. Physical addresses are hashed to determine which slice of the L3 cache they will be stored in. The L3 cache is distributed and ring-based. There is one slice per core, but all the cores in a CPU can access all the cache slices via a ring bus which connects all the cores and their caches together. When a core accesses a memory location, the location will be slightly slower to access if it maps to a different core's cache slice, because it would take one or two hops around the ring bus to access it. The protocol used on the ring bus is based on QPI (Intel's QuickPath Interconnect). (QPI is a protocol used for connecting multiple CPUs together on high-end multi-socket systems.) Each cache slice contains 2048 cache sets. On lower-end CPUs, cache sets are 12-way associative, so a cache slice is 1.5MB in size (2048 sets * 12 ways * 64 bytes per cache line = 1.5MB). On higher-end CPUs, cache sets are 16-way associative, so a cache slice is 2MB in size (2048 sets * 16 ways * 64 bytes per cache line = 2MB). Cache mapping The researchers (Hund et al) figured out that the L3 cache uses the bits of a physical address as follows: Bits 0-5: These give the 6-bit byte offset within a 64-byte cache line. Bits 6-16: These give the 11-b ## Caches: LRU v. random DevFeed: [Caches: LRU v. random]() Original publisher: [Read original article]() Published: 2014-11-03T00:00:00Z Content type: article Language: en Sources: [Danluu]() Topics: [CPU Cache](), [Cache Coherence](), [NUMA](), [Algorithms]() Tags: [algorithm](), [benchmark](), [cache](), [cpu](), [cpu-caches](), [l3-cache](), [lru]() ### AI overview The article compares LRU, random, and multi-choice random cache eviction policies using SPEC CPU benchmarks across cache sizes and levels. LRU performs better in smaller caches, while 2-random and pseudo 3-random can outperform it in larger caches; the results also examine associativity and multi-level cache effects. ### Source excerpt Once upon a time, my computer architecture professor mentioned that using a random eviction policy for caches really isn't so bad. That random eviction isn't bad can be surprising -- if your cache fills up and you have to get rid of something, choosing the least recently used (LRU) is an obvious choice, since you're more likely to use something if you've used it recently. If you have a tight loop, LRU is going to be perfect as long as the loop fits in cache, but it's going to cause a miss every time if the loop doesn't fit. A random eviction policy degrades gracefully as the loop gets too big. In practice, on real workloads, random tends to do worse than other algorithms. But what if we take two random choices (2-random) and just use LRU between those two choices? Here are the relative miss rates we get for SPEC CPU1 with a Sandy Bridge-like cache (8-way associative, 64k, 256k, and 2MB L1, L2, and L3 caches, respectively). These are ratios (algorithm miss rate : random miss rate); lower is better. Each cache uses the same policy at all levels of the cache. Policy L1 (64k) L2 (256k) L3 (2MB) 2-random 0.91 0.93 0.95 FIFO 0.96 0.97 1.02 LRU 0.90 0.90 0.97 random 1.00 1.00 1.00 Random and FIFO are both strictly worse than either LRU or 2-random. LRU and 2-random are pretty close, with LRU edging out 2-random for the smaller caches and 2-random edging out LRU for the larger caches. To see if anything odd is going on in any individual benchmark, we can look at the raw results on each sub-benchmark. The L1, L2, and L3 miss rates are all plotted in the same column for each benchmark, below: As we might expect, LRU does worse than 2-random when the miss rates are high, and better when the miss rates are low. At this point, it's not clear if 2-random is beating LRU in L3 cache miss rates because it does better when the caches are large or because it does better because it's the third level in a hierarchical cache. Since a cache line that's being actively used in L1 or L2 isn't