# NVMe NVMe is a specification and interface defining how host software communicates with non-volatile memory across transports such as PCIe, RDMA, and TCP, and is an industry standard for solid-state drives. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## TrueNAS Releases Native Proxmox VE Storage Plugin for iSCSI and NVMe/TCP DevFeed: [TrueNAS Releases Native Proxmox VE Storage Plugin for iSCSI and NVMe/TCP]() Original publisher: [Read original article]() Author: Lyle Smith Published: 2026-09-17T16:19:04Z Content type: news Language: en Sources: [StorageReview.com]() Topics: [truenas](), [Proxmox](), [NVMe](), [systems](), [Authentication]() Tags: [enterprise](), [plugin](), [proxmox](), [snapshot](), [software](), [storage](), [truenas](), [vm]() ### AI overview TrueNAS released a native Proxmox VE storage plugin that provisions and manages VM disks as OpenZFS zvols on TrueNAS 25.10 or later. The plugin supports iSCSI and NVMe/TCP storage transports and lets administrators manage disk creation, snapshots, resizing, migration, and deletion from Proxmox. ### Source excerpt TrueNAS has released a native Proxmox VE storage plugin that lets Proxmox provision and manage VM disks directly on TrueNAS systems. A disk requested in Proxmox becomes an OpenZFS zvol on TrueNAS 25.10 or later, published as an iSCSI LUN or an NVMe/TCP namespace, and the plugin handles creation, snapshots, resizing, migration, and deletion from The post TrueNAS Proxmox Plugin Turns Every VM Disk Request Into an Automated zvol Over iSCSI or NVMe/TCP appeared first on StorageReview.com. ## TrueNAS Releases a Native Proxmox VE Plugin for Managing VM Storage DevFeed: [TrueNAS Releases a Native Proxmox VE Plugin for Managing VM Storage]() Original publisher: [Read original article]() Author: Brandon Lee Published: 2026-09-16T13:40:55Z Content type: article Language: en Sources: [Virtualization Howto]() Topics: [truenas](), [Proxmox](), [virtualization](), [NVMe]() Tags: [bug](), [home-lab](), [home-server](), [nvme](), [proxmox](), [release](), [testing](), [truenas](), [virtualization](), [zfs]() ### AI overview TrueNAS released an official Proxmox VE storage plugin that lets administrators provision and manage TrueNAS-backed VM disks through Proxmox workflows. The plugin supports iSCSI, NVMe over TCP, and OpenZFS snapshots, but the article describes it as an early-adopter release that is not yet recommended for production workloads. ### Source excerpt It really feels like Proxmox is on fire lately with announcements and momentum behind the solution. We just came off the heels of Proxmox announcing they were bringing on 24x7... The post TrueNAS Just Fixed One of Proxmox's Biggest Storage Headaches appeared first on Virtualization Howto. ## Sandisk 600 and 800 NAS SSDs Revealed DevFeed: [Sandisk 600 and 800 NAS SSDs Revealed]() Original publisher: [Read original article]() Author: Rob Andrews Published: 2026-08-20T12:00:27Z Content type: news Language: en Sources: [NAS Compares]() Topics: [Hardware](), [NVMe](), [M.2 NVMe](), [Specifications]() Tags: [asustor-ssd](), [m-2-nvme](), [nas-600](), [nas-800](), [nas-drive-ssd](), [nas-hardware](), [nas-ssd](), [network-attached-storage-nas](), [nvme](), [qnap-ssd](), [sandisk](), [sandisk-1000](), [sandisk-200](), [sandisk-2026](), [sandisk-2027](), [sandisk-400](), [sandisk-600](), [sandisk-800](), [sandisk-nas-600](), [sandisk-nas-800](), [sandisk-nas-ssd](), [sandisk-ssd](), [specifications](), [ssd](), [ssd-for-nas](), [ssd-nas-drive](), [synology-ssd](), [ugreen-ssd](), [uncategorised](), [wd-2026](), [wd-2027](), [wd-nas-ssd](), [wd-red-nas-ssd](), [wd-red-ssd]() ### AI overview Sandisk has introduced the NAS 600 and NAS 800 SSD families for NAS systems. The NAS 600 is a 2.5-inch SATA SSD for storage and all-flash arrays, while the NAS 800 is an M.2 2280 PCIe 5.0 NVMe SSD for caching, tiered storage and primary storage in compatible systems. ### Source excerpt New SSDs for NAS Released by Sandisk - Too late? Sandisk has introduced 2 SSD families designed specifically for NAS use, covering both conventional 2.5-inch SATA storage and higher-performance M.2 NVMe applications. The SANDISK NAS 600 and NAS 800 series are intended for prosumers, creative professionals and small businesses using always-on systems for shared storage, [...] ## How fsync affects data durability, latency, and group commit DevFeed: [How fsync affects data durability, latency, and group commit]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-08-14T00:00:00Z Content type: article Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [data](), [Latency](), [NVMe](), [Replication]() Tags: [databases](), [durability](), [flush](), [fsync](), [postgresql](), [replication](), [ssd](), [stack](), [storage]() ### AI overview This article explains that a successful write() may leave data only in the kernel page cache, so a power loss can destroy it. It describes fsync as the operation that pushes data to the device, discusses its latency cost, and explains how group commit amortizes the synchronization barrier. It also distinguishes disk durability from replication across failure domains. ### Source excerpt A successful write() only copies your data into the kernel page cache, where a power loss destroys it. Only fsync pushes it to the device, and because fsync is a barrier rather than a write, a system that syncs once per commit is capped at roughly one divided by the fsync latency, no matter how fast the rest of the stack is. Group commit exists to amortise that barrier, and replication solves a different failure domain than fsync does, which is why acks=all does not mean the data is on any disk. ## Building a Modern Analytics Stack Around ClickHouse DevFeed: [Building a Modern Analytics Stack Around ClickHouse]() Original publisher: [Read original article]() Author: Sebastian Insausti Published: 2026-07-01T10:35:21Z Content type: article Language: en Sources: [SeveralNines]() Topics: [clickhouse](), [Databases](), [Open Source](), [MySQL](), [PostgreSQL](), [real-time](), [NVMe](), [Replication](), [IO]() Tags: [analytics](), [analytics-stack](), [clickhouse](), [cta](), [database-general](), [databases](), [distributed](), [mysql](), [nvme](), [olap](), [open-source](), [postgresql](), [real-time]() ### AI overview This article explains how ClickHouse fits into a modern analytics stack alongside relational databases, Redis, and Prometheus. It describes ClickHouse as an open-source column-oriented DBMS for analytical workloads and discusses columnar storage, distributed processing, replication, and near-real-time ingestion. ### Source excerpt Historically, relational databases did double duty. The same PostgreSQL / MySQL instance that handled your application's writes also answered your business questions. A well-indexed schema, a few GROUP BY reports, done. And that works, right up until it doesn't: the reports get slower, the dashboards start eating the same I/O budget as the application; or, [...] The post Building a Modern Analytics Stack Around ClickHouse appeared first on Severalnines. ## Recent Homelab Lessons: Raspberry Pi NVMe, ARM64, ZFS, and Tools DevFeed: [Recent Homelab Lessons: Raspberry Pi NVMe, ARM64, ZFS, and Tools]() Original publisher: [Read original article]() Author: Christian Hollinger Published: 2025-10-04T00:00:00Z Content type: article Language: en Sources: [Christian Hollinger]() Topics: [Homelab](), [Proxmox](), [Raspberry Pi](), [NVMe](), [Grafana]() Tags: [hardware](), [home-lab](), [homelab](), [linux](), [networking](), [nvme](), [proxmox](), [raspberry-pi](), [server](), [zfs]() ### AI overview A homelab operator describes their Proxmox, TrueNAS, networking, monitoring, and alerting setup, then shares lessons from troubleshooting a slow Raspberry Pi 5 Proxmox node. The article discusses NVMe SSD support, ARM64 experimentation, and a likely failing SD card. ### Source excerpt From my series of "Christian's Home Lab adventures", here's some more things I've recently learned / discovered and figured are worth sharing: NVMe SSDs on a Raspberry PI, ARM64 adventures, more zfs troubles, and some cool tools you should try. ## Intel N100 Mini PC for R&D and Self-Hosting Compared with Raspberry Pi 5 DevFeed: [Intel N100 Mini PC for R&D and Self-Hosting Compared with Raspberry Pi 5]() Original publisher: [Read original article]() Author: Alex Ellis Published: 2025-08-18T08:09:48Z Content type: opinion Language: en Sources: [Alex Ellis' Blog]() Topics: [pc](), [intel](), [cpu](), [Self-hosted](), [Raspberry Pi](), [NVMe](), [Firecracker](), [Kubernetes](), [Linux](), [k3s](), [OpenFaaS](), [ddr5]() Tags: [ddr5](), [firecracker](), [github-actions](), [intel](), [k3s](), [kubernetes](), [linux](), [m-2-nvme](), [mini-pc](), [pc](), [processor](), [self-hosting](), [storage](), [threads](), [work]() ### AI overview The article evaluates the low-power Intel N100 Mini PC for R&D and self-hosting, comparing it with the Raspberry Pi 5. It discusses CPU, RAM, NVMe storage, Ethernet, virtualization, Kubernetes, Firecracker, and cost considerations. ### Source excerpt Exploring the capabilities of the Intel N100 Mini PC for work and self-hosting as an alternative to public cloud. ## Backup from broken NVMe drive with ddrescue DevFeed: [Backup from broken NVMe drive with ddrescue]() Original publisher: [Read original article]() Author: Tom Published: 2025-04-24T00:00:00Z Content type: tutorial Language: en Sources: [Tomasz Gągor]() Topics: [NVMe](), [Linux](), [data]() Tags: [backup](), [backup-failing-drive](), [bash](), [data](), [ddrescue-tutorial](), [debian](), [debian-data-rescue](), [errors](), [files](), [filesystem-repair](), [fix-input-output-error](), [gnu-ddrescue-guide](), [guide](), [how-to](), [linux](), [linux-disk-imaging](), [nvme](), [nvme-data-recovery](), [recover-corrupted-disk](), [ubuntu](), [ubuntu-disk-backup]() ### AI overview A guide to recovering data from a failing NVMe drive on Linux using ddrescue. It explains how to create an image despite read errors and restore usable files by repairing the filesystem and reinstalling damaged packages. ### Source excerpt Learn how to recover data from a failing NVMe drive using ddrescue on Linux. This guide covers step-by-step backup instructions, handling read errors, and restoring your files safely. ## ZFS Raidz Expansion Finally, Here in version 2.3.0 DevFeed: [ZFS Raidz Expansion Finally, Here in version 2.3.0]() Original publisher: [Read original article]() Author: Vivek Gite Published: 2025-01-14T09:19:20Z Content type: release Language: en Sources: [nixCraft: Linux Tips, Hacks, Tutorials, And Ideas In Blog Format (RSS/FEED)]() Topics: [Filesystems](), [RAID](), [releases](), [Linux](), [freebsd](), [Caching](), [NVMe](), [JSON](), [bug](), [make]() Tags: [bug](), [debian-linux](), [file-system](), [freebsd](), [hard-disk](), [io](), [json](), [linux](), [linux-news](), [nvme](), [raid](), [releases](), [sysadmins](), [testing](), [zfs]() ### AI overview OpenZFS 2.3.0 introduces RAIDZ expansion, allowing devices to be added to an existing RAIDZ pool to increase capacity without downtime. The release also includes faster deduplication, direct I/O, optional JSON output for ZFS commands, longer file and directory names, and bug fixes. ### Source excerpt After years of development and testing, the ZFS raidz expansion is finally here and has been released as part of version 2.3.0. ZFS is a popular file system for Linux and FreeBSD. RAIDz is like RAID 5, which you find with hardware or Linux software raid devices. It protects your data by spreading it across multiple hard disks along with parity information. A raidz device can have single, double, or triple parity to sustain one, two, or three hard disk failures, respectively, without losing any data. Hence, expanding or adding a new HDD is a very handy feature for sysadmins in today's data-sensitive apps. Love this? sudo share_on: Twitter - Facebook - LinkedIn - Whatsapp - Reddit The post ZFS Raidz Expansion Finally, Here in version 2.3.0 appeared first on nixCraft. ## SSD Performance Has Advanced Faster Than Cloud Vendor NVMe Instances DevFeed: [SSD Performance Has Advanced Faster Than Cloud Vendor NVMe Instances]() Original publisher: [Read original article]() Author: Viktor Leis (noreply@blogger.com) Published: 2024-02-19T08:00:00Z Content type: opinion Language: en Sources: [Database Architects]() Topics: [Cloud](), [Amazon EC2](), [NVMe](), [pcie](), [datacenter]() Tags: [aws](), [capacity](), [cloud](), [cost](), [data-center](), [ec2](), [io](), [nvme](), [pcie](), [performance](), [server](), [storage]() ### AI overview The article compares rapid advances in commodity PCIe and NVMe SSD performance with slower progress in cloud-based NVMe instances. It argues that AWS EC2 instance performance per SSD has remained around 2 GB/s since the i3 launch, leaving a substantial gap between leading data center SSDs and those offered by major cloud vendors. ### Source excerpt In recent years, flash-based SSDs have largely replaced disks for most storage use cases. Internally, each SSD consists of many independent flash chips, each of which can be accessed in parallel. Assuming the SSD controller keeps up, the throughput of an SSD therefore primarily depends on the interface speed to the host. In the past six years, we have seen a rapid transition from SATA to PCIe 3.0 to PCIe 4.0 to PCIe 5.0. As a result, there was an explosion in SSD throughput: At the same time, we saw not just better performance, but also more capacity per dollar: The two plots illustrate the power of a commodity market. The combination of open standards (NVMe and PCIe), huge demand, and competing vendors led to great benefits for customers. Today, top PCIe 5.0 data center SSDs such as the Kioxia CM7-R or Samsung PM1743 achieve up to 13 GB/s read throughput and 2.7M+ random read IOPS. Modern servers have around 100 PCIe lanes, making it possible to have a dozen of SSDs (each usually using 4 lanes) in a single server at full bandwidth. For example, in our lab we have a single-socket Zen 4 server with 8 Kioxia CM7-R SSDs, which achieves 100GB/s (!) I/O bandwidth: AWS EC2 was an early NVMe pioneer, launching the i3 instance with 8 physically-attached NVMe SSDs in early 2017. At that time, NVMe SSDs were still expensive, and having 8 in a single server was quite remarkable. The per-SSD read (2 GB/s) and write (1 GB/s) performance was considered state of the art as well. Another step forward occurred in 2019 with the launch of i3en instances, which doubled storage capacity per dollar. Since then, several NVMe instance types, including i4i and im4gn, have been launched. Surprisingly, however, the performance has not increased; seven years after the i3 launch, we are still stuck with 2 GB/s per SSD. Indeed, the venerable i3 and i3en instances basically remain the best EC2 has to offer in terms of IO-bandwidth/$ and SSD-capacity/$, respectively. Personally, I find this very s ## Booting the Raspberry Pi 5 from NVMe DevFeed: [Booting the Raspberry Pi 5 from NVMe]() Original publisher: [Read original article]() Author: Alex Ellis Published: 2023-12-28T17:55:43Z Content type: tutorial Language: en Sources: [Alex Ellis' Blog]() Topics: [NVMe](), [Raspberry Pi](), [boot](), [Linux](), [pcie](), [Ethernet](), [Edge](), [Google](), [Machine learning]() Tags: [boot](), [edge](), [ethernet](), [google](), [hdmi](), [linux](), [ml](), [mount](), [network](), [nvme](), [pcie](), [raspberry-pi](), [raspberry-pi-5](), [raspberry-pi-os]() ### AI overview A practical workflow for configuring a Raspberry Pi 5 to boot Raspberry Pi OS from NVMe for headless use. The article discusses first-generation PCIe breakout boards, recommends flashing a fresh image instead of cloning a booted SD card, and outlines setup steps using Ethernet, HDMI, or network scanning. ### Source excerpt Here's my workflow for setting up the Raspberry Pi 5 to boot from NVMe for headless use. ## Moving a Proxmox host with a SAS HBA as PCI passthrough for zfs + TrueNAS DevFeed: [Moving a Proxmox host with a SAS HBA as PCI passthrough for zfs + TrueNAS]() Original publisher: [Read original article]() Author: Christian Hollinger Published: 2023-10-26T00:00:00Z Content type: tutorial Language: en Sources: [Christian Hollinger]() Topics: [Proxmox](), [truenas](), [migration](), [Hardware](), [SAS](), [backups](), [Homelab](), [NVMe]() Tags: [backup](), [backups](), [debian](), [hardware](), [home-server](), [linux](), [luks](), [migration](), [networking](), [nvme](), [proxmox](), [sas](), [server](), [ssd](), [truenas](), [tutorial](), [zfs]() ### AI overview A tutorial on moving a Proxmox host to a new NVMe SSD while passing through an LSI SAS HBA to a TrueNAS SCALE virtual machine. It covers snapshots, backups, VM migration, host-root backup, and bootable installation media, with warnings about data loss and virtualizing TrueNAS. ### Source excerpt How to move a Proxmox host to a new m.2 SSD when using a LSI SAS HBA card to realize PCI passthrough for HDDs for zfs + TrueNAS. ## PINE64 August Community Update Covers Collaborative Publishing and Hardware and Software Progress DevFeed: [PINE64 August Community Update Covers Collaborative Publishing and Hardware and Software Progress]() Original publisher: [Read original article]() Published: 2023-08-15T00:00:00Z Content type: news Language: en Sources: [Community blog on PINE64]() Topics: [Pinephone-Pro](), [Embedded Software Dev](), [Pinephone](), [Ubuntu](), [NVMe](), [pcie](), [USB](), [ui]() Tags: [blog-post](), [community](), [community-update](), [devices](), [firmware](), [infinitime](), [news](), [update]() ### AI overview PINE64's August community update introduces collaborative authoring for future monthly posts and reviews recent progress across its devices and software. It covers PineTab2 and PineTab-V development, PinePhone Pro thermal-camera use, Ubuntu Touch, Quartz64 and SOQuartz, U-Boot support, PineTime and InfiniTime improvements, and related firmware and battery-life work. ### Source excerpt We apologize for the lack of regular community updates during the past few months. In short, the people who usually wrote them became very busy. However, we have a new way to write and publish these updates, involving you: the community. Let's talk about it, and about so much more of what's been happening. This community update consists of contributions by Caffeine, CounterPillow, Funeral, gamiee, JF002, and mpreditor. N.B. Comments on the blog post need to be in English and follow our Community Rules and Code of Conduct. TL;DR Housekeeping Community updates are now written collaboratively by several members of the community The public is invited to contribute to the monthly update posts Rework of the community website and logo Newsflash megi released rk2aw: a loader for your bootloader Progress on the PineTab2 and PineTab-V A PinePhone Pro connected to a thermal imaging camera Oren Klopfer's progress on Ubuntu Touch Quartz64 and SOQuartz U-Boot improvements thanks to Kwiboo and CounterPillow: USB, NVMe, and soon PXE Improved SOQuartz CM4 baseboard compatibility Improved PCIe devices compatibility QuartzPro64 What it means to "mainline" things, and why it takes time Things that have been mainlined: SATA, PCIe, USB, AV1, and more! Things still being worked on: cpufreq, GPU, video output InfiniTime 1.13 and other PineTime news Heart rate monitoring improvements thanks to Ceimour! Did you know that InfiniTime has a BLE weather service feature? Improved memory management Huge improvements to battery life (with help from Ayke!) InfiniSim UI update Improvements to ITD, the InfiniTime daemon Ayke is developing a brand new firmware for the PineTime Housekeeping You might have been wondering why nothing new has been posted on the PINE64.org community blog in quite a few months. As was already revealed in the opening section, the reason is that previously blogs were mainly authored and edited by one person (Lukasz) and said person is extremely busy these days. Going forward, ## The curious case of bad blocks on an SSD, and how I got rid of them DevFeed: [The curious case of bad blocks on an SSD, and how I got rid of them]() Original publisher: [Read original article]() Author: andreacorbellini Published: 2022-12-29T04:00:00Z Content type: article Language: en Sources: [Andrea Corbellini]() Topics: [NVMe](), [Linux](), [Kernel]() Tags: [badblocks](), [errors](), [exceptions](), [filesystem](), [information-technology](), [kernel](), [linux](), [nvme](), [smart](), [ssd](), [storage](), [ubuntu]() ### AI overview The author investigates an NVMe SSD that develops severe read errors, filesystem hangs, and resets after being moved from a coffee-damaged laptop. Kernel logs show repeated errors, while SMART data appears largely acceptable despite some warning signs. The supplied excerpt ends during the self-test investigation. ### Source excerpt I recently inherited a laptop that was broken by pouring some hot coffee on it. When I dissected it, it was pretty clear that most of it was unrecoverable: the CPU was completely fried, and its thermal paste splashed everywhere on the motherboard. (I wish I took a picture of it that I could share.) There were however a few pieces that looked in ... ## Are you sure you want to use MMAP in your database management system? DevFeed: [Are you sure you want to use MMAP in your database management system?]() Original publisher: [Read original article]() Author: Viktor Leis (noreply@blogger.com) Published: 2022-01-16T14:07:00Z Content type: article Language: en Sources: [Database Architects]() Topics: [Databases](), [Caching](), [Transactions](), [IO](), [systems](), [Kernel](), [NVMe]() Tags: [asynchronous](), [cache](), [caching](), [database](), [kernel](), [nvme](), [pcie](), [performance](), [systems](), [transactions]() ### AI overview The article discusses why database management systems often avoid mmap despite operating-system page caching. It explains that mmap limits control over write-back, transactions, crash recovery, asynchronous I/O, and error handling, and argues that Linux page-cache behavior may not keep pace with modern NVMe storage bandwidth. ### Source excerpt Many database management systems carefully manage disk I/O operations and explicitly cache pages in main memory. Operating systems implement a page cache to speed up recurring disk accesses as well, and even allow transparent access to disk files through the mmap system call. Why do most database systems then even implement I/O handling and a caching component if the OS provides these features through mmap? Andrew Pavlo, Andrew Crotty, and myself tried to answer this question in a CIDR 2022 paper. This is quite a contentious question as the Hacker News discussion of the paper shows. The paper argues that using mmap in database systems is almost always a bad idea. To implement transactions and crash recovery with mmap, the DBMS has to write any change out-of-place because there is no way to prevent write back of a particular page. This makes it impossible to implement classical ARIES-style transactions. Furthermore, data access through mmap can take a handful of nanoseconds (if the data is in the CPU cache) or milliseconds (if it's on disk). If a page is not cached, it will be read through a synchronous page fault and there is no interface for asynchronous I/O. I/O errors, on the other hand, are communicated through signals rather than a local error code. These problems are caused by mmap's interface, which is too high-level and does not give the database system enough control. In addition to discussing these interface problems, the paper also shows that Linux' page cache and mmap implementation cannot achieve the bandwidth of modern storage devices. One PCIe 4.0 NVMe SSD can read over 6 GB/s and upcoming PCIe 5.0 SSDs will almost double this number. To achieve this performance, one needs to schedule hundreds or even thousands (if one has multiple SSDs) of concurrent I/O requests. Doing this in a synchronous fashion by starting hundreds of threads will not work well. Other kernel-level performance issues are single-threaded page eviction and TLB shootdowns. Overall, ## Asus ROG STRIX Z590-E GAMING WIFI - my UEFI BIOS settings DevFeed: [Asus ROG STRIX Z590-E GAMING WIFI - my UEFI BIOS settings]() Original publisher: [Read original article]() Author: Tom Published: 2021-10-30T00:00:00Z Content type: tutorial Language: en Sources: [Tomasz Gągor]() Topics: [UEFI](), [ASUS](), [configuration](), [virtualization](), [cpu](), [DDR4](), [Linux](), [NVMe](), [pcie](), [Security](), [Windows 11](), [Artificial Intelligence]() Tags: [asus](), [asus-rog-strix-z590-e-bios-settings](), [configuration](), [cpu](), [ddr4](), [linux](), [nvme](), [optimal-bios-settings](), [pc](), [pcie](), [power-management-settings](), [ram-overclocking](), [security](), [uefi](), [uefi-bios-configuration](), [upgrade](), [usb](), [virtualization-settings](), [windows](), [windows-11]() ### AI overview A configuration guide for the Asus ROG STRIX Z590-E GAMING WIFI motherboard's UEFI BIOS. It lists settings for RAM performance, power management, virtualization, TPM, storage, PCIe, USB, networking, and boot behavior. ### Source excerpt Learn about the optimal UEFI BIOS settings for the Asus ROG STRIX Z590-E GAMING WIFI motherboard, including configurations for RAM, power management, virtualization, and more. ## AWS EC2 Hardware Trends: 2015-2021 DevFeed: [AWS EC2 Hardware Trends: 2015-2021]() Original publisher: [Read original article]() Author: Viktor Leis (noreply@blogger.com) Published: 2021-07-04T14:23:00Z Content type: article Language: en Sources: [Database Architects]() Topics: [Amazon EC2](), [Hardware](), [Amazon Web Services](), [cpu](), [intel](), [Network](), [NVMe](), [Cloud]() Tags: [aws](), [capacity](), [compute](), [cost](), [cpu](), [ec2](), [hardware](), [intel](), [network](), [nvme]() ### AI overview This article analyzes AWS EC2 hardware trends from 2015 to 2021 using hardware resources normalized by instance price. It finds that compute and DRAM costs changed little, while storage and network bandwidth saw significant improvements. ### Source excerpt Over the past decade, AWS EC2 has introduced many new instance types with different hardware configurations and prices. This hardware zoo can make it hard to keep track of what is available. In this post we will look at how the EC2 hardware landscape changed since 2015. This will hopefully help picking the best option for a given task. In the cloud, one can trade money for hardware resources. It therefore makes sense to take an economical perspective and normalize the hardware resource by the instance price. For example, instead of looking at absolute network bandwidth, we will use network bandwidth per dollar. Such metrics also allow us to ignore virtualized slices, reducing the number of instances relevant for the analysis from hundreds to dozens. For example, c5n.9xlarge is a virtualized slice of c5n.18xlarge with half the network bandwidth and half the cost. Data Set We use historical data from https://instances.vantage.sh/ and only consider current-generation Intel machines without GPUs. All prices are for us-east-1 Linux instances. Using these constraints, in July 2021 we can pick from the following instances: namevCPUmemory [GB]network [Gbit/s]storageprice [$/h]m4.16x6425625 3.20h1.16x64256258x2TB disk3.74c5n.18x72192100 3.89d3.8x322562524x2TB disk4.00c5.24x9619225 4.08r4.16x6448825 4.26m5.24x9638425 4.61c5d.24x96192254x0.9TB NVMe4.61i3.16x64488258x1.9TB NVMe5.00m5d.24x96384254x0.9TB NVMe5.42d2.8x362441024x2TB disk5.52m5n.24x96384100 5.71r5.24x9676825 6.05d3en.12x481927524x14TB disk6.31m5dn.24x963841004x0.9TB NVMe6.52r5d.24x96768254x0.9TB NVMe6.91r5n.24x96768100 7.15r5b.24x9676825 7.15r5dn.24x967681004x0.9TB NVMe8.02i3en.24x967681008x7.5TB NVMe10.85x1e.32x1283904252x1.9TB SATA26.69 && Compute Using our six-year data set, let's first look at the cost of compute: It is quite remarkable that from 2015 to 2021, the cost of compute barely changed. During that six-year time frame, the number of server CPU cores has been growing significantly, which may imply that I ## Allas November 2020 incident details DevFeed: [Allas November 2020 incident details]() Original publisher: [Read original article]() Author: Kalle Happonen (noreply@blogger.com) Published: 2020-12-22T12:31:00Z Content type: article Language: en Sources: [CSC - IT Center For Science - Cloud Team]() Topics: [ceph](), [incident](), [Amazon S3](), [rocksdb](), [NVMe](), [Processes](), [Vapor]() Tags: [ceph](), [deep-dive](), [incident](), [nvme](), [processes](), [rocksdb](), [s3](), [software](), [storage](), [swift](), [technical]() ### AI overview A technical deep dive into the November 2020 Allas downtime incident. It describes Allas as a Ceph cluster and explains its storage nodes, OSD processes, S3 and SWIFT access, replicated metadata pools, RocksDB metadata, placement groups, and erasure-coded object storage. ### Source excerpt Allas downtime November 2020 - technical deep-dive We use Ceph (https://ceph.io/) to provide software defined storage in our Allas service. I'll try to speak some Ceph here. There may be a lot of unfamiliar terms for most readers here as we dive deep into a technology. I'll try to clarify where I can, but architectural knowledge of Ceph helps when reading this. Allas is a Ceph cluster with 48 storage nodes (OSD nodes). Each has one nvme, 24 spinning disks, and 192 GiB of memory. Each OSD node runs one OSD process per disk. The Allas storage is provided over SWIFT and S3 protocols through radosgw daemons. The radowsgw layer is provided by separate internet facing servers. In addition we have 5 nodes running monitor processes (they control the state of the cluster), which also run Ceph manager (handles more advanced monitoring views into the cluster state). The object store metadata pools are 3-way replicated pools (data is stored in 3 different copies) on the nvmes across the OSD nodes. This metadata contains e.g. owners of buckets and objects, permissions, versions, etc. Everything that has to do with S3/SWIFT. The nvme also hosts the rocksdb (ceph internal metadata, as opposed to object store metadata) for the OSDs. This metadata is internal Ceph clustster status. Data on placement groups, etc. I'll use the term placement group (or pg) quite a lot in the post as they are central for Ceph. Basically when you store data objects into Ceph, they are mapped into placement groups, which define where in the cluster the data is are stored. The object store data is stored on an 8+3 erasure coded (data is split into 8 chunks + 3 parity chunks) pool on the spinning disks. The total amount of placement groups for the data pool is 16 384. In the graph the amount ~18 000 is shown, as the graph contains other pools too. The beginning Side note: I love that we collect trending data for our cluster (collectd+graphite in our case). As we had the data stored from a long time, this al