# en-blog-post Published articles for en-blog-post. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## \[CVE\] \[URGENT\] Squid v19.2.6 and Tentacle v20.2.4 released DevFeed: [\[CVE\] \[URGENT\] Squid v19.2.6 and Tentacle v20.2.4 released]() Original publisher: [Read original article]() Author: Patrick Donnelly Published: 2026-08-19T00:00:00Z Content type: release Language: en Sources: [Ceph Blog]() Topics: [releases](), [Authentication](), [Linux](), [client](), [monitor]() Tags: [authentication](), [blog-post](), [cve](), [cves](), [en-article](), [en-blog-post](), [errors](), [linux](), [process](), [release](), [releases](), [rest](), [rgw](), [squid](), [tentacle](), [upgrade]() ### AI overview The Ceph project released Tentacle 20.2.4 and Squid 19.2.6 as hotfixes addressing four CVEs. The article urgently recommends upgrading and explains changes to CephX key types, daemon-key rotation, authentication controls, recovery procedures, and multisite RGW request signing. ### Source excerpt The Ceph project has released Tentacle 20.2.4 and Squid 19.2.6 hotfixes for four CVEs across several components. This is the sixth backport release in the Squid series. This is the fourth backport release in the Tentacle series. We strongly recommend that all Ceph operators upgrade to one of these releases as soon as possible. Release Date ¶ August 19, 2026 Critical Upgrade Steps ¶ The fix for CVE-2025-30156 introduces a new CephX key type, aes256k. This is the first time Ceph has introduced a new key type for CephX credentials. Therefore, a new procedure exists for upgrading and rotating Ceph daemon keys as part of this CVE. Before upgrading a Ceph cluster, the operator should familiarize themselves with the procedure to upgrade CephX keys. These steps are for package-based deployments of Ceph. In other sections, the document also includes new information about key rotation procedures, monitor authentication controls, and emergency recovery procedures. Deployments using cephadm will automate the process except for client keys. You may notice that Cephadm spends more time than normal on the upgrade after updating all daemon images. This is due to a new process rotating the OSD and MDS keys. Please be patient. There are plans to improve this in the future. Deployments using Rook will also automate rotation of some client keys with some exclusions. Look to the Rook project's announcement for more details. Client and kernel upgrades are recommended to support aes256k. Note that upstream Linux kernel client support began in kernel 7.0 and has been backported to CentOS Stream 9 and 10. Check with your distribution vendor for backported key support before rotating client keys used by the kernel. As part of upgrading a Ceph cluster, six new health warnings and errors will be generated. This is normal. As part of the process to upgrade/rotate entity keys, you will address the warnings and errors as you go. These are documented beginning here. The CephX documentation has bee ## v20.2.3 Tentacle released DevFeed: [v20.2.3 Tentacle released]() Original publisher: [Read original article]() Author: Yuri Weinstein Published: 2026-08-05T00:00:00Z Content type: release Language: en Sources: [Ceph Blog]() Topics: [Deployment](), [Server](), [Hardware](), [TLS (Transport Layer Security)](), [Kafka](), [Grafana Cloud Metrics](), [Monitoring](), [Command-line interface](), [IO]() Tags: [2026](), [authentication](), [blog-post](), [cli](), [en-article](), [en-blog-post](), [hardware](), [io](), [kafka](), [metrics](), [monitoring](), [release](), [storage](), [tentacle](), [tls]() ### AI overview Ceph v20.2.3, the third minor release in the Tentacle series, is a recommended update containing fixes and improvements across metadata servers, object storage daemons, RADOS Gateway, client libraries, the dashboard, ceph-volume, and NVMe-oF. Changes include storage reliability fixes, TLS 1.3 cipher customization, encrypted object copying, Kafka notification authentication, hardware monitoring, CSV host uploads, and role-management updates. ### Source excerpt This is the third minor release in the Tentacle series. We recommend that all users update to this release. Release Date ¶ August 5, 2026 Notable Changes ¶ MDS (Metadata Server) ¶ Fixed a crash and shutdown hang that could occur when ephemeral pins were active and max_mds was set to 0. Session reclaim could miss blocklisting an old session; this is now handled correctly. Fixed a case where scrub was unable to identify damage from an irreparable hard link. OSD (Object Storage Daemon) ¶ BlueStore: Fixed a BlueFS WAL envelope-mode bug that caused write buffer misalignment. Erasure Coding: Fixed truncate+write planning for EC shard sizes. Fixed a condition that prevented rolling forward of PG log entries. Prevented OSDMap::check_health() from asserting when new OSDs are found in a subtree. Scrub: "repairing" scrubs -- deep scrubs re-run specifically to fix a previously detected inconsistency -- are no longer blocked by the configured allowed scrub hours/days; they can now run at any time. RGW (RADOS Gateway) ¶ Added the ssl_ciphersuites option for the Beast front-end to support TLS 1.3 cipher customization. Fixed PutObject's canned_acl comparison bug affecting BlockPublicAcls. Fixed crashes occurring on realm reload. Implemented CopyObject support for encrypted objects. Added mTLS client-certificate authentication support for Kafka notifications. Fixed removal of delete markers in lifecycle processing. RADOS / librados / neorados ¶ neorados: Fixed an infinite trim loop on empty data log shards. Fixed cancellation-slot cleanup in librados/asio's associated executor. Dashboard ¶ Added a hardware-monitoring dashboard backed by node-proxy metrics. Fixed a bind-address regression caused by CherryPy isolation. Added support for adding hosts via CSV upload. NVMeoF: multiple CLI and UI backports (EC pool support, IO statistics, role management). Object: The global RGW Roles tab has been removed. Role management is now scoped under RGW Accounts, where roles can be listed, created, ## v19.2.5 Squid released DevFeed: [v19.2.5 Squid released]() Original publisher: [Read original article]() Author: Yuri Weinstein Published: 2026-07-14T00:00:00Z Content type: news Language: en Sources: [Ceph Blog]() Topics: [changelog](), [Amazon S3](), [bug](), [TLS (Transport Layer Security)](), [Authentication](), [Authorization](), [SSL]() Tags: [authentication](), [authorization](), [blog-post](), [bug](), [changelog](), [concurrent](), [configuration](), [en-article](), [en-blog-post](), [object-storage](), [queuing](), [release](), [s3](), [squid](), [ssl](), [tls](), [vector]() ### AI overview Ceph v19.2.5 Squid is the fifth backport release in the Squid series. It fixes issues across MDS, OSD, and RGW, including request queuing faults, rollback and erasure-coding behavior, TLS 1.3 cipher configuration, S3 ACL handling, multisite synchronization, policy evaluation, presigned URL authentication, object tagging, frontend concurrency, and bucket statistics. ### Source excerpt This is the fifth backport release in the Squid series. We recommend all users update to this release. Notable Changes ¶ MDS (Metadata Server) ¶ Fixed a segmentation fault relating to MDSContext completion and request queuing. OSD (Object Storage Daemon) ¶ Rollback & Vector Fixes: Corrected rollback logic for partial write object information (OI) and optimized Erasure Coding (EC) by ensuring Twiddle creates a full-sized vector. RGW (RADOS Gateway) ¶ TLS 1.3 Ciphersuites: Introduced the ssl_ciphersuites configuration option for the Beast front-end to support TLS 1.3 cipher customizations. S3 PutObject ACLs Fix: Fixed a bug where PutObject requests were erroneously rejected under BlockPublicAcls due to an incorrect comparison return value from canned_acl.compare(). Multisite Sync Robustness: Resolved an infinite loop scenario in RGWBucketFullSyncCR when a source bucket was deleted mid-sync by introducing a mechanism to clear stateful data in reused bucket_list_result objects. SNS Policy Evaluation: Updated ListTopics to use the account root ARN for policy evaluation when invoked by a non-root account user, preventing unexpected implicit denies from identity-based policies. Presigned URLs: Fixed an internal server error occurring during the authentication process of presigned URLs. Prerequest Hook: Restructured the order of execution so that the prerequest hook now runs after the authorization process has completed. Object Tagging: Added handling for plain-text object tags during execution of RGWObjTags::decode(). Beast Frontend Stability: Applied a strand executor to the Beast timeout timer to avoid concurrent socket access and potential race conditions. Bucket Statistics: Added a fix for bucket stats execution when the underlying bucket index does not exist. Changelog ¶ [Stretch Mode] mon: restrict changing mon election strategy post stretch mode (pr#65458, Kamoltat Sirivadhna) Check if HTTP\_X\_AMZ\_COPY\_SOURCE header is empty (pr#66028, Suyash Dongre) Fix multifs ## Recover the Ceph Monitor Store Using OSDs DevFeed: [Recover the Ceph Monitor Store Using OSDs]() Original publisher: [Read original article]() Author: Eugen Block, crossposted by Anthony D'Atri Published: 2026-07-04T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [ceph](), [monitor](), [Script](), [Containers](), [log management]() Tags: [blog](), [blog-post](), [ceph](), [containers](), [en-article](), [en-blog-post](), [how-to](), [logging](), [logs](), [mon](), [monitor](), [recovery]() ### AI overview This blog post presents a general procedure for recovering a failed Ceph Monitor store by collecting osdmaps from existing OSDs, with additional considerations for cephadm-managed container deployments. It automates osdmap collection and basic logging while leaving the store rebuild and client authentication steps for careful manual inspection. The procedure applies only to non-encrypted OSDs, and the article emphasizes the importance of backing up dmcrypt/LUKS keys and reviewing host logs. ### Source excerpt Introduction ¶ A few weeks ago I helped a Ceph user to recover his broken cluster (see this thread). Basically, after his Monitors stopped working he re-deployed a new cluster with the same Ceph FSID and attached the existing OSDs to the re-deployed hosts. But it's not that easy to re-activate those OSDs because the new Monitors don't have the old osdmap, hence they don't know anything about the existing OSDs. So how did we fix that? This scenario can be considered a total Monitor store failure. There's a documented procedure in the upstream docs, unfortunately it's written for non-cephadm clusters and doesn't contain many details about subsequent steps. When Ceph daemons run within containers there are more things to consider. So I decided to write this blog post and add some more details to the procedure, targeting mainly clusters managed by cephadm. But the recovery procedure doesn't specifically require cephadm-specific commands, so it can be considered as a general guideline how to recover from a Monitor store loss, just with some extra details about cephadm deployments. This procedure only works for non-encrypted OSDs. If you are using dmcrypt (LUKS) OSDs, make sure you have a backup of the keys, otherwise your data is lost forever! But there is some development, a user in Slack pointed me to this PR which adds a backup mechanism for Monitors, probably available in the upcoming Umbrella release. Note that the PR also contains this statement: Monitor backups complement, but do not replace, the existing Monitor recovery procedures The procedure covered by said docs is already written in script form, so I used that as a template and extended it for cephadm usage and included some very basic logging. It collects the osdmaps from all OSDs and contains the necessary considerations regarding containers. I decided to automate only the osdmap collection, not all required steps of the store rebuild procedure (e. g. Monitor store rebuild, client auth, etc.) because the r ## Reclaim Ceph Capacity Through CephFS Transcoding DevFeed: [Reclaim Ceph Capacity Through CephFS Transcoding]() Original publisher: [Read original article]() Author: Anthony D'Atri Published: 2026-07-01T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [Transcodings](), [Software](), [data]() Tags: [2026](), [4k](), [article](), [blog-post](), [ceph](), [cephfs](), [dram](), [efficiency](), [en-article](), [en-blog-post](), [enterprise](), [enterprise-storage](), [filesystem](), [fujitsu](), [git](), [hardware](), [performance](), [software](), [space](), [storage](), [tentacle]() ### AI overview This article addresses rising storage demands and hardware costs by discussing CephFS capacity efficiency. It describes replicated pools, Erasure Coding, and Fast EC in Ceph Tentacle, while the title identifies CephFS transcoding as the article's focus. ### Source excerpt Data expands to fill available storage (and beyond)! ¶ It used to be that enterprise storage meant 6RU rackmount Fujitsu 2351 Eagles, each holding a mind-boggling 380 MiB of data: enough for a whole company! Today that 380 MiB can't even hold a 4k pickleball video. Enterprises, educational instutitions, and really just about anyone these days demand storage capacities that start on the order of hundreds of tebibytes and rapidly grow to pebibytes. As this article is written in the spring of 2026, the memory market, which includes DRAM, SSDs, and legacy HDDs, has experienced a dramatic escalation of pricing. It is not uncommon to be quoted a price four times what the same hardware cost a year ago, and there are signs that it is going to get worse before it gets better. What's a poor ammonite to do?? Cephers find themselves between the Charybdis of quotes approaching Disaster Area's hypermathematics and the Scylla of hungry users armed with torches and git forks. Git forks, pitchforks. Get it? Sigh. Tough room. Anyway... Short of nuking the site from orbit, how do we make everyone happy, or at worst mildly discontented? Efficiency! CephFS ¶ CephFS is a popular, highly available and scalable software-defined POSIX-style distributed filesystem that can easily store tens of pebibytes of precious data. Or, alternately, cat videos. Ceph deployments often begin small, with replicated pools for perceived performance needs. As the cluster grows to more nodes and more data, it may become feasible and desirable to switch to Erasure Coding (EC) to make more efficient use of raw capacity. An EC pool thus can require substantially less raw storage for a given amount of user data, or store gobs more user data on a given amount of raw capacity This EC overhead table presents efficiency (space amplification) factors for a spectrum of EC profiles. Replicated pools usually maintain three copies of data, so for comparison they manifest an overhead factor of 3.0. EC 4+2 or 6+3 presents a ## Ceph Q2 2026 Newsletter DevFeed: [Ceph Q2 2026 Newsletter]() Original publisher: [Read original article]() Author: Anthony Middleton Published: 2026-06-25T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [ceph](), [releases]() Tags: [2026](), [blog-post](), [ceph](), [ceph-events](), [collaboration](), [communication](), [community](), [contribute](), [developers](), [documentation](), [ecosystem](), [en-article](), [en-blog-post](), [event](), [governance](), [growth](), [metrics](), [newsletter](), [operational](), [operations](), [releases](), [resources](), [support]() ### AI overview The Ceph Q2 2026 Newsletter covers governance coordination, community elections, documentation initiatives, contributor resources, regional Ceph Days events, project releases, and ways to contribute. It emphasizes collaboration among users, developers, operators, and organizations across the Ceph ecosystem. ### Source excerpt During the second quarter, the Ceph Foundation focused on strengthening collaboration across the project, expanding community programs, supporting documentation efforts, and investing in regional events. This edition highlights recent governance discussions, community elections, new contributor resources, Ceph Days events, and opportunities for community members to get involved. If you have an idea for a Ceph-related event, outreach effort, or project, we encourage you to submit a funding request and work with the Foundation to help bring it to life. In This Issue ¶ Board and CSC Alignment on Community Priorities 2026 CSC Election Recap Meet the New Ceph Technical Writer Ceph Days Continue to Grow Latest Ceph Releases Community Committees and New Ways to Contribute Board and CSC Alignment on Community Priorities ¶ The Ceph Governing Board and Community Steering Committee (CSC) met this quarter to discuss the long-term health of the project, governance, documentation, community growth, and Foundation-supported initiatives. The discussion reaffirmed the distinct roles of each organization within the Ceph ecosystem. The CSC continues to provide technical governance for the project, while the Ceph Foundation focuses on community growth, events, outreach, and funding. The Linux Foundation provides operational, legal, and financial support that enables both groups to operate effectively. A significant area of focus was improving coordination between technical priorities and Foundation-supported programs. Current documentation initiatives include improvements to the documentation backport process, development of a new OSD operations guide, clarification of package and container documentation, and the removal of outdated content. The Board and CSC also discussed the future of community events and the importance of establishing clear goals and success metrics for Foundation-funded programs. Both groups reaffirmed their commitment to supporting collaboration between users, de ## v20.2.2 Tentacle released DevFeed: [v20.2.2 Tentacle released]() Original publisher: [Read original article]() Author: Yuri Weinstein, Patrick Donnelly Published: 2026-06-16T00:00:00Z Content type: release Language: en Sources: [Ceph Blog]() Topics: [bug](), [Vulnerabilities](), [Memory Leaks](), [Encryption](), [configuration](), [IO]() Tags: [2026](), [aio](), [blog-post](), [bug](), [ceph](), [configuration](), [en-article](), [en-blog-post](), [encryption](), [io](), [memory-leaks](), [rados](), [release](), [rgw](), [storage](), [tentacle](), [ui](), [update](), [vulnerabilities]() ### AI overview Ceph v20.2.2 Tentacle is a minor release that adds Rocky 10 package-install support and recommends updating. It fixes issues across MDS, OSD, RGW, RADOS libraries, watch/notify operations, asynchronous utilities, and the dashboard, including security vulnerabilities, memory leaks, encrypted-object lifecycle problems, and UI and configuration improvements. ### Source excerpt This is the second minor release in the Tentacle series. We recommend that all users update to this release. Release Date ¶ June 16, 2026 Notable Changes ¶ Rocky 10 package-based installs are now supported starting with v20.2.2. Please see the supported platforms for current and planned support in Ceph. MDS (Metadata Server) ¶ Segmentation fault fixed due to incorrect queueing of request retries. OSD (Object Storage Daemon) ¶ PGLog Missed List: Fixed a bug to ensure the correct version is attached to the missing list when ignoring log entries. Data Integrity Asserts: Added assertions to explicitly catch potential corruption in the OSD missing list. RGW (RADOS Gateway) ¶ Lifecycle Management: Fixed lifecycle transition issues affecting encrypted multipart objects. REST & Query Handling: RESTArgs::get_string() now properly URL-decodes incoming query parameters. RADOS / librados / neorados ¶ Linger Operations: Rewrote safety checks to eliminate use-after-free vulnerabilities and LingerOp memory leaks when an unwatch operation returns ENOTCONN. Replaced the flawed is_valid_watch() check with a safe linger_by_cookie() lookup which safely manages LingerOp references using intrusive_ptr. Ensured librados linger callbacks hold a persistent reference to LingerOp to protect against races with simultaneous linger_cancel() requests. Configured librados::IoCtxImpl::aio_unwatch() to asynchronously deliver ENOTCONN to AioCompletion instead of returning the error directly. Watch/Notify: Fixed neorados notification queue bounds enforcement so that an overflow marker is only appended on the first message exceeding capacity, rather than duplicating it on every subsequent message. Prevented double-cleanup triggers in watch/notify operations when incoming errors arrive after maybe_cleanup() runs. Fixed a bug where notify would lose original error values by ensuring it no longer attempts to decode empty responses. Enhanced io_context shutdown procedures to clear handlers and route cleanl ## Ceph Days Seattle 2026 - Cloud Transition to Tape DevFeed: [Ceph Days Seattle 2026 - Cloud Transition to Tape]() Original publisher: [Read original article]() Author: John Shubeck Published: 2026-06-15T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [Open Source](), [Amazon S3](), [ibm](), [Cloud](), [Software]() Tags: [2026](), [blog-post](), [ceph](), [cloud](), [conferences](), [developers](), [en-article](), [en-blog-post](), [ibm](), [network](), [object-storage](), [open-source](), [s3](), [seattle](), [storage]() ### AI overview This article recaps Ceph Days Seattle 2026, a community-driven event about open-source storage. It introduces Ceph as a software-defined platform for block, file, and object storage, and highlights a presentation on transitioning Ceph cloud workloads to S3-enabled tape. ### Source excerpt I'm John Shubeck, an information technology professional with over 44 years of industry experience spanning both the customer and technology provider experience. I'm currently serving as a Senior Storage Technical Specialist for IBM Object Storage platforms across all market segments in the Americas. On Thursday, May 28th, 30 members, myself included, of the Ceph faithful descended on the SURF business incubator in downtown Seattle for "Ceph Days Seattle 2026". Following the traditional Ceph Days agenda, the Seattle program consisted of a series of "Ceph Talks" designed to introduce new ideas and share lessons learned. Ceph Days are community-driven events designed to bring together Ceph users, developers, architects, administrators, and anyone interested in open-source storage. Held throughout the year in locations around the world, these one-day gatherings offer a unique opportunity to learn from real-world deployments, explore new technologies, connect with peers, and contribute to discussions that help shape the future of Ceph. Did you know? ¶ Ceph is a software-defined, hardware-independent storage solution that provides block, file, and object storage on a single unified platform. Ceph is championed by developers, administrators, users, IT leaders, and Fortune 500 enterprise customers. It is a living, vibrant, and active group that has adopted Ceph in its IT operations. The Ceph project and community are supported by a Foundation (https://ceph.io/en/foundation/) comprising organizations, stakeholders, and industry leaders who collaborate to coordinate investment, development, and community activities for Ceph. With the setting, plot, and characters stated above, we turn to today's topic. A key activity of the Ceph community is a series of "Ceph Days" conferences. A Ceph Day is a full-day meetup where participants learn, network, make new acquaintances, and share ideas on how to enhance the value of Ceph. Sometimes IBM participates by providing the venue, a gue ## For whom the door-bell tolls DevFeed: [For whom the door-bell tolls]() Original publisher: [Read original article]() Author: Kyle Bader Published: 2026-06-11T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [Caching](), [Large Language Model](), [Inference Performance](), [Low-Latency Inference](), [GPU](), [cpu]() Tags: [2025](), [ai](), [architecture](), [blog-post](), [cache](), [caching](), [ceph](), [cpu](), [en-article](), [en-blog-post](), [gpu](), [gpu-initated](), [inference](), [llm](), [nvme](), [rados](), [rados-nkv](), [storage]() ### AI overview The article explores using Ceph RADOS through NVMe key-value commands to support GPU-initiated storage access for LLM KV caching. It connects content-addressable caching with the efficiency of GPU-initiated NVMe access and discusses a possible vendor-specific extension for executing classes against objects. ### Source excerpt For whom the door-bell tolls In a previous post we extolled the benefits of KV caching, a technique to save the KV states from the prefill step of LLM-based inference to reduce time to first token (TTFT) and skip redundant computation. I co-presented this with Tushar Gohad at Cephalocon. Since then I've been thinking a lot about how to improve the state of the art. Really move the needle. We've made strides in a lot of areas in Ceph, especially the work going into Fast EC -- if you don't know what I'm talking about you should check it out, it promises huge benefits to a large category of workloads. That's not what we're here for today, though. Last year there was one paper that stuck in my mind, which is fairly remarkable because I read on the order of 130. That paper was GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture. I struggled with this. It describes a system where a CPU loads a kernel into the GPU that allows the GPU to serve as an NVMe initiator. The struggle was rooted in the fact that block just felt like the wrong interface for KV caching. If you use block, then you need a lookup table that maps the hash of the sequences representing a cache block to a particular (device, offset, length) tuple. It begs for a content-addressable approach with no centralized lookup or coordination. On the other hand, what was described in the paper was just flat-out more electrically efficient. I couldn't have my cake and eat it too. The idea: RADOS, spoken as NVMe key-value ¶ In 2025 we also saw the first ratified version of the NVMe key-value command set specification. We already have a Ceph implementation of NVMe/TCP that leverages SPDK, and it seemed like we could add support fairly easily for the key-value command set. For those who don't know the internal mechanics of Ceph, the native API operates against RADOS objects. RADOS is richer than most object stores: it supports reads and writes to arbitrary offsets, deletes, key-value sto ## Managing Multiple Lua Scripts with Ceph Object Storage DevFeed: [Managing Multiple Lua Scripts with Ceph Object Storage]() Original publisher: [Read original article]() Author: Kirby Chin Published: 2026-06-10T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [Lua](), [Script](), [Scripting](), [Security](), [Command-line interface](), [cloud-infrastructure](), [ransomware]() Tags: [availability](), [blog](), [blog-post](), [ceph](), [cli](), [data](), [developer](), [en-article](), [en-blog-post](), [examples](), [feature](), [go](), [lua](), [management](), [new-feature](), [object-storage](), [operations](), [rados](), [ransomware](), [rgw](), [s3](), [scripting](), [security](), [storage]() ### AI overview This article introduces a Ceph RADOS Gateway feature for managing multiple Lua scripts within the same request context and tenant. It demonstrates scripts for enforcing bucket object locks to mitigate ransomware risks and for optimizing storage through object auto-tiering. ### Source excerpt Since the Pacific release, Lua scripting in Ceph's RADOS Gateway (RGW) has provided users the ability to interpolate a single script to upload operations per request context and tenant. This way of working might be completely fine for a storage deployment with limited scripting customizations. However, script management becomes increasingly difficult as more than one team wants to get involved in managing the Lua script within the same context and tenant. For this reason, we've released a new feature in RGW allowing you to manage more than one Lua script at a time. In this blog, we'll go over a couple of examples to walk you through how this feature can help your team to reduce runtime errors and lower developer friction when managing Lua scripts in RGW. To begin, we will create a script to enforce a bucket security control and then add another script to demonstrate the new feature. Script 1: Bucket security ¶ Let's suppose we are part of a security team and want to mitigate the risk of ransomware threats happening on our storage platform. One possible way to achieve this is to implement a Write Once, Read Many (WORM) strategy to add an object lock onto any newly created bucket. By using an object lock, we can add a constraint to write objects to disk only once, ensuring that an infected client cannot delete or override objects at a later time. To set our WORM strategy, we can create a Lua script in the prerequest context that aborts when a create_bucket operation is made without the write-once (object lock) requirement. objectlock.lua -- enforcing object lock on bucket creation if Request.RGWOp == "create_bucket" and Request.HTTP.Metadata["x-amz-bucket-object-lock-enabled"] ~= "true" then RGWDebugLog("object lock is missing on bucket: " .. Request.Bucket.Name) Request.Response.Message = "Bucket must have object lock enabled" return RGW_ABORT_REQUEST end You can create a new objectlock.lua file with the contents above and run the CLI commands below to upload the scr ## v19.2.4 Squid released DevFeed: [v19.2.4 Squid released]() Original publisher: [Read original article]() Author: Yuri Weinstein Published: 2026-06-01T00:00:00Z Content type: release Language: en Sources: [Ceph Blog]() Topics: [cephfs](), [Kafka](), [dashboards](), [Authorization](), [Tooling]() Tags: [blog-post](), [bug](), [cephfs](), [en-article](), [en-blog-post](), [json](), [kafka](), [release](), [squid](), [update]() ### AI overview Ceph v19.2.4 Squid is the fourth backport release in the Squid series, and the article recommends that all users update. Changes include BlueFS storage adjustments, fixes in RADOS and RADOS Gateway, Kafka 3.9.2 support, CephFS and MDS diagnostic and consistency improvements, Dashboard service and access-control fixes, and an RBD lock policy change. ### Source excerpt This is the fourth backport release in the Squid series. We recommend all users update to this release. Notable Changes ¶ OSD / BlueStore ¶ BlueFS: Increased the default WAL volume size to 1GB to prevent ENOSPC failures. Volume Selection: Fixed the usage of bluestore_volume_selection_reserved_factor and updated row naming conventions in RocksDBBlueFSVolumeSelector. Health: Added a time_added field to mon_info_t to prevent freshly added monitors from incorrectly triggering a MON_DOWN health status. Tooling: Updated monmaptool to correctly respect set features when adding addresses. RADOS / Manager (mgr) ¶ RADOS pybind: Fixed a parameter reversal bug in WriteOp.zero() where offset and length were swapped when calling the underlying C API. Daemon Health: Modified the manager to clear health metrics for down or out OSDs instead of removing them entirely from the daemon state. Autoscaler: Updated pg_autoscale_mode logic to accurately reflect the no_autoscale flag in both standard and JSON command outputs. RADOS Gateway (RGW) ¶ Notifications: Resolved a reserved_size drift in the 2pc_queue causing ENOSPC errors and fixed a leak where overhead was not properly decremented during commit/abort. Multipart Uploads: Fixed an issue where tags could not be retrieved from objects created via multipart uploads. Kafka Integration: Updated the internal Kafka version support to 3.9.2. CephFS / MDS ¶ Fragments: Simplified fragment printing and corrected frag_t endianness conversion for network and storage consistency. FUSE Client: Updated the FUSE client to clarify that fallocate disk space reservation is only supported with specific flags like FALLOC_FL_KEEP_SIZE. Metadata: Updated the MDS to dump frag_t as an object and included sysinfo in the status command output for better diagnostics. Dashboard ¶ Services: Fixed a bug where changing MDS placement to a label incorrectly triggered the creation of an entirely new MDS service. Access Control: Fixed role-based access control permissio ## From Laminated Pages to Petabytes: Why I'm Joining the Ceph Foundation DevFeed: [From Laminated Pages to Petabytes: Why I'm Joining the Ceph Foundation]() Original publisher: [Read original article]() Author: Emmanuel Ameh Published: 2026-05-06T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [Usability](), [Software](), [systems](), [incident](), [Disk image](), [CRUD]() Tags: [blog-post](), [ceph](), [community](), [developer](), [documentation](), [en-article](), [en-blog-post](), [guide](), [how-to](), [incident](), [logging](), [production](), [software](), [storage](), [systems](), [technical](), [usability](), [validation](), [workflow]() ### AI overview A new Ceph Foundation technical writer explains a documentation philosophy centered on usability and practical outcomes. The article introduces test-driven documentation: validate commands and workflows in clean environments, log user friction, and organize information so urgent how-to guidance is easy to find while deeper explanations remain accessible. ### Source excerpt Hello, Ceph Community! I'm Emmanuel Ameh, and I'm thrilled to be joining as your new Technical Writer. My path into technical writing started during a university internship at a manufacturing plant, where I was handed a problem that, looking back, shaped how I think about documentation to this day. The first piece of documentation I wrote was a one-page troubleshooting guide. It was a simple list of common production-line issues and their fixes, taped to a workstation. But it worked. Operators stopped escalating the same problems, uptime held steady, and that single laminated page did the job of a dozen phone calls. I think about that page a lot. It wasn't elegant. It wasn't comprehensive. It didn't win any awards. But it sat exactly where the people who needed it could find it, and it told them exactly what to do. That, to me, is what good documentation looks like, whether it's protecting a production line or a petabyte-scale storage cluster. That experience taught me my core philosophy: good documentation isn't measured by how thorough it looks. It's measured by what it lets people do. A doc that keeps a cluster running is worth more than a hundred pages nobody reads. That belief is what brings me to the Ceph community today. My Approach: Test-Driven Documentation ¶ My background is in enterprise software and developer advocacy, where I've focused on making complex systems feel approachable. I treat usability as seriously as technical accuracy. To me, a "perfectly correct" doc that a user can't follow is a broken feature. I advocate for test-driven documentation. In practice, this means: Validation: Running every command and workflow in a clean environment to ensure it actually works. If a tutorial says "run this, and you'll see X," then somewhere, on a fresh install, I should be able to run it and see X. If I can't, neither can the reader. Friction Logging: Identifying the exact moment a newcomer gets stuck and smoothing that path. The most valuable feedback I ca ## v20.2.1 Tentacle released DevFeed: [v20.2.1 Tentacle released]() Original publisher: [Read original article]() Author: Yuri Weinstein Published: 2026-04-06T00:00:00Z Content type: release Language: en Sources: [Ceph Blog]() Topics: [bug](), [configuration](), [monitor](), [Command-line interface](), [API](), [dashboards](), [JSON]() Tags: [2026](), [api](), [blog-post](), [bug](), [cli](), [en-article](), [en-blog-post](), [json](), [monitor](), [nvme](), [osd](), [peering](), [policy](), [recovery](), [reef](), [release](), [rgw](), [tentacle](), [ui](), [workflows]() ### AI overview Ceph v20.2.1, the first minor release in the Tentacle series, is recommended for all users. The release fixes storage, recovery, peering, validation, and dashboard issues; restricts unsafe fast EC configurations; improves NVMeoF Gateway failover; adds listener and subsystem CLI commands; introduces a transient exclusive RBD lock policy; and updates dashboard navigation and forms. ### Source excerpt This is the first minor release in the Tentacle series. We recommend that all users update to this release. Release Date ¶ April 06, 2026 Notable Changes ¶ OSD / BlueStore ¶ EC Recovery: Fixed a length calculation bug in erase_after_ro_offset() that caused empty shards to retain data, leading to shard_size >= tobj_size assertion failures when recovering small objects in EC pools. BlueFS Volume Selector: Updated the BlueFS volume selector to properly account for file size changes when recovering the WAL in envelope mode. BlueFS: Fixed a bug where stat() missed the actual file size update after indexing WAL envelope files. Monitor (mon) ¶ Fast EC Restrictions: Denied the ability to enable EC optimizations ("fast EC") for non-4K-aligned chunk sizes. Unaligned chunk sizes handled by fast EC perform poorly and suffer from bugs, so attempts to force this configuration are now rejected. Peering: Ensured ceph pg repeer proposes a correctly sized pg temp, as optimized EC cannot cope with mismatched sizes. NVMeoF Gateway: Added a new nvme-gw listeners command to display all existing listeners (including auto-listeners) inside a pool/group. NVMeoF Failover: Overhauled the NVMeoF Gateway fast-failover logic. Beacon timeouts are now evaluated within prepare_beacon to support shorter intervals, and the mechanism for detecting monitor slowness was improved. librbd & rbd-mirror ¶ RBD: Introduced a new RBD_LOCK_MODE_EXCLUSIVE_TRANSIENT policy for rbd_lock_acquire(). This is a low-level interface intended to allow a peer to grab exclusive lock manually for short periods of time with other peers pausing their activity and waiting for the lock to be released rather than instantly aborting I/O and returning an error. It's possible to switch from RBD_LOCK_MODE_EXCLUSIVE to RBD_LOCK_MODE_EXCLUSIVE_TRANSIENT policy and vice versa even if the lock is already held. Ceph Object Gateway (RGW) ¶ Multi-Part Operations: Fixed conditional validation handling in MultiWrite, Delete, and MultiDelete ## Ceph Q1 2026 Newsletter DevFeed: [Ceph Q1 2026 Newsletter]() Original publisher: [Read original article]() Author: Anthony Middleton Published: 2026-03-31T00:00:00Z Content type: news Language: en Sources: [Ceph Blog]() Topics: [Developer experience](), [Slack](), [ci]() Tags: [2026](), [blog-post](), [ceph](), [ceph-events](), [ci](), [communication](), [community](), [complexity](), [contribute](), [contributors](), [developer-experience](), [ecosystem](), [en-article](), [en-blog-post](), [event](), [events](), [governance](), [guide](), [onboarding](), [slack]() ### AI overview The Ceph Q1 2026 Newsletter covers the Ceph Foundation's governance, event planning, financial updates, community funding opportunities, and communication initiatives. It also reports on efforts to improve developer and contributor experience by addressing operational complexity, CI complexity, review capacity, onboarding, and communication between the Foundation Board and the Ceph Steering Committee. ### Source excerpt During this quarter, the Ceph Foundation focused on strengthening its structure by establishing new governance charters, event strategies, and financial plans. To enhance transparency as our community evolves, this newsletter offers a look behind the scenes at these foundational details. Our aim is to update Ceph Community members on the latest developments within the Foundation and to clarify how they can contribute to our ongoing growth. There are several ways to get involved with the foundation. If you have a concept for a Ceph-related project, we encourage you to take the first step toward bringing your idea to the next level and submit a funding request whenever you are ready. Feedback and suggestions will be offered along your journey with Ceph. In this issue ¶ CSC and Ceph Foundation Board Meeting Ceph Foundation Charters Approved Ceph Governing Board Hiring a Technical Writer OVHcloud Spending Update Ceph Tech Talks Are Back (Monthly Schedule) Upcoming Ceph Days Events Ceph Community Slack Upgraded to Pro CSC and Ceph Foundation Board Meeting ¶ The Ceph Foundation Board recently hosted the Ceph Steering Committee (CSC) for a collaborative discussion on the current state of the Ceph project and how both sides can work together to support the Ceph community. These quarterly meetings are designed to foster communication between the two committees working to build Ceph for the benefit of its users and contributors. The Ceph Board's goal is to help provide greater context to the CSC as they make decisions and to support their missions, thereby bridging the communication gap. The meeting's agenda is available here. Key discussion areas ¶ Operational complexity Friction in contributing and getting reviews Fragmented communication Unclear strategy in some areas Unclear ownership across parts of the ecosystem Major takeaways ¶ The Board will continue to work on closing the gap between developer experience and real-world operator needs, with help from the CSC around f ## v18.2.8 Reef released DevFeed: [v18.2.8 Reef released]() Original publisher: [Read original article]() Author: Yuri Weinstein Published: 2026-03-20T00:00:00Z Content type: release Language: en Sources: [Ceph Blog]() Topics: [Vulnerabilities](), [Security](), [configuration](), [SSL](), [Disaster Recovery](), [data]() Tags: [2026](), [aws](), [blog-post](), [bug](), [configuration](), [disaster-recovery](), [en-article](), [en-blog-post](), [reef](), [release](), [security](), [ssl](), [vulnerability]() ### AI overview Ceph v18.2.8 Reef is the eighth and expected final backport release in the Reef series, released on March 20, 2026. It includes upgrade guidance, security fixes for CephFS clients and Mgr Alerts, RGW and CephFS reliability fixes, configurable JWKS URL verification for AWS compliance, and additional MDS and CephFS changes. ### Source excerpt This is the eighth, and expected to be last, backport release in the Reef series. We recommend that all users update to this release. Release Date ¶ March 20, 2026 Known Issues ¶ During QA for v18.2.8, it was found that there was a bug for upgrades from Pacific to Reef. Pacific OSDs (and other Ceph daemons) were still using a deprecated connection feature bit that was adopted to indicate a Reef OSD. This can cause a OSD_UPGRADE_FINISHED warning before all OSDs are actually upgraded to Reef. There are no known issues associated with Pacific and Reef OSDs interoperating where Pacific OSDs are "advertising" Reef compatibility; however, out of an abundance of caution, we no longer recommend upgrading from Pacific to Reef directly. Security Fixes ¶ CephFS Client: A fix was merged to prohibit unprivileged users from modifying the sgid or suid bits on a file. Previously, unprivileged users were inadvertently permitted to set these bits if they were the sole bits being modified. Mgr Alerts: The SMTP SSL context was enforced in the mgr/alerts module to resolve a security vulnerability (GHSA-xj9f-7g59-m4jx). Notable Changes ¶ RGW (RADOS Gateway): Fixed an issue where bucket rm --bypass-gc was mistakenly removing head objects instead of tail objects, potentially causing data inconsistencies. Fixed rgw-restore-bucket-index to handle objects with leading hyphens and to process versioned buckets correctly. Addressed an issue in the msg/async protocol that caused memory locks and hangs during connection shutdown. RGW STS: Made JWKS URL verification configurable for AWS compliance via the rgw_enable_jwks_url_verification configuration. CephFS / MDS: Prevented the MDS from stalling (up to 5 seconds) during rename/stat workloads by forcing the log to nudge for unstable locks after early replies. Fixed cephfs-journal-tool so it no longer incorrectly resets the journal trim position during disaster recovery, which was causing stale journal objects to linger forever in the metadata pool ## Assessing the performance of the CLAY Erasure Code Plugin DevFeed: [Assessing the performance of the CLAY Erasure Code Plugin]() Original publisher: [Read original article]() Author: Jake Squelch (IBM) Published: 2026-02-11T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [benchmarking](), [IO](), [Code]() Tags: [4k](), [benchmark](), [benchmarking](), [benchmarks](), [blog-post](), [ceph](), [en-article](), [en-blog-post](), [io](), [performance]() ### AI overview This article assesses the performance of the CLAY Erasure Code Plugin using CBT benchmarks. It reports that CLAY and JErasure have practically identical read performance for 4K random reads and 1024K sequential reads, while CLAY has substantially lower write performance, especially at higher bandwidths. The article attributes this to CLAY's more complex encoding process, which performs two encoding stages plus an additional memory copy. It also reports worse performance with an OSD down, including increased latency for degraded reads, but the supplied text ends before the discussion is complete. ### Source excerpt CBT Performance Benchmarking - Part 4. What can we say about CLAY? Outline of the Blog Series ¶ Part 1 - How to start a Ceph cluster for a performance benchmark with CBT Part 2 - Defining YAML contents Part 3 - How to start a CBT performance benchmark Part 4 - Assessing the performance of the CLAY erasure code plugin Contents: Client IO results for CLAY Client IO with an OSD down What is CLAY good at? Problems with using CLAY How does CLAY read data from the drive? CLAY is broken in tentacle Summary Client IO results for CLAY ¶ As a refresher lets quickly look back on the client IO results of CLAY compared to JErasure: If we look back to Step 3 in Part 3 of the blog (Generating a comparison report), we saw that reads had practically identical curves between CLAY & JErasure for both 4K random reads and 1024K sequential reads. However, when we compared writes we saw that the performance hit to CLAY was substantially larger, particularly for higher bandwidths. The 1024k Sequential Writes diagram represents this: Click to see Part 3 diagrams So why was this? This is because of CLAY's encoding process, it is significantly more complex. While JErasure performs a single encoding pass, CLAY uses three phases: 50% of data is encoded using PRT (Product Recovery Transform), 50% of the data is copied to form an intermediate set of buffers All the intermediate data is encoded using RS (Reed-Solomon) to form a second set of intermediate buffers 50% of the result is encoded using PFT (Parity Fractional Transform), 50% of the data is copied to form the output buffers Essentially, CLAY performs 2x the encoding plus an additional memcpy (memory copy) compared to JErasure's 1x encoding. This overhead therefore directly translates to lower write throughput for CLAY, as shown by the diagrams above. The performance impact increases for larger IO sizes because more data is being encoded. Referenced the following: 'Clay Codes: Moulding MDS Codes to Yield an MSR Code' above for information ## RGW Bucket Resharding Without Pausing DevFeed: [RGW Bucket Resharding Without Pausing]() Original publisher: [Read original article]() Author: Daniel Alexander Parkes, Anthony D'Atri Published: 2026-02-01T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [systems](), [Scalability](), [Architecture & Design](), [cloud-infrastructure]() Tags: [2026](), [article](), [blog-post](), [ceph](), [concurrent](), [deep-dive](), [en-article](), [en-blog-post](), [io](), [operational](), [operations](), [performance](), [production](), [rgw](), [s3](), [scalability](), [scale](), [storage](), [systems]() ### AI overview The article explains how Ceph Tentacle introduces near-zero-impact RGW bucket resharding. Its two-phase architecture moves intensive work into a non-blocking background phase, avoiding the prolonged write unavailability and 504 errors seen in Ceph Squid and earlier versions. ### Source excerpt Introduction: The Foundation of Scalable Object Storage ¶ In the modern data landscape, object storage has evolved from a simple file repository into the foundational layer for AI/ML pipelines, data lakehouses, real-time analytics, and massive-scale archival systems. At the heart of this evolution is a deceptively simple question: How do you efficiently locate and access billions of objects stored in a single bucket? The answer lies in one of Ceph's most critical performance mechanisms: bucket index sharding. This architectural pattern divides a bucket's index into multiple parallel structures, enabling concurrent operations across thousands of objects while maintaining the consistency and reliability that enterprise workloads demand. But there's always been a catch. As workloads grow and evolve, buckets need to be resharded. Historically, when the buckets to be resharded had a vast number of objects, this operation came with a painful trade-off: blocking client writes from seconds to minutes, with a chance of causing application disruptions, 504 Gateway errors, and operational headaches. With Ceph Tentacle, we're eliminating this trade-off. The new near-zero impact bucket resharding architecture transforms what was once a maintenance window event into a seamless background operation that your applications will never notice. Note: As of 2026/02/05, the functionality described in this article is expected in an upcoming Tentacle update. Executive Summary ¶ The Challenge: In Ceph Squid, resharding a 20-million-object bucket blocked writes for 4+ minutes, returning 504 errors. Even larger buckets (500M objects) required 94 minutes of complete write unavailability. The Solution: Ceph Tentacle's two-phase architecture moves the heavy lifting to a non-blocking background phase, eliminating the impact on clients IO. The Results: (note: in this graphic 8.1 refers to Squid and 9.0 to Tentacle) In this deep dive, we'll explore: Why bucket sharding is essential for modern workl ## Mastering IAM in Ceph: Multi-Tenancy, Access Control, and Why ACLs Must Die DevFeed: [Mastering IAM in Ceph: Multi-Tenancy, Access Control, and Why ACLs Must Die]() Original publisher: [Read original article]() Author: Daniel Alexander Parkes, Anthony D'Atri Published: 2026-01-24T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [IAM](), [Authorization](), [Security](), [Multi-tenancy](), [Amazon S3](), [Amazon Web Services](), [Architecture & Design]() Tags: [access-control](), [aws](), [blog-post](), [ceph](), [defense-in-depth](), [en-article](), [en-blog-post](), [iam](), [identity](), [multi-tenancy](), [operations](), [production](), [rgw](), [s3](), [security]() ### AI overview This article explains why ACLs in Ceph Object Gateway should be disabled in favor of centralized IAM policies. It discusses S3 permission failures, multi-tenant access problems, AWS's ACL-disabled defaults, and defense-in-depth measures such as Block Public Access and explicit denies for ACL operations. ### Source excerpt Introduction ¶ Introduction: When Security Theater Becomes a Real Disaster ¶ In March 2017, a misconfigured S3 bucket at Verizon exposed the personal information of 14 million customers. The root cause wasn't a sophisticated attack; it was a simple oversight in access permissions. The bucket was set to be publicly accessible due to S3 permission misconfiguration, and no one noticed because ACLs were managed separately from the company's centralized IAM policies. The security team had implemented careful, identity-based access controls, but a resource-level ACL silently bypassed them by granting access to "All Users." This scenario repeats constantly across the industry: ACLs creating invisible access paths that security teams don't know exist, buckets accidentally exposed to the public internet, and contractors uploading data that the bucket owner cannot reliably read or administer, while still consuming capacity. Between 2017 and 2019, major companies exposed hundreds of millions of records via misconfigured S3 permissions (ACLs and/or bucket policies): Verizon (2017): 14 million customers - An AWS S3 bucket configured for public access exposed names, addresses, account PINs Facebook (2019): 540 million records - Third-party apps stored user data in publicly accessible S3 buckets Instagram (2019): 49 million records - Marketing firm left influencer database unprotected in AWS S3 The AWS response was clear: since April 2023, all new S3 buckets default to "ACLs disabled" (BucketOwnerEnforced) and Block Public Access enabled. AWS strongly recommends disabling ACLs on existing buckets and migrating to a pure policy-based model with IAM Accounts architecture. If you're running the Ceph Object Gateway (RGW), you have access to the same IAM Accounts model introduced in Ceph Squid 19.2.0. This post explains why ACLs must be disabled immediately and how to implement modern, secure access control with IAM policies. Do This First (Quick Security Wins) Before reading further, ## Breaking the Static Key Habit: Modernizing Ceph RGW S3 Security with STS DevFeed: [Breaking the Static Key Habit: Modernizing Ceph RGW S3 Security with STS]() Original publisher: [Read original article]() Author: Daniel Alexander Parkes, Anthony D'Atri Published: 2025-12-18T00:00:00Z Content type: tutorial Language: en Sources: [Ceph Blog]() Topics: [Security](), [Amazon S3](), [Credential theft](), [configuration](), [GitHub](), [CI/CD](), [App]() Tags: [amazon-s3](), [app](), [aws](), [blog-post](), [ceph](), [ci-cd](), [code](), [configuration](), [credential-theft](), [en-article](), [en-blog-post](), [github](), [rgw](), [s3](), [security]() ### AI overview This tutorial explains how to replace long-lived S3 credentials in Ceph Object Gateway (RGW) applications with temporary credentials issued through Security Token Service (STS). It uses the Uber breach to illustrate how stolen static keys can enable persistent unauthorized access, and discusses credentials stored in configuration files, scripts, and CI/CD variables. ### Source excerpt Introduction: The USD 148 Million Lesson ¶ In late 2016, Uber learned that intruders had accessed a trove of personal data stored in an Amazon S3 bucket. The entry point was painfully mundane: attackers accessed Uber's source code on GitHub using stolen credentials, found an AWS credential, and used it to access Uber's data. That single, long-lived credential exposed data on roughly 57 million users and 600,000 drivers. The breach was bad; the duration risk was worse. Static access keys do not expire. Once leaked, they remain active until someone notices, locates every instance in use, and rotates them. That makes credential theft uniquely dangerous in cloud and S3-style storage, because an attacker can repeatedly return, automate access, and quietly expand their footprint. Uber ultimately agreed to a $148 million multistate settlement related to how the incident was handled and disclosed. The exact dollar figure is not the main lesson, though. The lesson is this: a single static key can turn a small mistake into a durable breach. If you are running the Ceph Object Gateway (RGW), you face the same dynamic: S3 credentials in an application configuration file config.yaml, embedded in scripts, or stored in CI/CD variables. Each one is a long-lived credential that, once copied, can be used from anywhere the S3 endpoint is reachable. This post shows you how to eliminate static credentials using Security Token Service (STS) with temporary credentials that expire automatically. By the end, you'll understand how to implement the same security model that prevented these breaches from being even worse, and how to adapt it for Ceph RGW. The Static Credential Problem ¶ Let's take a look at some examples of how most applications access S3 storage today: # app-config.yaml (application config file) s3: endpoint: https://s3.example.com access_key: AKIA1234567890ABCDEF secret_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY bucket: production-data Or with the credentials embedded direc ## RocksDB Compression in Ceph: Space Savings with No Performance Cost DevFeed: [RocksDB Compression in Ceph: Space Savings with No Performance Cost]() Original publisher: [Read original article]() Author: Daniel Alexander Parkes, Anthony D'Atri Published: 2025-12-17T00:00:00Z Content type: article Language: en Sources: [Ceph Blog]() Topics: [Compression](), [Database](), [Hardware](), [ibm]() Tags: [blog-post](), [ceph](), [compression](), [configuration](), [cost](), [devices](), [en-article](), [en-blog-post](), [ibm](), [mon](), [nvme](), [osd](), [performance](), [rados](), [reef](), [rocksdb](), [space](), [storage]() ### AI overview This article reports Ceph performance tests showing that enabling RocksDB compression can substantially reduce metadata database space, especially for smaller objects, without harming throughput or resource consumption. The tests used IBM Storage Ceph 7.1 with BlueStore OSDs, HDD object storage, and NVMe devices for the RocksDB WAL and database. ### Source excerpt Introduction ¶ In the world of data storage, engineers and architects constantly face a fundamental dilemma: the trade-off between performance and efficiency. It's a balancing act. When you want to save space, you typically enable features like compression, but the common assumption is that this will cost you performance, a CPU cycle tax that slows throughput. But what if you could significantly reduce your metadata storage footprint without slowing things down? This search for an answer to this question started with research work from Mark Nelson, who published a blog post on ceph.io that covers RocksDB tuning in depth, exploring RocksDB compression with positive results. These promising results sparked a conversation on the upstream GitHub about enabling compression by default; a link to the PR is available here. To build on the previous investigation, the Ceph performance team ran tests on a robust hardware configuration running IBM Storage Ceph 7.1 (Reef). The cluster used the BlueStore OSDs for an erasure-coded (EC 4+2) pool, with a hybrid OSD storage setup: HDDs for object data and fast NVMe drives for the BlueStore WAL+DB. To understand the test, it's helpful to know what the WAL+DB is. In modern Ceph, the BlueStore storage engine manages all data on the OSDs (physical devices). To do this, it must maintain a vast catalog of internal metadata: think of it as a high-speed index that quickly locates every piece of data. RocksDB, a high-performance key-value database, manages this critical index. In our hybrid cluster, the RocksDB database runs on the fast NVMe deviceses, while the actual object data resides on the slower HDDs. Because this metadata can grow very large, RocksDB's efficiency, how much space it consumes on those expensive NVMe drives, is a critical factor in the cluster's overall cost and performance. Our test, therefore, focuses on a simple, high-stakes question: Can we compress this metadata to save space without paying a performance penalty? Ex