# object storage Published articles for object storage. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## STRG Review: Can It Make Cloud Storage Costs More Predictable? DevFeed: [STRG Review: Can It Make Cloud Storage Costs More Predictable?]() Original publisher: [Read original article]() Author: WDD Staff Published: 2026-11-06T13:46:00Z Content type: article Language: en Sources: [Web Designer Depot]() Topics: [Cloud](), [Amazon S3](), [MSP MCP](), [cost-optimization](), [restic]() Tags: [aes](), [agents](), [ai](), [ai-agents](), [ai-tools](), [ai-workloads](), [api](), [aws-cli](), [aws-s3-alternatives](), [backups](), [budgeting](), [candidate](), [cloud](), [cloud-infrastructure](), [cloud-storage](), [compatibility](), [connect](), [console](), [costs](), [developer-tools](), [egress](), [mcp](), [object-storage](), [permissions](), [s3](), [s3-compatible-storage](), [sponsored](), [sponsored-review](), [strg]() ### AI overview STRG's main cost advantage is a published $6.80 per TB monthly rate that includes egress and standard S3 requests, with the first 10 GB free and no retrieval fees or minimum storage duration listed. It supports familiar S3 tools and common object operations, but features such as SSE-KMS, IAM roles, tagging, and static website configuration are not listed as supported. Its console Assistant and MCP Server let customers scope agent access through selected credentials and permissions; STRG says prompts and customer data are not used to train general-purpose models. STRG also describes encryption, integrity checks, Object Lock, and optional cross-region replication, with EU-Central and US-East storage regions. The excerpt recommends estimating actual workload costs and piloting required operations, resilience, and recovery. The published price excludes taxes, larger deployments may have separate terms, and standard availability commitments or service credits require an executed agreement. ### Source excerpt Cloud storage has a catch: uploading your files is only part of the bill. STRG says it bundles storage, egress, and standard S3 requests for $6.80 per TB per month--and lets AI agents manage your storage with permissions you control. ## Dropbox acquires Tigris Data for $350M, releases new AI productivity features DevFeed: [Dropbox acquires Tigris Data for $350M, releases new AI productivity features]() Original publisher: [Read original article]() Author: Maria Deutscher Published: 2026-11-06T01:28:58Z Content type: news Language: en Sources: [SiliconANGLE]() Topics: [dropbox](), [Artificial Intelligence](), [AI Chat](), [Cache](), [Amazon S3]() Tags: [acquisition](), [agent](), [agents](), [ai](), [ai-agent](), [ai-agents](), [ai-cloud](), [ai-infrastructure](), [ai-models](), [ai-productivity](), [ai-workloads](), [artificial-intelligence](), [artificial-intelligence-agents](), [cloud](), [cloud-storage](), [data](), [dbx-agent-cloud](), [dropbox](), [nasdaq-dbx](), [news](), [object-storage](), [startup](), [the-latest](), [tigris-data]() ### AI overview Dropbox is acquiring Tigris Data for $350 million and introducing DBX Agent Cloud, an S3-compatible object storage service optimized for AI agents. Tigris's multi-cloud storage uses bucket forms to speed data loading while preventing agents from changing information used by other workloads; it also copies records closer to newly detected access locations and caches frequently used files on high-speed storage. Dropbox plans to add a file system and content processing capabilities later. Separately, Dropbox's file-sharing platform gains descriptive image search, natural-language file summaries and file actions such as changing formats, moving files or combining records into PDFs, including multistep workflows. These features initially roll out to Standard and Advanced subscribers in the U.S., U.K. and Australia, with broader access planned next year. ### Source excerpt Dropbox Inc. is acquiring cloud storage startup Tigris Data Inc. for $350 million. The company announced the deal today in conjunction with a major update to its product portfolio. It introduced DBX Agent Cloud, a service that artificial intelligence agents can use to store their data. Dropbox is also adding new AI productivity features to [...] The post Dropbox acquires Tigris Data for $350M, releases new AI productivity features appeared first on SiliconANGLE. ## Lakebase Postgres is built on object storage, but 99.99999% of reads never touch it DevFeed: [Lakebase Postgres is built on object storage, but 99.99999% of reads never touch it]() Original publisher: [Read original article]() Author: Carlota Soto Published: 2026-11-05T12:00:00Z Content type: article Language: en Sources: [Neon Docs]() Topics: [Amazon S3](), [Cache](), [data lake]() Tags: [2](), [3](), [about](), [accessible](), [agents](), [all](), [also](), [analytics](), [architecture](), [argument](), [but](), [conclusion](), [engineering](), [isn-t](), [lakebase](), [object](), [object-storage](), [postgres](), [right](), [running](), [storage]() ### AI overview Lakebase Postgres keeps object storage off the active query path through layered caching: the compute node serves about 99.46% of page accesses from its local cache, pageserver NVMe serves about 0.54%, and object storage handles roughly 39 requests per second--about one in 28 million page accesses. Each tier caches data fetched from the tier below. Pageservers answer GetPage@LSN requests by combining WAL deltas with image layers; after base pages were created more eagerly, p99 replay fell to around six deltas, while p90 stayed below one. Object storage fetches mainly occur on first reads after long inactivity or when accessing old branch or restore points; subsequent reads use caches. The architecture also separates compute from durable storage: safekeepers replicate WAL, pageservers persist history asynchronously, and object storage holds immutable history, enabling scale-to-zero, branching, and restores without copying data. ### Source excerpt The most common argument against running Postgres on object storage is that a Postgres backed by object storage must be a slow Postgres. The premise is right, but the conclusion isn't. ## VMware gets simpler AI deployment and low-cost version for basic private clouds DevFeed: [VMware gets simpler AI deployment and low-cost version for basic private clouds]() Original publisher: [Read original article]() Author: Paul Gillin Published: 2026-11-04T17:00:32Z Content type: news Language: en Sources: [SiliconANGLE]() Topics: [VMware Cloud Foundation](), [Advanced AI workflows](), [Disaster Recovery]() Tags: [ai](), [ai-deployment](), [also](), [artificial-intelligence](), [basic](), [broadcom](), [broadcom-ai-infrastructure-quickstart](), [broadcom-inc](), [capabilities](), [cloud](), [cloud-platform](), [company](), [confidential-computing](), [cost](), [customers](), [deployment](), [disaster](), [disaster-recovery](), [don-t](), [kubernetes](), [multi-tenant-model-sharing](), [news](), [object-storage](), [private](), [the-latest](), [vcf-9](), [version](), [virtual-machine-migration](), [virtualization](), [vmware](), [vmware-cloud-foundation](), [vmware-compliance-and-configuration-management](), [vmware-hyperarmor](), [vmware-kubernetes-service](), [vmware-live-recovery](), [vmware-vsan](), [vmware-vsphere-express]() ### AI overview Broadcom is adding AI deployment, storage, security and disaster recovery capabilities to VMware Cloud Foundation (VCF), while introducing a lower-cost virtualization option for smaller environments. AI Infrastructure Quickstart combines vCenter cluster deployment, vSphere Supervisor activation and VMware Kubernetes Service provisioning in one workflow. Multitenant model sharing, available in VCF 9.1.1, lets business units use centrally deployed models with separate security and privacy boundaries; Broadcom says this can reduce GPU needs and duplicate deployments. VCF's planned S3-compatible vSAN storage targets AI workloads using unstructured data. Confidential computing for AMD and Intel processors is already available in VCF 9.1.1; policy-based connectivity and configuration compliance are planned. Orchestrated disaster recovery is moving into core VCF and VMware vSphere Foundation. VMware vSphere Express, based on vSphere 9, supports environments up to 128 processor cores and includes migration, high availability, networking, security and simplified patching. It targets customers needing basic virtualization, with upgrades to larger platforms as needs grow; Broadcom disclosed,, ### Source excerpt Broadcom Inc. is expanding its VMware portfolio with new capabilities that simplify artificial intelligence workload deployment, strengthen infrastructure security and simplify disaster recovery. The company is also introducing a lower-cost virtualization platform aimed at smaller customers that don't need a full private cloud platform. Today's updates span VMware Cloud Foundation, the company's flagship private cloud [...] The post VMware gets simpler AI deployment and low-cost version for basic private clouds appeared first on SiliconANGLE. ## System Design in 6 Points, Explained with a Chai Stall That Went Viral DevFeed: [System Design in 6 Points, Explained with a Chai Stall That Went Viral]() Original publisher: [Read original article]() Author: Shrey Vijayvargiya Published: 2026-10-28T09:00:00Z Content type: tutorial Language: en Sources: [iHateReading]() Topics: [system-design-project](), [Cache](), [nginx](), [least connections](), [Network]() Tags: [apache-2-0](), [api](), [asynchronous](), [aws](), [azure](), [backblaze-b2](), [backend](), [backups](), [batching](), [blockchain](), [bsd](), [cache](), [cache-control](), [cache-vs-queue](), [caching](), [cdn](), [coding](), [development](), [frontend](), [javascript](), [load-balancer](), [load-balancer-explained](), [logs](), [node-js](), [object-storage](), [product](), [programming](), [queue](), [react](), [read-replicas](), [read-replicas-explained](), [retries](), [retries-with-exponential-backoff-and-jitter](), [system-design](), [system-design-basics](), [system-design-cheat-sheet](), [system-design-for-beginners](), [typescript](), [web-development]() ### AI overview The article explains six system design patterns through a viral chai stall: cache repeated reads, queue slow background work, balance traffic across healthy app servers, use read replicas for read scaling, store files in object storage behind a CDN, and add timeouts, bounded jittered retries, and structured logs around network calls. It describes key limits: caches can serve stale data and need shared storage across servers; queues defer work and require durable jobs, idempotency, and monitoring; load balancers need shared sessions and can expose database connection limits; asynchronous replicas lag and do not reduce write load; file storage has provider-specific egress costs and private files need signed URLs; retries can duplicate unsafe operations or multiply across layers. Suggested order: logs and timeouts, object storage/CDN, then cache, queue, load balancer, and replicas after query and index fixes. ### Source excerpt System design basics in six points: cache, queue, load balancer, read replicas, object storage + CDN, and retries with timeouts and logs. One chai stall story,... ## Merkle-Tree-Based Verifiable Public Data Auditing for Cloud Object Storage Using Smart Contracts DevFeed: [Merkle-Tree-Based Verifiable Public Data Auditing for Cloud Object Storage Using Smart Contracts]() Original publisher: [Read original article]() Author: Siri Reddy Tadi, K. Gangadhara Rao, M. Venu Gopalachari, C. Chetann Reddy, Jaswanth Bhimireddy Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security](), [arXiv: Distributed Computing]() Topics: [Ethereum](), [smart contract](), [Cryptography](), [Data integrity in software engineering](), [sha256](), [Amazon S3]() Tags: [audit](), [auditing](), [blockchain](), [cloud](), [contracts](), [cost](), [cs-cr](), [cs-dc](), [data](), [dataset](), [downloads](), [ethereum](), [experiments](), [files](), [hash](), [merkle-tree](), [object](), [object-storage](), [public](), [smart-contracts](), [storage](), [third-party]() ### AI overview The system audits cloud objects against a public Ethereum smart-contract record: each object's name and SHA-256 hash forms a Merkle-tree leaf, and an auditor checks a downloaded object against the stored root using a short proof. In five experiments using S3-compatible storage and a local Hardhat blockchain, it detected all tested modifications and deletions without false alarms. A proof for 10,000 objects used 14 hashes; auditing one object took about 10 ms and a 100-object sample about 150 ms, largely independent of dataset size. A full audit of 10,000 objects took 14.3 seconds, about as long as downloading them, so the reported time savings come mainly from sampling. Storing the root cost 91,863 gas, 252 times less than storing 1,000 hashes on-chain. ### Source excerpt arXiv:2610.36001v1 Announce Type: new Abstract: Owners of data in cloud object storage cannot directly check whether the provider still keeps every object unchanged. Existing research mostly checks blocks inside files or relies on third-party auditors, and less attention has been given to checking each object against a public blockchain reference while measuring the cost. A full-data audit downloads every object and compares it with a saved hash list, which is slow and only as reliable as that list. We propose a public auditing system in which the SHA-256 hash of each object and its name forms a leaf of a Merkle tree, so the whole dataset is represented by one Merkle root. This root is stored in an Ethereum smart contract as a trusted public record, while the objects stay with the provider. An auditor downloads an object, rebuilds the root using a short Merkle proof and compares it with the stored root, marking the object as intact, modified or missing. In five experiments on an S3-compatible store and a local Hardhat blockchain (normal audit, single- and multiple-object tampering, deletion and scalability), the system detected 15 of 15 single modifications, 2,205 of 2,205 modified objects and 830 of 830 deleted objects with no false alarms. The proof needed only 14 hashes for 10,000 objects, so checking one object took about 10 ms and a 100-object sample about 150 ms regardless of dataset size, while a full audit of 10,000 objects (14.3 s) took about as long as downloading them (13.5 s). Storing the root cost 91,863 gas, 252 times less than storing 1,000 hashes on-chain. The system thus shows exactly which objects were modified or are missing, with its main savings coming from checking selected objects. ## Spectra Logic Storage Reclaim Packages Tape, BlackPearl, and Data Movers Into Six Configurations From 1PB to 24PB to Free Flash DevFeed: [Spectra Logic Storage Reclaim Packages Tape, BlackPearl, and Data Movers Into Six Configurations From 1PB to 24PB to Free Flash]() Original publisher: [Read original article]() Author: Harold Fritts Published: 2026-10-27T16:44:07Z Content type: release Language: en Sources: [StorageReview.com]() Topics: [save](), [data migrations](), [Tool](), [Amazon S3]() Tags: [analysis](), [arrays](), [capacity](), [enterprise](), [enterprise-storage](), [flash](), [object-storage](), [storage](), [unstructured-data]() ### AI overview Spectra Storage Reclaim analyzes files on flash and disk in a read-only pass, applies access, retention, and business rules to identify inactive data, then moves qualifying files to a protected tier while keeping them visible and recallable. Reclaiming capacity returns the freed primary storage to active workloads. Customers can use Spectra's StorCycle, or the approved tools Komprise, Starfish Storage, and rclone. The destination is a BlackPearl Object Gateway with S3-compatible access and policy-based placement across flash, disk, and tape; tape provides long-term retention with a physical air gap and no power draw while at rest. Six packaged configurations range from 1.05PB to 24.3PB and include assessment, planning, installation, and three years of 5x9 next-business-day support. Spectra says configuration choice should be validated for each environment; the analysis determines how much capacity is actually reclaimable. ### Source excerpt Spectra Logic today announced availability of Spectra Storage Reclaim, a packaged combination of analysis software, data movers, BlackPearl object storage, tape, and services that finds inactive unstructured data on flash and disk arrays and moves it to tape so the primary capacity can go back to active workloads. The pitch is timed to the flash The post Spectra Logic Storage Reclaim Packages Tape, BlackPearl, and Data Movers Into Six Configurations From 1PB to 24PB to Free Flash appeared first on StorageReview.com. ## AI storage and LLMs: 4 critical needs to look out for DevFeed: [AI storage and LLMs: 4 critical needs to look out for]() Original publisher: [Read original article]() Author: CoreWeave Published: 2026-10-22T16:27:39Z Content type: article Language: en Sources: [Coreweave Blog Posts]() Topics: [AI Architecture](), [big-data](), [enterprise data protection](), [Cache](), [Disaster Recovery]() Tags: [ai-storage](), [content-type](), [coreweave-ai-object-storage](), [llms](), [object-storage](), [recovery](), [scalability](), [security]() ### AI overview The article argues that LLM workloads need storage that can keep GPUs supplied with data, recover quickly from interruptions, scale with compute, and protect sensitive datasets. CoreWeave says its AI Object Storage uses LOTA, a proxy on GPU nodes that bypasses object storage gateways and indexes and caches data locally, to speed access. It cites up to 2 GB/s per GPU, and 25 GB/s throughput and 5,000 requests per second per customer account for each petabyte reserved; it also claims 99.9% uptime and eleven nines of durability, alongside observability and auditing. The service is described as scaling to hundreds of thousands of GPUs while maintaining 2 GB/s per GPU. For security, the article names encryption at rest and in transit, identity access management, authentication, and role-based access controls. These are product claims in the supplied text; it offers no independent comparison or verification. ### Source excerpt Learn about what object storage needs for LLMs: fast access, quick recovery, scalability, and strong security--and how CAIOS provides it all. ## Expanding CoreWeave AI Object Storage: Unified Dataset, Lower Costs, and Unmatched Scale DevFeed: [Expanding CoreWeave AI Object Storage: Unified Dataset, Lower Costs, and Unmatched Scale]() Original publisher: [Read original article]() Author: CoreWeave Published: 2026-10-22T16:27:39Z Content type: release Language: en Sources: [Coreweave Blog Posts]() Topics: [cost-optimization](), [big-data]() Tags: [accelerated](), [acceleration](), [agentic](), [ai](), [ai-cloud](), [ai-labs](), [ai-native](), [ai-workloads](), [analyst](), [automated](), [content-type](), [coreweave](), [coreweave-ai-object-storage](), [costs](), [dataset](), [datasets](), [global](), [object-storage](), [pricing](), [scale](), [storage]() ### AI overview CoreWeave says its AI Object Storage now supports one dataset across CoreWeave regions, public clouds, and on-premises environments without replication or synchronization overhead. Its Local Object Transport Accelerator (LOTA) is claimed to deliver 7 GB/s per GPU, scaling linearly with workloads. New automated Hot, Warm, and Cold usage-based billing applies to current customers and is said to cut storage costs by over 75% for a typical AI workload, without manual data tiering or archived-data access fees. CoreWeave says cross-region LOTA availability begins October 16, with multi-cloud and hybrid expansion planned for early 2026; the source does not specify a year for October 16. ### Source excerpt CoreWeave expands AI Object Storage with unified global datasets and automated pricing that cuts storage costs by over 75%. ## AI Storage Without Limits: Exploring the Latest CoreWeave AI Object Storage Expansions DevFeed: [AI Storage Without Limits: Exploring the Latest CoreWeave AI Object Storage Expansions]() Original publisher: [Read original article]() Author: CoreWeave Published: 2026-10-22T16:27:39Z Content type: article Language: en Sources: [Coreweave Blog Posts]() Topics: [AI Architecture](), [Amazon S3](), [Infrastructure as a Service (IaaS)](), [gpu-initated]() Tags: [ai](), [ai-developers](), [ai-infrastructure](), [ai-networking](), [ai-storage](), [ai-workloads](), [architecture](), [authentication](), [automated](), [availability](), [cache](), [caching](), [content-type](), [coreweave](), [coreweave-ai-object-storage](), [global](), [gpu](), [object](), [object-storage](), [storage](), [throughput]() ### AI overview CoreWeave says its managed AI Object Storage combines LOTA, a cache running on GPU cluster nodes, with distributed parallel reads and writes to deliver up to 7 GB/s per GPU. The announced expansion lets customers access a single dataset across CoreWeave regions, third-party clouds, and on-premises environments, aiming to reduce replication and data divergence; LOTA support for other clouds and on-premises is scheduled for early 2026. The service advertises no ingress, egress, or request fees and automatically assigns data to hot, warm, or cold billing levels by access frequency; CoreWeave says typical AI workloads for existing customers can see storage costs fall by more than 75%. The text also states 99.9% uptime, eleven nines of durability, encryption, role-based access, SAML/SSO, and Prometheus/Grafana observability. These are provider-reported capabilities and claims. ### Source excerpt AI Storage Without Limits: CoreWeave Expands Object Storage with Global Access and 7 GB/s/GPU Throughput ## GTC 2025: A Recap of an Awe-Inspiring Week DevFeed: [GTC 2025: A Recap of an Awe-Inspiring Week]() Original publisher: [Read original article]() Author: CoreWeave Published: 2026-10-22T16:27:39Z Content type: news Language: en Sources: [Coreweave Blog Posts]() Topics: [Artificial Intelligence](), [systems](), [GB200](), [InfiniBand](), [performance-engineering](), [BlueField DPU](), [RTX PRO](), [NVIDIA Blackwell]() Tags: [2025](), [ai](), [content-type](), [coreweave](), [coreweave-ai-object-storage](), [gb200](), [gtc](), [highlighting](), [launches](), [like](), [major](), [nvidia](), [nvl72](), [nvl72-racks](), [object](), [object-storage](), [partnerships](), [product](), [recap](), [storage](), [tech]() ### AI overview CoreWeave's GTC 2025 recap highlights rising AI computing demand, large scale GPU infrastructure, and product launches. Sessions covered scaling NVIDIA Quantum-2 InfiniBand networks, GPU cluster practices such as model loading, checkpointing, and fleet management, and using BlueField-3 DPUs and automation to improve training resilience. Announcements included general availability of CoreWeave AI Object Storage, planned RTX PRO 6000 Blackwell instances, and support for NVIDIA AI Enterprise and Cloud Functions on CoreWeave, including on GB200 NVL72. The recap reports early GB200 NVL72 throughput gains over HGX H100 and a CoreWeave study finding over 50% MFU on H100, with up to 1.2x higher performance than alternative solutions; it gives no benchmark methodology or detailed comparative results. ### Source excerpt Check out our GTC 2025 recap, highlighting AI breakthroughs, new partnerships, keynotes, tech demos, and major product launches like CoreWeave AI Object Storage and NVIDIA GB200 NVL72 racks. ## Benchmark Results: CoreWeave AI Object Storage Delivers 2+ GB/s per GPU Throughput Across any Number of GPUs DevFeed: [Benchmark Results: CoreWeave AI Object Storage Delivers 2+ GB/s per GPU Throughput Across any Number of GPUs]() Original publisher: [Read original article]() Author: CoreWeave Published: 2026-10-22T16:27:39Z Content type: article Language: en Sources: [Coreweave Blog Posts]() Topics: [Benchmark](), [Multi-GPU](), [software delivery metrics](), [NVMe](), [Hopper](), [distributed-training]() Tags: [ai](), [ai-model-training](), [ai-storage](), [ai-workflows](), [ai-workloads](), [benchmark](), [benchmarking](), [benchmarks](), [cache](), [caching](), [cloud-storage](), [cluster](), [content-type](), [coreweave](), [coreweave-ai-object-storage](), [gpu](), [object-storage](), [performance](), [scale](), [storage](), [throughput]() ### AI overview CoreWeave reports that CAIOS read throughput reached 2.3 GiB/s per GPU after its Local Object Transport Accelerator (LOTA) cache filled in a 20-node NVIDIA H200 test. Each node used a 1 TiB local NVMe cache; Warp wrote 10,000 50 MiB objects per node, then performed random reads for 10 minutes. Aggregate throughput rose from about 24 GiB/s during cache warm-up to 368 GiB/s, with subsequent reads served from local storage. The benchmark used CPU-driven I/O, not direct GPU reads, and measured read performance; write benchmarking is described as future work. The article says workloads can use the cwlota.com S3-compatible endpoint without custom code and that accessed objects are cached in full, with dataset pre-staging also supported. Its claim that performance scales to any number of GPUs is an expectation based on local storage performance, rather than a result demonstrated by this 20-node test. ### Source excerpt Discover how CAIOS achieves 2+ GB/s per GPU throughput--redefining performance for AI model training at scale. ## CoreWeave Launches AI Object Storage: A Cutting-Edge Object Storage Service Optimized for AI Workloads DevFeed: [CoreWeave Launches AI Object Storage: A Cutting-Edge Object Storage Service Optimized for AI Workloads]() Original publisher: [Read original article]() Author: CoreWeave Published: 2026-10-22T16:27:39Z Content type: release Language: en Sources: [Coreweave Blog Posts]() Topics: [AI Infrastructure](), [GPU](), [Cache]() Tags: [accelerate](), [access-control](), [ai](), [ai-and-ml](), [ai-infrastructure](), [ai-model](), [ai-training](), [ai-training-and-inference](), [ai-workloads](), [content-type](), [coreweave](), [coreweave-ai-object-storage](), [gpu](), [high-performance](), [inference](), [integration](), [launch](), [launches](), [object-storage](), [performance](), [training]() ### AI overview CoreWeave AI Object Storage is a managed, S3-compatible service for AI training and inference, designed to scale to exabytes and support up to 2 GB/s per GPU. Its LOTA feature caches frequently used datasets or prestages them on GPU nodes' local NVMe disks to reduce network latency; the text says it works without extra tools or complex configuration. The service integrates with CoreWeave NVIDIA GPU clusters, Kubernetes Service, SAML/SSO, and Terraform, and includes managed Grafana dashboards for monitoring request rates, throughput, and latency. CoreWeave lists encryption, role-based access controls, audit logging, lifecycle policies, and pricing without egress or per-request fees. The performance figures and benefits are vendor claims in the supplied launch text. ### Source excerpt Learn more about the launch of CoreWeave AI Object Storage, a high-performance, scalable, and secure storage service built to accelerate AI training and inference with seamless GPU integration. ## WaRP: Workload-Aware Request Planning for Vector Retrieval on Object Storage DevFeed: [WaRP: Workload-Aware Request Planning for Vector Retrieval on Object Storage]() Original publisher: [Read original article]() Author: Ruipeng Wan, Xinyu Zeng, Yiming Qiao, Huanchen Zhang Published: 2026-10-22T04:00:00Z Content type: article Language: en Sources: [arXiv: Databases]() Topics: [ann](), [Amazon S3](), [Vectors & LLMs]() Tags: [ann](), [aws](), [concurrency](), [cs-db](), [latency](), [object-storage](), [optimization](), [planning](), [request](), [s3](), [scheduler](), [vector]() ### AI overview WaRP plans how to merge vector-cluster byte ranges into object-storage requests, reducing per-request overhead and billed GETs while preserving ANN recall. It uses sorted intervals, a per-query request limit, and an estimate of request concurrency to optimize a latency surrogate; its dynamic program and adaptive frontier search return a plan within (1+ε) of optimal, with planning under 10 ms in the most demanding tested settings. An offline reordering pass improves mergeability without changing ANN semantics. In an AWS S3 evaluation integrated with Lance's I/O scheduler across seven IVFPQ datasets, WaRP plus reordering achieved 1.09-3.48x higher QPS and 2.12-5.19x fewer GETs than Lance, with the same recall. ### Source excerpt arXiv:2610.29689v1 Announce Type: new Abstract: Modern vector search systems keep payload vectors in cloud object storage and only a compact routing index on fast local storage, typically using cluster-based ANN indexes. This design inherits a problem existing systems do not address. On object storage, small Range-GETs have latency dominated by fixed per-request overhead and are billed per request. Existing readers either issue one request per probed cluster, or coalesce adjacent ranges with hand-tuned gap or span thresholds that do not transfer across datasets or query workloads. We formulate the grouping of probed-cluster byte intervals into object-store requests as a global per-query optimization problem and present WaRP, a workload-aware request planner that solves it efficiently. Given the sorted byte intervals, a limit on the number of requests per query, and an estimate of how many object-store requests from one query can proceed concurrently under the target serving configuration, WaRP searches for a merge plan that minimizes a concurrency-aware latency surrogate while satisfying the request limit. Using an optimized dynamic program and adaptive frontier search, WaRP returns a plan within $(1{+}\epsilon)$ of the optimum, with per-query planning overhead under $10$ms even in the most demanding tested settings. A lightweight offline reordering pass further improves request mergeability without changing the semantics of ANN search. We implement WaRP inside Lance's I/O scheduler and evaluate it on AWS S3. We build IVFPQ indexes on seven datasets, including SIFT100M and DEEP100M at the 100M-vector scale. WaRP + Reorder improves QPS by $1.09--3.48\times$ and reduces GETs by $2.12--5.19\times$ relative to Lance, while preserving Lance's recall. ## SBD: We Have a Filesystem at Home DevFeed: [SBD: We Have a Filesystem at Home]() Original publisher: [Read original article]() Author: Daniel Botha Published: 2026-10-22T00:00:00Z Content type: release Language: en Sources: [Fly]() Topics: [Ext4](), [Persistence](), [Disk image](), [Integration testing](), [mount](), [NVMe]() Tags: [block](), [cdn](), [close-to-users](), [database](), [deploy-app-servers](), [device](), [docker](), [elixir](), [ext4](), [filesystem](), [fly](), [fly-io](), [heroku-alternative](), [heroku-competitor](), [hosting](), [i](), [infrastructure](), [linux](), [metadata](), [networking](), [object](), [object-storage](), [postgresql-clusters](), [reliability](), [servers](), [storage](), [the-fly-blog]() ### AI overview Sprites replaces its heavily modified JuiceFS setup with the Sprites Block Device (SBD): a daemon presents bucket-backed storage to Linux as an NBD block device, which Sprite formats as ext4. The prior design stored a 100 GB disk image and relied on separate JuiceFS metadata and checkpoint databases; writeback could leave metadata pointing to an object that was never uploaded, and cloned checkpoints could share that missing chunk. SBD keeps volume metadata with the blocks in the bucket, caches locally, uploads dirty blocks in the background, and flushes when durability must be guaranteed. Checkpoints record block snapshots and volume versions in SBD's catalog; restore rewinds the volume in place. SBD verifies blocks read from storage and restarts the Sprite if its daemon dies, favoring a clean restart over continued use of an untrusted disk. The post reports that stress testing and design partner feedback found SBD orders of magnitude more reliable, and says it will replace the old filesystem. ### Source excerpt I have unkind things to say about people who make their own pasta. Here, I'll only use them as an example of how not to think when you're building infrastructure. It's unfairly easy to believe that the harder you work on something, the better it gets. But infra is about making the $1 bag of spaghetti: a simple thing that works well (and doesn't require you to spend a week scraping flour out of the corners of your kitchen). When work on Sprites started, it was a new kind of infra for a new kind of product. So when we couldn't find the right filesystem in the pasta section, we made our own. It was clever, and that's exactly why we didn't like it. This post introduces its replacement, the Sprites Block Device (SBD). It's the bag of spaghetti you grab off the shelf, and the only difference you'll notice is better reliability. It was always object storage Our stance on one core aspect of Sprites' storage hasn't changed from v1: the root of a Sprite's disk is object storage, not the NVMe in whatever server the Sprite happens to be running on. That's what lets a Sprite sleep, wake on a different server, and still find its files where it left them. The durable state of a Sprite is, more or less, a location in a bucket. We wrote about why, at length, with feeling. The thing about object stores is they can't change part of an object. To change one byte, you have to replace the whole thing. That's a bad cultural fit for Linux programs which rewrite a few bytes in the middle of a file all the time. Something has to handle that translation. At launch, that something was a heavily modded version of JuiceFS. In stock form, JuiceFS splits storage into the data (as chunks in the bucket) and the metadata (a map of which chunks make up which files) in a database. That's not how we used it, though. Our hot rodded version ran a rewritten SQLite metadata backend, kept durable with Litestream. It stored exactly one file: a 100 GB disk image, mounted on a loop device with ext4 inside. Linu ## The bridge to agent-scale Git infrastructure DevFeed: [The bridge to agent-scale Git infrastructure]() Original publisher: [Read original article]() Author: Eleanor White Published: 2026-10-21T16:00:00Z Content type: article Language: en Sources: [GitHub Blog](), [GitHub Engineering]() Topics: [Git](), [Azure](), [Big data infrastructure](), [Infrastructure as a Service (IaaS)]() Tags: [2](), [2025](), [agent](), [architecture-optimization](), [azure](), [blog](), [bridge](), [copy](), [engineering](), [first](), [git](), [github](), [here](), [infrastructure](), [object](), [object-storage](), [over](), [post](), [reference](), [repositories](), [scale](), [storage](), [technical-deep-dive]() ### AI overview GitHub says per-reference write quorums enabled its busiest repositories to move to Azure despite roughly 25 ms cross-region round trips. The old repository-wide lock made each write wait for the slowest healthy replica; serializing only updates to the same reference lets independent branch updates proceed concurrently. In realistic internal tests, writes to different references achieved 3-4x higher throughput at lower latency. To scale reads separately, GitHub adds Azure Blob Storage as a virtual quorum replica: writes are acknowledged only after becoming durable there, while ephemeral read replicas warm caches from storage and serve traffic without joining the quorum. The added durability step costs some write latency, and background reconciliation limits which read workloads virtual replicas currently support. GitHub identifies quorum-bound write throughput and reconciliation for broader read support as remaining challenges. ### Source excerpt We moved a copy of 99.99999% of GitHub's repositories to Azure. Here's how per-reference quorums will bring over the rest, and how object storage lets reads scale independently. The post The bridge to agent-scale Git infrastructure appeared first on The GitHub Blog. ## Stop renting access to your own files DevFeed: [Stop renting access to your own files]() Original publisher: [Read original article]() Author: Ahson Shaikh Published: 2026-10-16T09:12:31Z Content type: article Language: en Sources: [DevOps.dev]() Topics: [Self Hosted File Indexing and Searching](), [dropbox](), [on-prem]() Tags: [business](), [drive](), [dropbox](), [files](), [google](), [google-drive](), [google-one](), [object-storage](), [onedrive](), [r2](), [s3](), [self-hosted](), [self-hosting](), [server](), [storage](), [yourdrive]() ### AI overview yourdrive is a self-hosted web drive that puts a familiar file interface over object storage in an account the user controls. It runs in Docker on a Linux server and supports AWS S3, Azure Blob Storage, Cloudflare R2, Wasabi, MinIO, and other S3-compatible services. Browsers upload directly to the bucket, so files do not use the server's disk. The article contrasts its per-server licensing with per-user plans: Community is free for up to 3 users, Team costs $119/year for up to 25, and Business is listed at $599/year for up to 250, with SAML SSO, audit log, and groups described as coming soon. It presents this as a lower-cost option for small teams with substantial storage, while acknowledging that a household under Google One's 2 TB cap may pay less with Google. Users must maintain a Linux server; the product is a web app rather than a desktop sync client or office suite. ### Source excerpt Why I built yourdrive, a self-hosted drive that keeps your photos, videos and documents in a bucket you already own. Count the drives you pay for. A personal Google plan for the family photos. A Dropbox seat for every person on your team. Maybe a OneDrive that came bundled with something else and now holds files you forgot about. Each one charges you every month, per person, for access to files you made. Stop paying and the files sit behind a login you no longer control. That is not storage. That is rent. I built yourdrive because I wanted the opposite: a drive that looks and feels like the one you already use, but runs on your own server and keeps every file in your own bucket. No per-seat fees. No middleman holding the keys. A drive that's actually yours. The quiet problem with rented drives Google Drive is fine for one household. Extra storage comes through Google One, at $99.99 a year for 2 TB. But that is a personal plan with a hard cap, and the moment a team needs a shared drive, the pricing switches to seats. Team drives are priced per person, not per file. Dropbox Standard costs $15 per user per month. Box Business costs $15 per user per month on an annual plan. Hire someone, and the bill goes up whether they upload one file or ten thousand. That model hides three costs: The bill tracks headcount, not storage. A 25-person team pays $4,500 a year on either plan, before anyone uploads a file. Your files live under someone else's terms. Prices, limits and features change on their schedule. You adapt. Leaving is a project. Moving years of photos and documents out is slow enough that most people never do it. Meanwhile, plenty of teams already pay for object storage: an S3 bucket for backups, an R2 bucket behind a website. It is cheap, durable and in their own account. What it lacks is a friendly drive on top, one a designer or an accountant can use without touching a terminal. What yourdrive is yourdrive is a self-hosted drive for photos, videos and documents. Yo ## Apache Iceberg Architecture Explained DevFeed: [Apache Iceberg Architecture Explained]() Original publisher: [Read original article]() Published: 2026-10-16T08:00:04Z Content type: article Language: en Sources: [Starburst Data]() Topics: [Apache Iceberg](), [parquet](), [data lake](), [table-partitioning](), [Amazon S3]() Tags: [2](), [3](), [analytics](), [apache](), [apache-iceberg](), [apache-iceberg-architecture](), [architecture](), [format](), [high-performance](), [iceberg](), [object](), [object-storage](), [open](), [performance](), [storage](), [table](), [table-format]() ### AI overview Apache Iceberg tracks tables through a catalog pointer to immutable metadata and data files, rather than inferring table contents from directory layouts. Reads use snapshot and manifest statistics to prune files; writes create new files and metadata, then use an optimistic, atomic catalog pointer swap. This supports concurrent engine access, snapshots and time travel, while row level deletes and updates use delete files. The document distinguishes v2 position and equality deletes from v3 deletion vectors, which replace position deletes, and notes v3's VARIANT type and other additions. Engine and catalog compatibility must be checked before upgrading. Ongoing maintenance--compaction, snapshot expiration, orphan file removal and manifest optimization--helps control small file fragmentation, metadata growth and storage costs. ### Source excerpt Apache Iceberg is an open table format designed for high-performance analytics running on object storage. ... ## Disaster Recovery for Apache Polaris: Backup and Restore DevFeed: [Disaster Recovery for Apache Polaris: Backup and Restore]() Original publisher: [Read original article]() Author: Alex Merced Published: 2026-10-12T19:28:19Z Content type: tutorial Language: en Sources: [Dremio]() Topics: [Disaster Recovery](), [parquet](), [Amazon S3](), [Apache Cassandra](), [RDS for PostgreSQL]() Tags: [apache-iceberg](), [apache-polaris](), [apache-software-foundation](), [backup](), [backups](), [cloud](), [configuration](), [data-lakehouse](), [database](), [disaster-recovery](), [dremio-blog-open-data-insights](), [object-storage]() ### AI overview This guide explains disaster recovery for Apache Polaris by treating the metastore database and Iceberg object storage as separate systems. It recommends point-in-time database recovery, object versioning and replication, and restoring storage before the metastore so catalog references resolve. It also covers realm-level recovery limits and table-level rollback options. ### Source excerpt Recovering Polaris means recovering a metastore and object storage separately. What to back up, the order to restore in, and the failure mode nobody plans for. The post Disaster Recovery for Apache Polaris: Backup and Restore appeared first on Dremio. ## How I Fixed a Data Ingestion Bottleneck With Date Partitioning DevFeed: [How I Fixed a Data Ingestion Bottleneck With Date Partitioning]() Original publisher: [Read original article]() Author: Rahul N Published: 2026-10-11T19:36:29Z Content type: tutorial Language: en Sources: [HackerNoon]() Topics: [data-engineering](), [Data Quality](), [table-partitioning]() Tags: [compatibility](), [data](), [data-engineering](), [data-ingestion](), [data-ingestion-pipelines](), [data-partitioning](), [data-pipeline-reliability](), [delivery](), [distributed-systems](), [exactly-once](), [failed](), [idempotent-data-processing](), [incremental-data-ingestion](), [object-storage](), [partitioning]() ### AI overview A data engineer worked with a vendor to add date-partitioned exports, letting the consumer target date-specific prefixes instead of enumerating the full export history. The article explains the benefits for discovery, retries, and backfills, along with late-arrival policies, replay safety, and migration planning. It describes an engineering approach rather than a benchmark or a guarantee of runtime improvements. ### Source excerpt I fixed a recurring data ingestion bottleneck by working with a vendor to introduce date-partitioned exports and reduce unnecessary file discovery. ## Cloudflare K2 Builds Event Streams on R2 Object Storage, at a Second of Produce Latency DevFeed: [Cloudflare K2 Builds Event Streams on R2 Object Storage, at a Second of Produce Latency]() Original publisher: [Read original article]() Author: Steef-Jan Wiggers Published: 2026-10-09T05:06:00Z Content type: news Language: en Sources: [InfoQ]() Topics: [Event-Streaming](), [Apache Kafka](), [Amazon S3](), [Cloudflare Workers]() Tags: [amazon-msk](), [architecture](), [architecture-design](), [cloud](), [cloudflare](), [cloudflare-k2-serverless-streams](), [development](), [devops](), [event-driven-architecture](), [latency](), [news](), [object-storage](), [r2](), [serverless](), [storage](), [streaming]() ### AI overview Cloudflare's K2 is a serverless event streaming service in public beta that builds a durable log on R2 object storage. Cloudflare reports about one second of p99 produce latency; a tester reported p99 end-to-end delivery latency of 7.5 seconds, which a team representative said was higher than expected. The article discusses K2's storage architecture, pricing, API design, and planned features. ### Source excerpt Cloudflare has launched K2 in public beta, a serverless event streaming service implementing a durable log on R2 object storage. Cloudflare states produce latency of about one second at p99; a tester reported end-to-end p99 of 7.5 seconds, which a commenter answering for the team said was higher than expected. Consume charges match produce charges per GB. By Steef-Jan Wiggers ## Scality AI Inference Factory Serves KV Cache From Object Storage Over RDMA, 14x Faster Than Recompute DevFeed: [Scality AI Inference Factory Serves KV Cache From Object Storage Over RDMA, 14x Faster Than Recompute]() Original publisher: [Read original article]() Author: Harold Fritts Published: 2026-10-08T16:32:44Z Content type: news Language: en Sources: [StorageReview.com]() Topics: [AI Infrastructure](), [vllm](), [rdma](), [gpudirect rdma](), [Dynamo-Triton](), [Amazon S3](), [versioning]() Tags: [ai](), [ai-data-infrastructure](), [ai-inference](), [bandwidth](), [code](), [concurrent](), [enterprise](), [infrastructure](), [object-storage](), [stack]() ### AI overview Scality launched AI Inference Factory, an open-code stack for running open-weight models on customer-owned infrastructure. It uses ADI object storage as a shared KV cache, with vLLM and Dynamo separating prefill and decode, and an RDMA path for moving cache data to GPUs. In a small lab, Scality reported that restoring a 14K-token context took 166 ms, compared with 2.3 seconds to recompute it; it also reported serving 8 TB of KV cache with first-token latency between 355 and 373 ms across the tested cache sizes. ### Source excerpt Scality today launched Scality AI Inference Factory, an open-code software stack for running open-weight models on infrastructure the customer owns. Scality validates, ships, and maintains the whole stack, which places its AI Data Infrastructure (ADI) object storage under the inference layer as a shared key-value (KV) cache, and it is aimed at enterprises, government agencies, The post Scality AI Inference Factory Serves KV Cache From Object Storage Over RDMA, 14x Faster Than Recompute appeared first on StorageReview.com. ## A DuckDB Database with No Data in It DevFeed: [A DuckDB Database with No Data in It]() Original publisher: [Read original article]() Author: The DuckDB team Published: 2026-10-07T00:00:00Z Content type: article Language: en Sources: [DuckDB]() Topics: [DuckDB](), [parquet](), [data lake](), [semantic-layer](), [SQL](), [Database]() Tags: [catalog](), [data-lake](), [database](), [duckdb](), [object-storage](), [parquet-files](), [using-duckdb](), [view]() ### AI overview A DuckDB database can act as a small, shareable catalog of views over remote Parquet files without storing their rows. Install `httpfs`, create a persistent database with views that use fully qualified URLs or paths, then publish the file; consumers attach it read-only and query the named relations. The publisher can update view definitions to accommodate repartitioned or renamed files while keeping consumer queries stable. Consumers need access to both the catalog and every referenced data source, plus suitable credentials for S3. HTTPS and S3 attachments are read-only, so edits require updating a local copy and uploading it again. Query performance depends on network conditions and file layout. ### Source excerpt A DuckDB database file does not have to hold any data. It can store nothing but view definitions that point at Parquet files on object storage, so a few hundred kilobytes behave like a shared catalog over a whole data lake. This post shows how to build one, host it, and attach it read-only. ## Dell's AI Data Platform gets a knowledge graph for agents and faster Nvidia processing DevFeed: [Dell's AI Data Platform gets a knowledge graph for agents and faster Nvidia processing]() Original publisher: [Read original article]() Author: Duncan Riley Published: 2026-10-06T13:00:11Z Content type: news Language: en Sources: [SiliconANGLE]() Topics: [semantic-layer](), [Local AI](), [Artificial Intelligence](), [GPUDirect](), [NVIDIA NIM](), [databricks]() Tags: [ai](), [ai-agents](), [apache-arrow](), [apache-spark](), [arthur-lewis](), [artificial-intelligence](), [benchmarks](), [big-data](), [cudf](), [data-processing](), [dell](), [dell-ai-data-platform](), [dell-ai-data-platform-event-from-ambition-to-ai-at-scale-2026](), [dell-data-processing-engine](), [dell-powerscale](), [dell-storage-performance-tool](), [dell-technologies](), [enterprise-knowledge-graph](), [graphics-processing-units](), [infrastructure-solutions-group](), [knowledge-agents](), [knowledge-graph](), [multitenancy](), [mutual-transport-layer-security](), [nemotron-retriever](), [news](), [nvidia](), [object-storage](), [ontologies](), [poweredge-r770](), [role-based-access-control](), [rtx-pro-4500-blackwell-server-edition](), [semantic-layer](), [the-latest](), [top-story-4](), [unified-semantic-layer]() ### AI overview Dell expanded its AI Data Platform with a semantic layer, enterprise knowledge graph, and scoped Knowledge Agents intended to provide AI agents with trusted context from company data. It also announced GPU accelerated data processing, PowerScale multitenancy and security updates, and an S3-compatible storage benchmarking tool. Dell reports its September tests found Apache Spark jobs averaged 3.9 times faster on Nvidia GPUs than on CPUs alone; the announced features have staggered availability dates extending into 2027. ### Source excerpt Dell Technologies Inc. today expanded the Dell AI Data Platform with a semantic layer and a knowledge graph intended to give artificial intelligence agents trusted context from a company's own data. Dell is also speeding up data processing on Nvidia Corp. graphics chips. The new features are aimed at enterprise data that was never organized [...] The post Dell's AI Data Platform gets a knowledge graph for agents and faster Nvidia processing appeared first on SiliconANGLE. [Next page]()