# RAPIDS

NVIDIA's open-source GPU-accelerated data science software suite, including libraries for DataFrames, machine learning, and graph analytics.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Streaming Right Multiplication over Grammar-Compressed Matrices: A Memory-Bounded GPU Engine for Genotype and Graph Data

DevFeed: [Streaming Right Multiplication over Grammar-Compressed Matrices: A Memory-Bounded GPU Engine for Genotype and Graph Data](<https://devfeed.tech/articles/streaming-right-multiplication-over-grammar-compressed-matrices-a-memory-bounded-gpu-engine-for-genotype-and-graph-data-113764.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.24971>)

Author: Francesco Tosoni, Gabriele Mencagli

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [Graphs](<https://devfeed.tech/topics/graphs.md>)

Tags: [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-ds](<https://devfeed.tech/tags/cs-ds.md>), [cs-ms](<https://devfeed.tech/tags/cs-ms.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [engine](<https://devfeed.tech/tags/engine.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grammar](<https://devfeed.tech/tags/grammar.md>), [graph](<https://devfeed.tech/tags/graph.md>), [matrices](<https://devfeed.tech/tags/matrices.md>), [memory](<https://devfeed.tech/tags/memory.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

### AI overview

The GPU engine computes right products over grammar-compressed matrices with a bottom-up gather sweep. Pass-through completion layers the SLP DAG, allowing each level to read from one buffer and write to another; memory use is bounded by two alternating buffers rather than the whole grammar. For genotype polygenic scores, the CUDA implementation uses 3.5-8x less device memory than materialized cuSPARSE CSR, runs single-vector products within a small factor of cuSPARSE, and uses less GPU energy. The same associative, commutative combine framework supports other commutative-monoid computations; its reachability sweep is also demonstrated on a billion-edge Software Heritage graph. Structural metrics are architecture-independent, while timing and energy results come from a single GPU board.

### Source excerpt

arXiv:2607.24971v2 Announce Type: replace-cross Abstract: Grammar-compressed matrices store a matrix's non-zero structure as a RePair straight-line program (SLP), supporting matrix-vector products in time and space proportional to the compressed size. We target the regime where this is decisive on a GPU: when the uncompressed matrix exceeds device memory, so footprint (not floating-point throughput) is the binding constraint. Our SLP is a directed acyclic graph (DAG) of out-degree 2, and the right product $y=Mx$ is a single bottom-up sweep (leaves to roots): a conflict-free gather. We make the grammar properly layered (every nonterminal child one level below its parent) via pass-through completion, which inserts identity nodes to carry values upward until consumed. This yields a streaming evaluation in which each level reads only the level below and writes the next, so the live set fits in two alternating read-only/write-only buffers instead of scaling with the whole grammar. On genotype matrices, where a polygenic score is exactly the right product $y=G\beta$, a CUDA implementation shows a clear space advantage: a device footprint 3.5 to 8 times smaller than a materialized cuSPARSE CSR baseline, single-vector times within a small factor of cuSPARSE, and consistently lower GPU energy. Because the sweep needs only an associative and commutative combine, the same engine evaluates any homomorphism into a commutative monoid over the grammar by swapping a small leaf/combine/emit policy; the same reachability sweep scales to a billion-edge Software Heritage graph (261 TB dense, 21 times smaller serialized than cuSPARSE's CSR). We frame this as an algorithm-engineering case study: structural metrics (depth, live-set width, completion cost) are measured, architecture-independent grammar properties, whereas time and energy are profiled on a single board.

## Turbo-GMM: FlashAttention-Inspired GPU-Accelerated Gaussian Mixture Model

DevFeed: [Turbo-GMM: FlashAttention-Inspired GPU-Accelerated Gaussian Mixture Model](<https://devfeed.tech/articles/turbo-gmm-flashattention-inspired-gpu-accelerated-gaussian-mixture-model-113812.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02449>)

Author: Momin Abbas, Sadia Asif, Asad Ul Haq, Kaoutar El Maghraoui

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accelerated](<https://devfeed.tech/tags/accelerated.md>), [approach](<https://devfeed.tech/tags/approach.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [axis](<https://devfeed.tech/tags/axis.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [but](<https://devfeed.tech/tags/but.md>), [chip](<https://devfeed.tech/tags/chip.md>), [complete](<https://devfeed.tech/tags/complete.md>), [component](<https://devfeed.tech/tags/component.md>), [components](<https://devfeed.tech/tags/components.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [flashattention](<https://devfeed.tech/tags/flashattention.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-accelerated](<https://devfeed.tech/tags/gpu-accelerated.md>), [model](<https://devfeed.tech/tags/model.md>), [turbo](<https://devfeed.tech/tags/turbo.md>)

### AI overview

Turbo-GMM fuses Gaussian mixture model expectation-maximization into a single component-axis sweep: it keeps the complete component axis on chip to normalize responsibilities and accumulate sufficient statistics without recomputing scores. The authors establish algebraic equivalence to standard EM in real arithmetic. Register pressure, occupancy, and component and feature dimensions limit this approach, so it is best suited to large datasets with relatively few components; larger configurations use a two-pass fallback. Across 16 configurations on an NVIDIA A100, the reported protocol shows a median 2.15x speedup over the evaluated Flash-GMM implementation, and up to 5.3x over PyTorch and 9.9x over scikit-learn.

### Source excerpt

arXiv:2611.02449v1 Announce Type: cross Abstract: Expectation-maximization (EM) for Gaussian mixture models (GMMs) requires scoring $N$ data points against $K$ mixture components. Prior fused GPU implementations, notably Flash-GMM, eliminate materialization of the $N \times K$ responsibility matrix but traverse the component dimension twice to compute normalization terms and accumulate sufficient statistics. We introduce Turbo-GMM, a single-sweep fused EM kernel that completes responsibility normalization and sufficient-statistic accumulation within one traversal of the component dimension. By retaining the complete component axis on chip, Turbo-GMM avoids the second score computation required by the released two-sweep Flash-GMM implementation. Its applicability is limited by register pressure, occupancy, and the component and feature dimensions, making it particularly suitable for large datasets with relatively small numbers of components. For larger configurations, Turbo-GMM employs a two-pass fallback following the established fused design. We establish algebraic equivalence to standard EM in real arithmetic and evaluate sixteen configurations on an NVIDIA A100. Under the reported benchmarking protocol, Turbo-GMM achieves a median $2.15\times$ speedup over the evaluated Flash-GMM implementation and up to $5.3\times$ and $9.9\times$ over PyTorch and \texttt{scikit-learn}, respectively. The results demonstrate the potential of single-sweep component traversal while identifying on-chip resource constraints as the principal limitation of this approach.

## How Many and How Much? Contrastive Learning with Negative Samples for Knowledge Graph Embedding

DevFeed: [How Many and How Much? Contrastive Learning with Negative Samples for Knowledge Graph Embedding](<https://devfeed.tech/articles/how-many-and-how-much-contrastive-learning-with-negative-samples-for-knowledge-graph-embedding-104079.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37389>)

Author: Sooho Moon, Yunyong Ko

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [contrastive-learning](<https://devfeed.tech/tags/contrastive-learning.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [false-negative](<https://devfeed.tech/tags/false-negative.md>), [knowledge-graph](<https://devfeed.tech/tags/knowledge-graph.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>)

### AI overview

RADAR adapts both the number and influence of negative samples used to train knowledge graph embeddings. It estimates false-negative risk by treating an open-world graph as a stochastic partial observation of a closed-world graph, then allocates each positive example a risk-aware negative budget. It also calibrates negative-sample weights using dispersion and concentration in their score distribution. Across three real-world knowledge graphs and four embedding models, the abstract reports the best average performance across model-graph combinations and over 90% fewer sampled false negatives than conventional fixed-budget methods.

### Source excerpt

arXiv:2610.37389v1 Announce Type: new Abstract: Knowledge graph embedding (KGE), a fundamental technique for knowledge-centric Web and KG-augmented AI applications, is commonly trained through contrastive learning with negative samples to learn discriminative representations. Despite its empirical success, two crucial questions regarding negative samples remain underexplored: (Q1) how many negative samples should be used for each positive sample and (Q2) how much should each negative sample contribute to learning? To answer these questions, we propose RADAR, a novel contrastive learning framework for KGE models with two effective strategies, respectively: (1) Risk-aware negative budget allocation quantifies false-negative sampling risk by modeling an open-world KG as a stochastic partial observation of its closed-world counterpart and adaptively assigns a negative budget to each positive sample based on its risk; (2) Distribution-calibrated negative weighting assigns negative weights by calibrating the weighting strength for each positive based on two complementary characteristics of the negative-score distribution: dispersion and concentration. Extensive experiments on 3 real-world KGs with 4 KGE models reveal that RADAR achieves the best KGE performance on average across all model--KG combinations and reduces sampled false negatives by over 90% compared with conventional fixed-budget methods.

## Leading European Financial Services Organizations Embrace Open Models to Build Proprietary Intelligence

DevFeed: [Leading European Financial Services Organizations Embrace Open Models to Build Proprietary Intelligence](<https://devfeed.tech/articles/leading-european-financial-services-organizations-embrace-open-models-to-build-proprietary-intelligence-95045.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/european-financial-services-organizations-open-models-proprietary-intelligence/>)

Author: Jason Renders

Published: 2026-10-22T05:00:56Z

Content type: news

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Payment Fraud](<https://devfeed.tech/topics/payment-fraud.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [digital sovereignty](<https://devfeed.tech/topics/digital-sovereignty.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [Dynamo-Triton](<https://devfeed.tech/topics/dynamo-triton.md>)

Tags: [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [banking](<https://devfeed.tech/tags/banking.md>), [banks](<https://devfeed.tech/tags/banks.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [blackwell-ultra](<https://devfeed.tech/tags/blackwell-ultra.md>), [civo](<https://devfeed.tech/tags/civo.md>), [cudf](<https://devfeed.tech/tags/cudf.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [detection](<https://devfeed.tech/tags/detection.md>), [dynamo-triton](<https://devfeed.tech/tags/dynamo-triton.md>), [financial](<https://devfeed.tech/tags/financial.md>), [financial-services](<https://devfeed.tech/tags/financial-services.md>), [gtc-2026](<https://devfeed.tech/tags/gtc-2026.md>), [nvidia-blackwell](<https://devfeed.tech/tags/nvidia-blackwell.md>), [open](<https://devfeed.tech/tags/open.md>), [open-weight-models](<https://devfeed.tech/tags/open-weight-models.md>), [revolut](<https://devfeed.tech/tags/revolut.md>)

### AI overview

The excerpt presents European financial organizations using open or specialized models and accelerated computing on their own or sovereign infrastructure to build financial intelligence. Revolut is scaling its PRAGMA foundation model with NVIDIA cuDF and Dynamo-Triton on Blackwell Ultra GPUs for an initial 80 million customers, with engineering for 3x scale. Starling trained Starling Intelligence on its transaction data in U.K. sovereign cloud; Lloyds built ATUM from banking transactions using RTX PRO 6000 Blackwell GPUs; SBS offers a platform for models that keep data within banks. Other examples include bunq's Finn, reported to match leading closed-model quality at 60% of the cost per answer on Nebius, and Santander's use of NVIDIA Kumo models for structured financial data. For research, LSEG reports up to 10x faster historical tick-data analysis with Polars GPU and cuDF. For fraud pipelines, HSBC cites 5x faster data processing, while Stripe reports 42x faster pipelines and 92% cost savings in offline testing with GPU-accelerated Ray Data.

### Source excerpt

At GTC Berlin, banks, payment companies and financial data providers are showing what production-scale intelligence looks like -- built on NVIDIA accelerated computing, powered by open weight models and running on their own infrastructure. Revolut is rebuilding its technology stack on a single foundation model PRAGMA, powered by NVIDIA accelerated computing. Using the NVIDIA cuDF [...]

## NVIDIA Agent Toolkit Expands, Grounding Enterprise Agents in Company Data

DevFeed: [NVIDIA Agent Toolkit Expands, Grounding Enterprise Agents in Company Data](<https://devfeed.tech/articles/nvidia-agent-toolkit-expands-grounding-enterprise-agents-in-company-data-92599.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/enterprise-platforms-agent-toolkit/>)

Author: Erik Pounds

Published: 2026-10-21T10:10:01Z

Content type: release

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [NemoClaw](<https://devfeed.tech/topics/nemoclaw.md>), [OpenShell](<https://devfeed.tech/topics/openshell.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [NeMo Retriever](<https://devfeed.tech/topics/nemo-retriever.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [company](<https://devfeed.tech/tags/company.md>), [control](<https://devfeed.tech/tags/control.md>), [data](<https://devfeed.tech/tags/data.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprises](<https://devfeed.tech/tags/enterprises.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [gtc-2026](<https://devfeed.tech/tags/gtc-2026.md>), [know](<https://devfeed.tech/tags/know.md>), [leading](<https://devfeed.tech/tags/leading.md>), [monitor](<https://devfeed.tech/tags/monitor.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [openshell](<https://devfeed.tech/tags/openshell.md>), [run](<https://devfeed.tech/tags/run.md>), [software](<https://devfeed.tech/tags/software.md>), [source](<https://devfeed.tech/tags/source.md>), [toolkit](<https://devfeed.tech/tags/toolkit.md>), [tools](<https://devfeed.tech/tags/tools.md>), [useful](<https://devfeed.tech/tags/useful.md>)

### AI overview

NVIDIA Agent Toolkit combines company-data tools and models with an agent runtime and blueprints to help enterprises build, control and monitor agents. Its new Agent Knowledge Foundation uses CUDA-X libraries, Auto-Ontology to map raw data to business concepts, and Nemotron Retriever and Kumo models to find information and support predictions. NVIDIA says it can work without moving data, rebuilding workflows or changing governance controls. OpenShell controls what agents can access and do; NemoClaw provides reference blueprints, including a step-by-step optimization blueprint. Reported examples include Palantir Foundry, Ontology and AIP for supply-chain agents, Databricks Genie Ontology with Kumo Anomaly, and Snowflake Notebook support for Kumo models. NVIDIA also cites faster or more relevant search and reranking results from Amazon OpenSearch Service, SAP and ServiceNow.

### Source excerpt

Agents are only as useful as what they know, and only as trustworthy as the boundaries they run within. Leading enterprises are using the open source NVIDIA Agent Toolkit -- now expanded with new tools -- to customize and control agents, use company data to inform their answers and actions, monitor what they do and [...]

## Benchmarking NVIDIA RAPIDS on GKE: Estimated vs. Actual Spark Speedups

DevFeed: [Benchmarking NVIDIA RAPIDS on GKE: Estimated vs. Actual Spark Speedups](<https://devfeed.tech/articles/benchmarking-nvidia-rapids-on-gke-estimated-vs-actual-spark-speedups-91833.md>)

Original publisher: [Read original article](<https://hackernoon.com/benchmarking-nvidia-rapids-on-gke-estimated-vs-actual-spark-speedups?source=rss>)

Author: chetan

Published: 2026-10-20T21:20:16Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [Gke](<https://devfeed.tech/topics/gke.md>), [Apache Spark](<https://devfeed.tech/topics/spark.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>)

Tags: [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [gke](<https://devfeed.tech/tags/gke.md>), [google-kubernetes-engine](<https://devfeed.tech/tags/google-kubernetes-engine.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-acceleration](<https://devfeed.tech/tags/gpu-acceleration.md>), [gpu-scheduling](<https://devfeed.tech/tags/gpu-scheduling.md>), [kubernetes-gpu-workloads](<https://devfeed.tech/tags/kubernetes-gpu-workloads.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-rapids](<https://devfeed.tech/tags/nvidia-rapids.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [rapids](<https://devfeed.tech/tags/rapids.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [spark](<https://devfeed.tech/tags/spark.md>), [spark-benchmarking](<https://devfeed.tech/tags/spark-benchmarking.md>), [spark-performance-tuning](<https://devfeed.tech/tags/spark-performance-tuning.md>)

### AI overview

The Qualification Tool estimated 4-5x speedups for several Spark jobs, yet some NVIDIA RAPIDS runs on GKE took longer end to end than CPU runs. The estimate covers supported SQL stage work; it excludes GPU node provisioning and time executors spend Pending. The author recommends qualifying jobs from event logs, then comparing submit-to-done time against a CPU run before rollout. RAPIDS should remain opt-in: supported, scan-heavy or shuffle-heavy workloads are candidates, while jobs with substantial unsupported work may stay on CPU. On GPU executors, multiple tasks can share an L4 using spark.task.resource.gpu.amount=0.001 and spark.rapids.sql.concurrentGpuTasks=3, with batch size adjusted for memory. These settings improve utilization but do not address startup delays; GPU memory failures may require smaller batches, fewer concurrent tasks, or repartitioning skewed keys.

### Source excerpt

NVIDIA RAPIDS predicted 4-5x faster Spark jobs, but some GPU runs were slower. Here's what GKE scheduling overhead revealed.

## How to Rewrite Apache Spark UDFs for GPU Acceleration with AI Agents

DevFeed: [How to Rewrite Apache Spark UDFs for GPU Acceleration with AI Agents](<https://devfeed.tech/articles/how-to-rewrite-apache-spark-udfs-for-gpu-acceleration-with-ai-agents-86355.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/rewrite-apache-spark-udfs-gpu-acceleration-ai-agents/>)

Author: Elizabeth Goodman

Published: 2026-10-13T21:33:21Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Apache Spark](<https://devfeed.tech/topics/spark.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [Loop Engineering](<https://devfeed.tech/topics/loop-engineering.md>), [Agent Skills](<https://devfeed.tech/topics/agent-skills.md>)

Tags: [agent-skills](<https://devfeed.tech/tags/agent-skills.md>), [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [code](<https://devfeed.tech/tags/code.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cudf](<https://devfeed.tech/tags/cudf.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu-acceleration](<https://devfeed.tech/tags/gpu-acceleration.md>)

### AI overview

The article explains how coding agents can rewrite Apache Spark user-defined functions (UDFs) into GPU-accelerated implementations using NVIDIA cuDF. It covers SQL expressions and the RapidsUDF interface, with testing and benchmarking feedback loops to verify correctness and performance. A production example reports a 22x speedup on a 600-billion-row workload.

### Source excerpt

Enterprise applications have become increasingly demanding. Agentic AI has compounded the need for vast amounts of enterprise data with frequent, complex...

## Dell AI Data Platform Adds Semantic Layer, cuDF GPU Acceleration, and 500-Tenant PowerScale Clusters

DevFeed: [Dell AI Data Platform Adds Semantic Layer, cuDF GPU Acceleration, and 500-Tenant PowerScale Clusters](<https://devfeed.tech/articles/dell-ai-data-platform-adds-semantic-layer-cudf-gpu-acceleration-and-500-tenant-powerscale-clusters-65659.md>)

Original publisher: [Read original article](<https://www.storagereview.com/news/dell-ai-data-platform-semantic-layer-cudf-500-tenant-powerscale>)

Author: Harold Fritts

Published: 2026-10-06T15:12:05Z

Content type: news

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [semantic-layer](<https://devfeed.tech/topics/semantic-layer.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-data](<https://devfeed.tech/tags/ai-data.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [enterprise-storage](<https://devfeed.tech/tags/enterprise-storage.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-acceleration](<https://devfeed.tech/tags/gpu-acceleration.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [nemotron-retriever](<https://devfeed.tech/tags/nemotron-retriever.md>), [semantic-layer](<https://devfeed.tech/tags/semantic-layer.md>)

### AI overview

Dell expanded its AI Data Platform with semantic and knowledge graph context layers, GPU accelerated data processing, and PowerScale multitenancy. The announcement describes NVIDIA Nemotron Retriever and cuDF integration, tenant isolation features, vendor tested processing gains, and a phased release schedule extending into 2027.

### Source excerpt

Dell Technologies has expanded the Dell AI Data Platform with context orchestration layers, GPU-accelerated processing engines, and high-density multitenancy across its unstructured storage infrastructure. The architectural enhancements target the enterprise retrieval path, connecting raw files, databases, and multi-site repositories directly to AI agent pipelines. The system integrates NVIDIA Nemotron Retriever models for document parsing, embedding, The post Dell AI Data Platform Adds Semantic Layer, cuDF GPU Acceleration, and 500-Tenant PowerScale Clusters appeared first on StorageReview.com.

## Run Massive-Scale UMAP in Minutes Using Multiple GPUs--Without Losing Accuracy

DevFeed: [Run Massive-Scale UMAP in Minutes Using Multiple GPUs--Without Losing Accuracy](<https://devfeed.tech/articles/run-massive-scale-umap-in-minutes-using-multiple-gpus-without-losing-accuracy-6933.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/run-massive-scale-umap-in-minutes-using-multiple-gpus-without-losing-accuracy/>)

Author: Tanya Lenz

Published: 2026-08-18T16:48:08Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Data analysis](<https://devfeed.tech/topics/data-analysis.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Graphs](<https://devfeed.tech/topics/graphs.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [cuda-x](<https://devfeed.tech/tags/cuda-x.md>), [data-analytics-processing](<https://devfeed.tech/tags/data-analytics-processing.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [feature](<https://devfeed.tech/tags/feature.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [graphs](<https://devfeed.tech/tags/graphs.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [post](<https://devfeed.tech/tags/post.md>), [scale](<https://devfeed.tech/tags/scale.md>), [simulation-modeling-design](<https://devfeed.tech/tags/simulation-modeling-design.md>), [training](<https://devfeed.tech/tags/training.md>), [vector](<https://devfeed.tech/tags/vector.md>), [visualization](<https://devfeed.tech/tags/visualization.md>)

### AI overview

This article explains how multi-GPU UMAP scales dimensionality reduction to datasets containing tens to hundreds of millions of vectors. A feature in NVIDIA cuML and cuVS 25.06 distributes all-neighbors kNN graph construction across multiple GPUs, enabling workloads of several hundred gigabytes to run in minutes while preserving nearest-neighbor relationships and accuracy.

### Source excerpt

Uniform Manifold Approximation and Projection (UMAP) is a dimensionality reduction technique widely used for visualization and feature extraction. Applications...

## Inside Snap's Experimentation Platform: Leveraging NVIDIA GPUs for Accelerated Data Processing

DevFeed: [Inside Snap's Experimentation Platform: Leveraging NVIDIA GPUs for Accelerated Data Processing](<https://devfeed.tech/articles/inside-snap-s-experimentation-platform-leveraging-nvidia-gpus-for-accelerated-data-processing-56706.md>)

Original publisher: [Read original article](<https://eng.snap.com/snap-nvidia-gcp>)

Published: 2026-03-16T07:00:00Z

Content type: article

Language: en

Sources: [Snap](<https://devfeed.tech/sources/snap.md>)

Topics: [experiments](<https://devfeed.tech/topics/experiments.md>), [Apache Spark](<https://devfeed.tech/topics/spark.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>)

Tags: [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cost-savings](<https://devfeed.tech/tags/cost-savings.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [experimentation](<https://devfeed.tech/tags/experimentation.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-accelerated](<https://devfeed.tech/tags/gpu-accelerated.md>)

### AI overview

Snap describes migrating large-scale experimentation data pipelines to GPU-accelerated Apache Spark on Google Cloud. The post covers reliability, operational complexity, shared GPU infrastructure, and reported cost savings.

### Source excerpt

Inside Snap's Experimentation Platform: Leveraging NVIDIA GPUs for Accelerated Data Processing Migrating Snapchat's AB Pipelines to GPU-Accelerated Spark At Snap, experimentation sits at the core of how we build products. Nearly every product decision is informed by controlled experiments, and the infrastructure that powers those experiments needs to be reliable, scalable, and fast.

## Up to 12x Faster Vector Indexing in Elasticsearch with NVIDIA cuVS: GPU-acceleration Chapter 2

DevFeed: [Up to 12x Faster Vector Indexing in Elasticsearch with NVIDIA cuVS: GPU-acceleration Chapter 2](<https://devfeed.tech/articles/up-to-12x-faster-vector-indexing-in-elasticsearch-with-nvidia-cuvs-gpu-acceleration-chapter-2-78784.md>)

Original publisher: [Read original article](<https://www.elastic.co/search-labs/blog/elasticsearch-gpu-accelerated-vector-indexing-nvidia>)

Author: Chris Hegarty,Hemant Malik,Corey Nolet,Manas Singh,Mithun Radhakrishnan,Mayya Sharipova,Lorenzo Dematte,Ben Frederickson

Published: 2025-12-03T00:00:00Z

Content type: article

Language: en

Sources: [Elasticsearch Labs](<https://devfeed.tech/sources/elasticsearch-labs.md>)

Topics: [elasticsearch](<https://devfeed.tech/topics/elasticsearch.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cpu-utilization](<https://devfeed.tech/tags/cpu-utilization.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data-processing](<https://devfeed.tech/tags/data-processing.md>), [database](<https://devfeed.tech/tags/database.md>), [elasticsearch](<https://devfeed.tech/tags/elasticsearch.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-acceleration](<https://devfeed.tech/tags/gpu-acceleration.md>), [has](<https://devfeed.tech/tags/has.md>), [indexing](<https://devfeed.tech/tags/indexing.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [vector](<https://devfeed.tech/tags/vector.md>), [vector-database](<https://devfeed.tech/tags/vector-database.md>)

### AI overview

The article describes Elasticsearch's integration of NVIDIA cuVS for GPU-accelerated vector indexing. In a benchmark using 2.6 million vectors, GPU indexing increased throughput by about 12 times and accelerated force-merging by about 7 times, while maintaining similar recall and lowering CPU use. A cost-adjusted comparison against a larger CPU-only instance reported about 5 times higher indexing throughput and 6 times faster force-merging. GPU-accelerated vector indexing was planned for Tech Preview in Elasticsearch 9.3.

### Source excerpt

Discover how Elasticsearch achieves nearly 12x higher indexing throughput with GPU-accelerated vector indexing and NVIDIA cuVS.

## Make your ZeroGPU Spaces go brrr with ahead-of-time compilation

DevFeed: [Make your ZeroGPU Spaces go brrr with ahead-of-time compilation](<https://devfeed.tech/articles/make-your-zerogpu-spaces-go-brrr-with-ahead-of-time-compilation-7575.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/zerogpu-aoti>)

Author: Charles Bensimon; Sayak Paul; Linoy Tsaban; Apolinário from multimodal AI art

Published: 2025-09-02T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [fly.io](<https://devfeed.tech/topics/fly-io.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [compilation](<https://devfeed.tech/tags/compilation.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [guide](<https://devfeed.tech/tags/guide.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [image-to-image](<https://devfeed.tech/tags/image-to-image.md>), [image-to-video](<https://devfeed.tech/tags/image-to-video.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [models](<https://devfeed.tech/tags/models.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [python](<https://devfeed.tech/tags/python.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [spaces](<https://devfeed.tech/tags/spaces.md>), [text-to-image](<https://devfeed.tech/tags/text-to-image.md>), [zerogpu](<https://devfeed.tech/tags/zerogpu.md>)

### AI overview

This tutorial explains how to use PyTorch ahead-of-time compilation in ZeroGPU Spaces. It covers faster model startup and inference, FP8 quantization, dynamic shapes, and the process-based GPU allocation model used by ZeroGPU, with reported speedups of 1.3x-1.8x on Flux, Wan, and LTX models.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Leveraging Spark 3 and NVIDIA's GPUs to Reduce Cloud Cost by up to 70% for Big Data Pipelines

DevFeed: [Leveraging Spark 3 and NVIDIA's GPUs to Reduce Cloud Cost by up to 70% for Big Data Pipelines](<https://devfeed.tech/articles/leveraging-spark-3-and-nvidia-s-gpus-to-reduce-cloud-cost-by-up-to-70-for-big-data-pipelines-31935.md>)

Original publisher: [Read original article](<https://medium.com/paypal-tech/leveraging-spark-3-and-nvidias-gpus-to-reduce-cloud-cost-by-up-to-70-for-big-data-pipelines-e0bc02ec4f88?source=rss----6423323524ba---4>)

Author: Ilay Chen

Published: 2024-02-21T16:42:14Z

Content type: article

Language: en

Sources: [PayPal Technology](<https://devfeed.tech/sources/paypal-technology.md>)

Topics: [Apache Spark](<https://devfeed.tech/topics/spark.md>), [big-data](<https://devfeed.tech/topics/big-data.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [cloud cost management](<https://devfeed.tech/topics/cloud-cost-management.md>), [RAPIDS](<https://devfeed.tech/topics/rapids.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>)

Tags: [3](<https://devfeed.tech/tags/3.md>), [ai](<https://devfeed.tech/tags/ai.md>), [apache-spark](<https://devfeed.tech/tags/apache-spark.md>), [big-data](<https://devfeed.tech/tags/big-data.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cost](<https://devfeed.tech/tags/cost.md>), [data](<https://devfeed.tech/tags/data.md>), [data-processing](<https://devfeed.tech/tags/data-processing.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>), [paypal](<https://devfeed.tech/tags/paypal.md>)

### AI overview

PayPal reports that tuning a large Spark workload before GPU migration--using larger input partitions and Spark 3 adaptive query execution--cut runtime by about 20% and cluster resources by about 30%, for an estimated 45% cost reduction. After migrating the job to NVIDIA Spark RAPIDS, the authors report potential savings up to 70% for eligible workloads. They identify large-data SQL operations such as joins, grouping and sorts as candidates, and cite NVIDIA's Qualification Tool for assessing jobs. GPU tuning required balancing concurrent tasks against memory limits; their Tesla T4 setup settled on two GPU tasks at a time. GPU memory spills also pressured local disks, so doubling SSDs and increasing host spill storage improved stability and runtime. Results depend on workload and configuration; the article notes Spark RAPIDS is not suitable for every job.

### Source excerpt

By Ilay Chen and Tomer Akirav At PayPal, hundreds of thousands of Apache Spark jobs run on an hourly basis, processing petabytes of data and requiring a high volume of resources. To handle the growth of machine learning solutions, PayPal requires scalable environments, cost awareness and constant innovation. This blog explains how Apache Spark 3 and GPUs can help enterprises potentially reduce Apache Spark's jobs cloud costs by up to 70% for big data processing and AI applications. Our journey will begin with a brief introduction of Spark RAPIDS -- Apache Spark's accelerator that leverages GPUs to accelerate processing via the RAPIDS libraries. We will then review PayPal's CPU-based Spark 2 application, our upgrade to Spark 3 and its new capabilities, explore the migration of our Apache Spark application to a GPU cluster, and how we tuned Spark RAPIDS parameters. We will then discuss some challenges we encountered and the benefits of the updates. Libra scales in the cloud, generated by AIBackground GPUs are everywhere, and their parallelism characteristics are perfect for processing AI and graphics applications, among other things. For those unfamiliar: what makes GPUs different from CPUs, computation-wise, is that CPUs have a limited amount of very strong cores, whereas GPUs have thousands, or even tens of thousands or more, relatively weak cores that work together very well. PayPal has been leveraging GPUs to train models for some time now, and so we decided to evaluate if the parallelism of the GPU can be helpful with processing big data applications based on Apache Spark. In our research, we encountered NVIDIA's Spark RAPIDS open-source project. It has many purposes, however we focused on Spark RAPIDS's cost reduction potential, because enterprises like PayPal spend lots of money on running Spark jobs in the cloud. Using Spark with GPUs isn't common in the industry yet, but according to our findings as described in this blog, the potential benefits could be enorm