# Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning

DevFeed: [Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning](<https://devfeed.tech/articles/dual-node-nvidia-dgx-spark-over-tailscale-a-remote-access-testbed-for-distributed-llm-training-and-cyber-threat-intelligence-fine-tuning-116734.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.07226>)

Author: Vasanth Iyer

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [DGX Spark](<https://devfeed.tech/topics/dgx-spark.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [tailscale](<https://devfeed.tech/topics/tailscale.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Threat Hunting & Intel](<https://devfeed.tech/topics/threat-hunting-intel.md>)

Tags: [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cyber](<https://devfeed.tech/tags/cyber.md>), [cyber-threat-intelligence](<https://devfeed.tech/tags/cyber-threat-intelligence.md>), [dgx](<https://devfeed.tech/tags/dgx.md>), [dgx-spark](<https://devfeed.tech/tags/dgx-spark.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [fine](<https://devfeed.tech/tags/fine.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [intelligence](<https://devfeed.tech/tags/intelligence.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-training](<https://devfeed.tech/tags/llm-training.md>), [node](<https://devfeed.tech/tags/node.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-dgx](<https://devfeed.tech/tags/nvidia-dgx.md>), [over](<https://devfeed.tech/tags/over.md>), [remote](<https://devfeed.tech/tags/remote.md>), [remote-access](<https://devfeed.tech/tags/remote-access.md>), [spark](<https://devfeed.tech/tags/spark.md>), [tailscale](<https://devfeed.tech/tags/tailscale.md>), [testbed](<https://devfeed.tech/tags/testbed.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

Two NVIDIA DGX Spark systems completed distributed NanoChat pretraining over a dedicated 200 Gb/s fiber link, with remote administration through Tailscale. Using PyTorch torchrun, DDP, and NCCL, the 896.5-million-parameter run processed about 5.22 billion tokens in 4,980 steps over roughly four days, averaging 63.4 seconds per step. The report documents setup and troubleshooting, including an initial-evaluation bug that caused NCCL timeouts. In fine-tuning experiments, adding data from 77 CISA advisories improved CTI-specific scores but weakened general-knowledge performance, with a small overall score increase. A second stage using a MITRE ATT&CK technique dictionary achieved 98.5% accuracy on dictionary lookups but named none of 50 held-out technique IDs correctly; related held-out threat knowledge remained near zero, and general ability declined. The authors frame the deployment as evidence of feasibility, not scaling efficiency.

## Source excerpt

arXiv:2608.07226v2 Announce Type: replace-cross Abstract: Practical evidence for multi-node training on desktop-class accelerators remains limited. This report presents a proof-of-concept deployment of distributed NanoChat pretraining across two NVIDIA DGX Spark systems, each with a GB10 Grace Blackwell system-on-chip and 128 GB of unified memory, administered remotely over a Tailscale mesh VPN and connected for training by a dedicated 200 Gb/s QSFP56 direct fiber link. PyTorch torchrun, DDP, and NCCL were configured with one process per node, a depth-20 NanoChat model, a device batch of 32 sequences per node, a 2,048-token context, and eight gradient-accumulation steps, giving 1,048,576 tokens per optimizer step for the 896.5-million-parameter model. The run took 4,980 steps over about four days, averaging 63.4 s of measured training time per step (about 16,540 tokens/s) and processing about 5.22 billion tokens. We document link configuration, container setup, interface binding, a step-zero evaluation bug that triggered NCCL timeouts, checkpointing, and troubleshooting lessons. We also built a cybersecurity fine-tuning dataset from 77 CISA advisories (338 training, 37 validation conversations) and compared a baseline SFT checkpoint against a CTI-augmented checkpoint on 17 held-out prompts with an Ollama-hosted LLM judge: CTI-specific categories improved while general-knowledge categories regressed, for a small overall change from 2.06 to 2.29 on a 0-10 scale. Version 2 adds a stage-2 fine-tune with a MITRE ATT&CK technique dictionary under a deterministic evaluation: in-dictionary ID-to-name lookup reaches 98.5%, yet none of 50 held-out IDs is named correctly, knowledge of held-out threat groups, software, campaigns, and vulnerabilities stays near zero, and general ability drops measurably. The study establishes feasibility rather than a scaling-efficiency claim.