Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio
Efforts to push a Mac Studio to 25 Gbps networking over Thunderbolt expose several bottlenecks, from macOS’s weak SMB performance and lack of SMB Direct (RDMA) support over Ethernet to NAS CPU limits and protocol overhead. Commenters compare this approach with alternatives like Thunderbolt/USB4 host‑to‑host networking, traditional PCIe NICs in external enclosures, and 10 Gbps setups, weighing cost, heat, and reliability issues such as flaky USB‑C Realtek dongles and SFP module quirks. The conversation highlights how only a small subset of users—high‑end video editors, NAS power users, and distributed AI workloads—truly benefit from 25–100 Gbps links, and how Apple’s current hardware and software choices cater poorly to these niches.
macOS networking, RDMA, and SMB
- Several comments suggest the bottleneck is macOS’ lack of SMB Direct (SMB over RDMA), not the 25 GbE link itself.
- macOS now supports RDMA over Thunderbolt, apparently targeted at distributed inference, but Ethernet RDMA and SMBDirect support are missing.
- Some report macOS SMB as fragile: heavy loads can cause stalls/disconnects; custom
nsmb.conftuning helps but doesn’t fix poor behavior. - Longstanding distrust of macOS network filesystems is mentioned, with references to past corruption and strong backup culture.
Thunderbolt / USB4 as a network fabric
- Thunderbolt networking between Macs is described as fast and nearly automatic; Linux can use
thunderbolt_netto expose interfaces. - USB4 essentially standardizes Thunderbolt 3; networking between hosts is possible but role‑switching is optional in USB4 and required only for newer Thunderbolt generations.
- Topology is limited: no USB switches, so only point‑to‑point or small meshes.
- Some see a potential product in Thunderbolt‑native switches, but this wouldn’t solve the RDMA-on-macOS issue.
Performance, bottlenecks, and NAS
- iperf tests show 25 GbE links can reach line rate; SMB file copies are seen as a poor benchmark across OSes.
- A low‑power Arm NAS CPU is identified as a possible bottleneck, not the NVMe array.
- Questions arise whether simply running dual 10 GbE with SMB multichannel could rival 25 GbE, depending on queue depth.
NIC power, heat, and physical layer issues
- 25G NICs are typically under 25 W but still run very hot in small Thunderbolt enclosures.
- Explanations cite: copper distance (25G over tens of meters vs Thunderbolt over ~1 m), isolation transformers, older process nodes, and intense PHY requirements.
- Even fiber optics and NVMe drives show similar “small device, big heat” behavior.
Use cases for 25–100 GbE
- Discussed workloads: multi‑stream/uncompressed or RAW 4K/8K editing, multicam, rendering, large project transfers, NAS acceleration, PXE/image distribution, and multi‑node LLM/AI setups.
- Many note that 10 GbE is “enough” for most workflows; faster links are often about reducing large transfer times and “because it’s fun.”
Hardware ecosystem: enclosures, SFPs, dongles
- Thunderbolt‑to‑PCIe NIC setups via eGPU enclosures are widely used; cheaper TB4–PCIe bridges work but require external PSUs and look messy.
- High‑end branded Thunderbolt+NIC bundles are criticized as overpriced compared to DIY NIC + enclosure solutions.
- SFP/SFP+ optics and compatibility are described as a “jungle”: minor hardware revisions, firmware, and vendor quirks can cause intermittent failures; others report smooth sailing with certain brands.
- Reports on USB‑C multi‑gig dongles are mixed: Realtek 2.5G parts often perform poorly on macOS for some, while others find them stable. Marvell/Aquantia‑based 5/10G dongles are generally praised but run hot.