Orkun Soylu

2 min read

W7800: bare metal vs Proxmox VM

Moving my W7800 from a bare-metal box into a Proxmox VM on another host changed generation by 2% (noise) and made prompt processing 8% faster. Passthrough itself costs nothing measurable — but I had to get past four setup traps first.

Tested
Setup

Radeon Pro W7800 48 GB — bare-metal workstationBare metal, Linux, ROCm (Ollama) · AMD Ryzen 5 7600 (6C/12T, AVX-512) · 64 GB DDR5-6400

Radeon Pro W7800 48 GB — Proxmox VM (GPU passthrough)Proxmox VE guest with PCIe passthrough, Ubuntu 26.04 · Intel Core i5-13400 host (6P+4E); guest 8 vCPU, 16 vCPU for the Strata test · Guest 40 GB (52 GB for the Strata test); host 64 GB DDR5-4800

Runtime
Ollama 0.34 (ROCm), Qwen3.8-27B Q8_0 + MTP, same benchmark script
Models

qwen3.8:27b-q8_0-mtpQ8_0 + MTPtested

Verdict

In throughput terms, passthrough is free. The costs are operational: a large-BAR MMIO window, firmware in the guest, enough guest RAM for the ROCm SVM mapping, and never touching the card from the host.

Works

  • Generation 43.0 → 42.1 tok/s (−2%, within noise)
  • 20K prompt processing 555 → 600 tok/s (+8%)
  • Model load 60 s → 20 s

Doesn’t

  • Guest hangs at boot until the OVMF 64-bit MMIO window is enlarged (64 GB BAR)
  • Model will not load with 24 GB guest RAM (SVM mapping limit) — needs ~40 GB
  • Binding amdgpu on the host sent the card to D3cold; only a host reboot recovered it
Generation, bare / VM
43.0 / 42.1tok/s
20K prefill, bare / VM
~555 / 600tok/s
Idle power, model loaded
13–32W
Power while generating
241W

Generation speed by prompt

higher is better

Same script, same four prompts, temperature 1

  • Bare metal (Ryzen 5 7600, 60 GB)
  • Proxmox VM (i5-13400 host, 40 GB guest)
Python code
53.1 tok/sBare metal (Ryzen 5 7600, 60 GB) · 53.1 tok/s
49.4 tok/sProxmox VM (i5-13400 host, 40 GB guest) · 49.4 tok/s
Bash script
47.0 tok/sBare metal (Ryzen 5 7600, 60 GB) · 47.0 tok/s
48.6 tok/sProxmox VM (i5-13400 host, 40 GB guest) · 48.6 tok/s
TCP explanation (English)
35.5 tok/sBare metal (Ryzen 5 7600, 60 GB) · 35.5 tok/s
36.2 tok/sProxmox VM (i5-13400 host, 40 GB guest) · 36.2 tok/s
Turkish prose
36.5 tok/sBare metal (Ryzen 5 7600, 60 GB) · 36.5 tok/s
34.3 tok/sProxmox VM (i5-13400 host, 40 GB guest) · 34.3 tok/s
Average
43.0 tok/sBare metal (Ryzen 5 7600, 60 GB) · 43.0 tok/s
42.1 tok/sProxmox VM (i5-13400 host, 40 GB guest) · 42.1 tok/s
Data
Bare metal (Ryzen 5 7600, 60 GB)Proxmox VM (i5-13400 host, 40 GB guest)
Python code53.1 tok/s49.4 tok/s
Bash script47.0 tok/s48.6 tok/s
TCP explanation (English)35.5 tok/s36.2 tok/s
Turkish prose36.5 tok/s34.3 tok/s
Average43.0 tok/s42.1 tok/s

Context

I moved the W7800 from a bare-metal workstation into a Proxmox VM on another host, swapping places with the RTX 4070 Ti SUPER. Same card, same model files (copied byte for byte, hashes checked), same benchmark script. The question was simple: does the VM cost me anything?

Performance

Bare metal (60 GB RAM) Proxmox VM (40 GB guest)
Generation, single stream (4-prompt average) 43.0 tok/s 42.1 tok/s
— Python code 53.1 49.4
— Bash script 47.0 48.6
— TCP explanation (English) 35.5 36.2
— Turkish prose 36.5 34.3
20K prompt processing (cold) 554–557 tok/s 600 tok/s
Model load ~60 s ~20 s
VRAM at 262K 39 GiB 39 GiB (30 GB model + KV)

No measurable cost. Generation differs by 2% (that’s noise), and prompt processing is actually 8% faster in the VM. A long generation came in at 29.5 tok/s — right inside the old host’s Turkish range (28–35).

llama-bench (llama.cpp b11081) couldn’t use the GPU inside this VM: it quietly fell back to the CPU and reported 2.4 tok/s with the card sitting at 0%. Ollama’s own runner was fine. Don’t trust that tool here for low-level comparisons.

Fan and power (read from inside the guest)

State Power Fan Temperature Clocks
VM on, no model 13 W ~1,008 rpm 48 °C sclk 0 / mclk 96 MHz
Model in memory, idle 13–32 W 1,000–1,400 rpm 48–56 °C sclk 0 / mclk 96 MHz
Generating 241 W ~1,030–1,200 rpm 51 °C sclk 2,137 / mclk 1,124 MHz

The blower never stops, but it stays at ~1,000–1,200 rpm even under load, because in this case the card sits around 50 °C. With the VM off, the card belongs to vfio-pci and no driver can read its sensors.

Four setup traps (I hit every one)

  1. The host’s network died. Swapping cards shifted the PCI bus numbers and renamed both NICs, so the bridges lost their ports. I fixed it from the console.
  2. The guest hung before the display came up. The card’s BAR is 64 GB, which doesn’t fit OVMF’s default 64-bit MMIO window. Fix: -fw_cfg name=opt/ovmf/X-PciMmio64Mb,string=131072.
  3. amdgpu crashed in the guest (early_init of IP block <psp> failed -19): the cloud image has no AMD GPU firmware. Fix: install linux-firmware-amd-graphics.
  4. The model wouldn’t load: amdgpu: SVM mapping failed, exceeds resident system memory limit. 24 GB of guest RAM was too little (bare metal had 60 GB); at 40 GB it loads in 20 s.

The lesson that cost me the most: with the VM off, I bound amdgpu on the host just to read fan and power. Runtime power management dropped the card into D3cold, vfio couldn’t wake it (Unable to power on device, stuck in D3), a PCI rescan didn’t help, and only a host reboot brought it back. Never bind a host driver to a card you pass through.