Orkun Soylu

1 min read

llama.cpp ROCm vs Vulkan @W7800

Current llama.cpp is only 3% faster than the runtime bundled with Ollama. Vulkan (RADV) decodes 4.6% faster than ROCm but prefills 16% slower. And the card is capped at 230 W, with no way to raise it.

Tested
Setup

Radeon Pro W7800 48 GB — bare-metal workstationBare metal, Linux, ROCm (Ollama) · AMD Ryzen 5 7600 (6C/12T, AVX-512) · 64 GB DDR5-6400

Runtime
llama-bench (llama.cpp b11081), ROCm and Vulkan/RADV builds; Ollama 0.34.0
Models

qwen3.8:27bQ8_0tested

Verdict

I have no speed reason to leave Ollama, and no reason to switch to Vulkan: the small decode gain is outweighed by slower prefill. Efficiency is normal — 66% of theoretical bandwidth for decode, 45% of the WMMA peak for prefill.

Works

  • Decode 574 GB/s ≈ 66% of the 864 GB/s spec — healthy
  • Vulkan: +4.6% decode

Doesn’t

  • Vulkan: −16% prefill
  • Power-limited at 229–230 W, ~80 °C, ~2,230 MHz; no overclock/undervolt interface
pp512, ROCm / Vulkan
744 / 624tok/s
tg128, ROCm / Vulkan
20.4 / 21.3tok/s
Board power under load
229–230W
Decode bandwidth efficiency
66%

Prompt processing (pp512)

higher is better
ROCm (what Ollama uses)
744 tok/spp512 · 744 tok/s
Vulkan (RADV)
624 tok/spp512 · 624 tok/s
Data
pp512
ROCm (what Ollama uses)744 tok/s
Vulkan (RADV)624 tok/s

Token generation (tg128)

higher is better
ROCm
20.4 tok/stg128 · 20.4 tok/s
Vulkan (RADV)
21.3 tok/stg128 · 21.3 tok/s
Ollama's bundled ggml
19.8 tok/stg128 · 19.8 tok/s
Data
tg128
ROCm20.4 tok/s
Vulkan (RADV)21.3 tok/s
Ollama's bundled ggml19.8 tok/s

Results

I wanted to know whether I was leaving speed on the table by staying on Ollama. llama-bench, Qwen3.8-27B Q8_0, no speculative decoding:

Backend pp512 tg128
ROCm (what Ollama uses) 744 tok/s 20.4 tok/s
Vulkan (RADV) 624 tok/s 21.3 tok/s
Ollama’s own bundled ggml — 19.8 tok/s
  • Upstream llama.cpp gains ~3% over Ollama’s runtime — not a reason to leave Ollama.
  • Vulkan: +4.6% decode, −16% prefill. I didn’t adopt it; I installed RADV for the test and removed it afterwards.

Power and efficiency

  • Under load the card sits at its power limit: 229–230 W (the limit is 230 W and can’t be raised), ~80 °C, ~2,230 MHz. The driver doesn’t expose pp_od_clk_voltage, so undervolting and overclocking are off the table.
  • The efficiency is normal: decode moves ~574 GB/s ≈ 66% of the 864 GB/s spec; prefill reaches ~30 TFLOPS ≈ 45% of the WMMA peak. Nothing is broken — this is simply what the card does.