Strata @W7800 + RX 7900 XTX

	1× W7800 (75% experts in VRAM)	2 GPUs, split 32, chunk 4K	2 GPUs, split 34, chunk 8K
7.5K-token prompt	840 tok/s	921 tok/s	949 tok/s
31K-token prompt	961 tok/s	1,116 tok/s	1,190 tok/s

	Decode
1× W7800 (VM, 75% in VRAM)	72.0 tok/s
2 GPUs, split 32, chunk 4K	74.0 tok/s
2 GPUs, split 34, chunk 8K	73.9 tok/s

Why two cards

On a single W7800, Strata could keep only 75% of Qwen3.8-Flash-Next’s 24,576 experts in VRAM. The rest lived in system RAM and got pulled over PCIe whenever prefill needed them. That bugged me. The IQ3_S experts add up to ~47 GiB, and there was an RX 7900 XTX (24 GB) sitting in my gaming PC. 48 + 24 = 72 GB, so on paper everything fits. Both cards are gfx1100, which means one engine build and one set of ROCm wheels covers both.

The box: bare metal this time (no hypervisor), Ryzen 5 7600, 64 GB DDR5-6400, both cards on PCIe 4.0 x8 straight from the CPU, display on the iGPU.

How Strata splits a model

Strata splits by layer. Each card holds the experts and KV cache for its own layers, plus its own prompt buffers — that last bit turns out to matter. With STRATA_STAGE_TRIM=1 and a manual split point, each card loads only its own dense weights; without it, every card carries a full 3.4 GiB copy.

The last card gets extra work: the output layer, the MTP draft layer and verification, all of which run on every token. So the faster card (7900 XTX, 96 CUs) goes last. Traffic between the cards is a few hundred KB per verification window, so x8 links are more than enough.

What I expected (before measuring)

Component (262K context)	Size
Dense weights (all layers)	~3.4 GiB
Output layer (Q5_K)	0.5 GiB
MTP draft layer	~1.1 GiB
KV cache (int8)	~3.8 GiB
All experts	~46.8 GiB

My guess: split around layer 31/32, decode ~70–85 tok/s, prefill 1.3–1.8× faster.

What actually happened

layer_split: auto got it wrong. It handed the W7800 just 2 layers and left 38 GB of it empty. I set the split by hand.
Split 32, chunk 4,096: the W7800 takes layers 0–31 (16,384 expert slots), the XTX takes 32–47 (8,192 slots), and 100% of the experts are in VRAM. The W7800 sits at 38.5 / 46 GB (including a 1.1 GB Whisper server), the XTX at 21.8 / 24 GB. But I couldn’t push the prefill chunk past 4,096: the buffers are per card, ~1.5 GB per 2,048 tokens, and the XTX had no room left.
Split 34, chunk 8,192: moving two more layers onto the W7800 freed enough XTX memory for an 8K chunk. A 16K chunk (~12 GB per card) won’t fit on the XTX even if every layer moves off it.

	1× W7800 (VM)	2 GPUs, split 32 / chunk 4K	2 GPUs, split 34 / chunk 8K
Experts in VRAM	75%	100%	100%
Experts in RAM	18.2 GiB	0	0
Decode	72.0 tok/s	74.0 tok/s	73.9 tok/s
Prefill 7.5K	840 tok/s	921 tok/s	878 / 949 tok/s (two runs)
Prefill 15K	—	—	1,102 tok/s
Prefill 31K	961 tok/s	1,116 tok/s	1,190 tok/s

Heat and power under sustained load

	Edge	Junction	Power
Radeon Pro W7800	61 °C	73 °C	247 W
RX 7900 XTX	61 °C	70 °C	204 W

One empty slot between the cards was enough. At idle the links drop to 2.5 GT/s x1 and come back to PCIe 4.0 x8 under load — that’s just power management, nothing to worry about.

What I make of it

Decode landed where I expected (+3%). On one card it was already serving 95–99% from VRAM, so there wasn’t much left to gain.
Prefill gained 24%, short of my 1.3–1.8× guess. Every expert is local now, but each card needs its own prompt buffers, so the chunk shrank from 16,384 (single card) to 8,192 — and big chunks were exactly what made single-card prefill fast. The gain shows up in long prompts, which is what agent sessions and tool-heavy answers produce anyway.
The real win is everything else. System RAM and CPU are almost idle, I no longer care which GPU service starts first, and the W7800 still has ~4 GB free.

So this is now my homelab’s main model server.

Thanks

Thanks to the Strata team: a 125B MoE running entirely from VRAM across two AMD cards is only possible because of their work.

Strata @W7800 + RX 7900 XTX

Verdict

Works

Doesn’t

Prefill speed by prompt length

Decode speed