HauhauCS Aggressive Q8 vs 3-bit @CVE triage
Running the same fine-tune at Q8 on my 48 GB W7800 lifts its CVE score from 0.92 to 0.97 — most of its weakness was the 3-bit compression. But it still doesn't beat ISTA's 3-bit, and it costs 2.4× the time and 2.5× the energy per token.
- Tested
- Setup
Radeon Pro W7800 48 GB — bare-metal workstationBare metal, Linux, ROCm (Ollama) · AMD Ryzen 5 7600 (6C/12T, AVX-512) · 64 GB DDR5-6400
GeForce RTX 4070 Ti SUPER 16 GB — Proxmox VM (GPU passthrough)Proxmox VE guest with PCIe passthrough, Ubuntu 24.04, NVIDIA 595 open driver · Intel Core i5-13400 host; guest 8 vCPU · Guest 32 GB; host 64 GB DDR5
- Runtime
- Ollama 0.34.0 (W7800, 262K) / 0.34.1 (4070 Ti SUPER, 65K); CVE MCP server 0.5.2
- Models
Qwen3.8-27B AggressiveQ8_K_Ptested
Qwen3.8-27B AggressiveIQ3_XSreference
Qwen3.8-27B GSQ-RCOISTA IQ3_Sbaseline
Verdict
Compression was most of HauhauCS's problem, but even at Q8 it only gets close to ISTA IQ3_S (0.97 vs 0.98), while taking 145 s per answer instead of 60. For this work I'm sticking with ISTA on the 16 GB card.
Works
- Q8: 0.92 → 0.97, research depth up (4.2 tool calls/answer), zero memory CVEs
- Spring4Shell and Jackson upgrades 3/3 correct
Doesn’t
- Language drift worse than at 3-bit: 3/45 answers entirely in English
- 18 tok/s decode, 145 s per answer, ~20 J/token
- Called a non-existent tool once
- Overall, Q8 vs IQ3_XS
- 0.97 / 0.92
- Seconds per answer
- 145s
- Decode
- 18.0tok/s
- Energy per token
- ~20.3J
CVE score
higher is betterData
| Overall score | |
|---|---|
| ISTA IQ3_S · 4070 Ti SUPER | 0.98 |
| HauhauCS IQ3_XS · 4070 Ti SUPER | 0.92 |
| HauhauCS Q8_K_P · W7800 | 0.97 |
Seconds per answer
lower is betterData
| Time | |
|---|---|
| ISTA IQ3_S · 4070 Ti SUPER | 60 s |
| HauhauCS IQ3_XS · 4070 Ti SUPER | 64 s |
| HauhauCS Q8_K_P · W7800 | 145 s |
The question
HauhauCS IQ3_XS trailed ISTA IQ3_S in prompt v2. Is that the model, or the 3-bit compression? To find out, I ran the same family’s Q8_K_P (31.46 GB, SHA-256 checked) on the W7800 (48 GB, 262K context, 100% GPU) with the same prompt, sampling, MCP server 0.5.2 and question set. I regenerated the truth right before the run. 15 × 3, zero errors.
Results
| ISTA IQ3_S (4070, v2) | HauhauCS IQ3_XS (4070, v2) | HauhauCS Q8_K_P (W7800) | |
|---|---|---|---|
| Overall | 0.98 | 0.92 | 0.97 |
| Upgrade: newest clean / vulnerable | 23 / 0 | 22 / 2 | 22 / 1 |
| Version tool used | 24/24 | 22/24 | 23/24 |
| Fix versions correct / wrong / missing | 40/0/2 | 38/1/3 | 40/0/2 |
| Memory / invented CVEs | 0 / 0 | 1 / 0 | 0 / 0 |
| Non-Turkish answers (real) | 0 | 1 | 3 |
| Summary block | 43/45 | 45/45 | 44/45 |
| Tool calls / seconds per answer | 3.2 / 60 s | 3.6 / 64 s | 4.2 / 145 s |
| Decode | ~43.5 tok/s | ~43.5 tok/s | 18.0 tok/s |
| Wall power while generating | 360–370 W | 360–370 W | 366 W |
| Energy per token | ~8.3 J | ~8.3 J | ~20.3 J |
(The language grader flagged one table-heavy Turkish answer as English; I corrected it by hand.)
What I make of it
- The compression gap is real. At Q8 the main weakness goes away: it researches more deeply (4.2 tool calls per answer), gets Spring4Shell and Jackson right 3/3, and cites no CVEs from memory. The score lands just below ISTA’s.
- What’s left: Django run 3 targeted 3.2.25 (the last release of an end-of-life line, still vulnerable). The
language drift is actually worse than at 3-bit — three answers entirely in English — so that comes from the model,
not the compression. And one answer called a tool that doesn’t exist,
get_epSS. - The cost: 2.4× the time and ~2.5× the energy per token for the same job, and it fills a 48 GB card all by itself.
Bottom line: ISTA IQ3_S on 16 GB is still the best choice for this task — equal or better accuracy, no language drift, 2.4× faster and 2.5× less energy per token.