Coder390 EfficientThink vs Qwen3.8-27B
A community fine-tune that promises to cure over-long thinking tied the official Qwen3.8-27B on quality in all three of my tests, and was 4–5% slower per token. The long tail it's built to fix never showed up in my workloads.
- Tested
- Setup
Radeon Pro W7800 48 GB — Proxmox VM (GPU passthrough)Proxmox VE guest with PCIe passthrough, Ubuntu 26.04 · Intel Core i5-13400 host (6P+4E); guest 8 vCPU, 16 vCPU for the Strata test · Guest 40 GB (52 GB for the Strata test); host 64 GB DDR5-4800
- Runtime
- Ollama 0.40.2 (ROCm), Q8_0 + MTP (draft_num_predict 4)
- Models
Qwen3.8-27B Coder390 EfficientThinkQ8_0 + MTPtested
qwen3.8:27b-q8_0-mtpQ8_0 + MTPbaseline
Verdict
My default stays. Quality is equal (CVE 0.98 vs 0.97, hardbench 20/20 vs 19/20), speed is equal or slightly behind, and grounding discipline is a notch worse: two CVEs from memory and one answer in the wrong language.
Works
- Thinks 31% less on everyday prompts; answers 12% sooner
- Hardbench 20/20 (official 19/20)
- CVE eval 0.98, no invented CVEs
Doesn’t
- On tool-heavy CVE work it thinks 23% more and answers 10% later
- Two CVEs cited from memory, one answer fully in English
- MTP acceptance slightly lower → 4–5% fewer tokens per second
- CVE score, Coder390 / official
- 0.98 / 0.97
- Hardbench
- 20/20 · 19/20
- Thinking per answer (thinkbench)
- 528 / 763tokens
- Generation, thinkbench
- 40.9 / 43.1tok/s
Seconds per answer
lower is better- Coder390
- Official Q8_0 + MTP
Data
| Coder390 | Official Q8_0 + MTP | |
|---|---|---|
| Thinkbench (everyday prompts) | 45.9 s | 52.4 s |
| CVE evaluation (tools) | 66.8 s | 60.8 s |
Thinking tokens per answer, thinkbench
lower is better- Coder390
- Official Q8_0 + MTP
Data
| Coder390 | Official Q8_0 + MTP | |
|---|---|---|
| Turkish knowledge | 617 tokens | 1,226 tokens |
| Code | 534 tokens | 864 tokens |
| Average (5 prompts) | 528 tokens | 763 tokens |
The candidate
nerkyor/Qwen3.8-27B-Coder390-EfficientThink-…-MTP-DFlash2 — a community fine-tune (SFT + SimPO + RLOO) meant to break the habit of “thinking on for up to 94K tokens after finding the answer”. The model card reports GPQA 178 / MMLU 450 / LCB 90 vs the official 177 / 444 / 83, under its own protocols. The promotional “rivals frontier models” line isn’t on the card — those names are the teacher models that generated the training data. Its lineage also passes through an uncensored intermediate model.
Baseline: qwen3.8:27b-q8_0-mtp, my default on this card.
Setup
Qwen3.8-27B-Coder390-EfficientThink-Q8_0-MTP.gguf(29.07 GB, MTP head built in asblk.64) plus the Q8_0 vision projector; both SHA-256 hashes matched Hugging Face.- I copied the Modelfile from the official Q8 (
RENDERER qwen3.8,PARSER qwen3.5, same sampling) and addeddraft_num_predict 4. - Smoke test: 262K context 100% on the GPU (30 GB), vision ✓, tools ✓, thinking kept separate ✓, MTP recognised (acceptance 73–87%, an average of 3.9–4.1 out of 4).
- The two models don’t fit in VRAM together, so I ran the tests one after the other, not interleaved.
1. Thinkbench — everyday prompts
5 prompts × 3 repetitions, thinking on, temperature 1.
| Coder390 | Official Q8 | |
|---|---|---|
| Tokens per answer | 1,816 | 2,185 |
| Thinking per answer | 528 (29%) | 763 (35%) |
| Speed | 40.9 tok/s | 43.1 tok/s |
| Seconds per answer | 45.9 | 52.4 |
Per prompt: Turkish knowledge 617 vs 1,226 thinking tokens, code 534 vs 864; no difference on the logic puzzle or the second coding prompt. So far, the promise holds.
2. Hardbench — hard problems with checkable answers
10 AIME-style problems with verified integer answers × 2 repetitions, 32K-token cap.
| Coder390 | Official Q8 | |
|---|---|---|
| Correct | 20/20 | 19/20 |
| Tokens median / mean / max | 1,334 / 2,164 / 7,298 | 1,446 / 2,016 / 4,996 |
| Runs hitting the cap | 0 | 0 |
| Total time | 891 s | 791 s |
| Speed | 51.0 tok/s | 52.7 tok/s |
The set turned out to be too easy for both: no run ever went into a long tail, so the problem Coder390 is built to fix simply never came up. The official model’s one miss (a single “grid” run) doesn’t tell the two apart.
3. CVE evaluation — real work with tools
Same truth snapshot, same system prompt, MCP server 0.5.2; Coder390’s 45 runs first, then the official model’s 45.
| Coder390 | Official Q8 | |
|---|---|---|
| Overall | 0.98 | 0.97 |
| Upgrade: newest clean version | 23/24 | 23/24 |
| Fix versions correct / wrong / missing | 40 / 0 / 2 | 39 / 0 / 3 |
| KEV, prioritisation, non-existent CVE | full | full |
| Invented CVEs | 0 | 0 |
| CVEs from memory (not in any tool result) | 2 | 0 |
| Non-Turkish answers (hand-checked) | 1/45 | 0/45 |
| Tool errors | 2 | 0 |
| Thinking chars, mean / median | 1,757 / 1,005 | 1,434 / 934 |
| Speed | 43.1 tok/s | 45.5 tok/s |
| Seconds per answer, mean / median | 66.8 / 51.7 | 60.8 / 51.7 |
The whole score gap comes from one question (Express upgrade, 0.75 vs 0.62) — noise level. What I found reading the answers:
- MOVEit run 2: entirely in English — real language drift.
- Recent-KEV run 2: the grader flagged it as English, but it’s Turkish; English product names in a table threw it off.
- OpenSSL run 2: CVE-2023-0286 and CVE-2023-0464 — real CVEs, but in no tool result — written up as “visible in NVD”, which breaks the “only from tool output” rule.
- Log4j run 1: version bounds that don’t exist, like “2.15.0–2.15.9” — sloppy range writing.
- Tool errors: a tool name with the wrong prefix (fixed after the UI’s hint) and an empty package list.
- On this task Coder390 actually thought longer (+23% on average) — the opposite of thinkbench.
My verdict
- Quality: equal on all three tests; slightly behind on grounding and language discipline in the CVE work.
- Speed: MTP acceptance is a bit lower → ~4–5% fewer tokens per second. Answers arrive 12% sooner in short chats (less thinking) and 10% later in tool-heavy CVE work (more thinking).
- The headline promise — cutting off a 94K-token thinking tail — fixes something that doesn’t happen in my workloads: the official model never hit a cap in hardbench or in the CVE work either.
My default stays the same. I deleted the model; the Modelfile plus the Hugging Face files are enough to rebuild it.
Still open
- To test the model’s real claim I’d need a set that genuinely produces long tails (LCB-hard / GPQA-type, 64K+ cap).
- I didn’t test multi-step agentic coding tasks.