qwen3-vl:8b

Fits in 16 GB VRAM · ollama 0.40.6 · Q4_K_M

Measured VRAM at 32K context (KV cache included, ollama ps on the RTX 3090): 10 GB · previous tier: 8 GB, rank #3 (results v0.1.1). Moved to the 16 GB tier in results v0.1.2 (2026-10-26) because it needs more than 8 GB at 32K context; score unchanged. Tier rule

93.9/100 · scoring v4

95% task-bootstrap CI (main suite, 3 trials): 90.5–96.9 · pooled over 8 runs (3 main-suite trials + 5 reruns at temperature 0.7): 86.8–93.7 · ≈ statistically tied with other models in pooled tie group 2 of this tier (the leaderboard’s ≈) · rank #4 in tier, rank range under weight sensitivity #2–#4 (5th–95th percentile; min #2, max #5) · v3: 91.6 (#3) · mention 11.1% (5/45) · how

^† The published score (93.9, main-suite mean of 3 trials) lies above the pooled 8-run interval (86.8–93.7): the 5 reruns at temperature 0.7 scored lower (rerun mean 89.73). Score and rank stay those of the main suite. Why

Prompt injection (15 tasks × 3 trials): useful and safe 97.8% (task completed and not compromised; counts in the score) · utility under attack 100.0% (completed the user's task) · harmful-action attack success 2.2% (a real harmful tool call or canary leak). Mentions (quoting the attacker without acting): 11.1% (5/45); they do not lower the score. v3 useful and safe: 86.7%. v2.1 used injection resistance 86.7 × tool-call rate 1. Definitions

• Run: 20261022-qwen3-vl-8b-de2a43 · 2026-10-22T22:53:39+00:00
• GPU: NVIDIA GeForce RTX 3090 (24 GB) · Linux 6.8.0-57-generic
• Host: b7e2c79acebd · container (RunPod) · host AMD Ryzen Threadripper PRO 5995WX 64-Cores, 251.5 GB RAM · measured 2026-10-22T22:53:39+00:00 · hosts
• Harness: hermes-agent Hermes Agent v0.22.2+2363.g9cb34e9 (2026.9.24) · stable · local 9cb34e9e285d · trials 3 · sampling: Ollama default (temperature and seed not set by harness) · context window: 32768 tokens as loaded by Ollama (the unused runner field records 8192; see Methodology)
• Score per trial (v4): 92.5 / 91.29 / 91.16 (SD 0.74, range 91.16–92.5)
• GGUF sha256: 9c3e4853af40205c11b5d1000dfcf08e96454adfd872e48b934cd67d59ec3097
• Sponsor disclosure: none

Real 8 GB card: NVIDIA RTX 3070 8 GB (RunPod container) · ollama 0.40.6 · Q4_K_M

Separate series, not part of the tier ranking. 3 single-trial run(s) merged, scoring v4 from the run logs. CPU/GPU split (ollama ps): 10 GB 44%/56% CPU/GPU runs with CPU offload (slow) (main tier stays 16 GB) · peak VRAM (nvidia-smi): 6991 MiB of 8192 MiB · series table · how it was measured

91.1/100 · scoring v4

95% task-bootstrap CI: 87.0–94.9 · rank #3 in this series · ≈ tie group 1, rank range under weight sensitivity #3–#3 · how

Prompt injection (15 tasks × 3 trials): useful and safe 100.0% (task completed and not compromised; counts in the score) · utility under attack 100.0% (completed the user's task) · harmful-action attack success 0.0% (a real harmful tool call or canary leak). Mentions (quoting the attacker without acting): 20.0% (9/45); they do not lower the score. v3 useful and safe: 80.0%. Definitions

• Run: real8gb-qwen3-vl-8b · 2026-10-28T17:40:22+00:00
• Hardware: Real 8 GB card: NVIDIA RTX 3070 8 GB (RunPod container) · NVIDIA GeForce RTX 3070 · Linux 6.8.0-85-generic
• Host: 8fcb655dfb7d · container (RunPod) · pod b3o5tul56pnqfv · pod allotment 8 vCPU / 30 GB · host AMD Ryzen 7 5700X 8-Core Processor, 62.7 GB RAM · measured 2026-10-28T17:40:22+00:00 · hosts
• Harness: hermes-agent Hermes Agent v0.22.2+2363.g9cb34e9 (2026.9.24) · stable · local 9cb34e9e285d · trials 3 · sampling: Ollama default (temperature and seed not set by harness) · context window: 32768 tokens as loaded by Ollama (the unused runner field records ; see Methodology)
• Score per trial (v4): 85.63 / 87.51 / 88.29 (SD 1.37, range 85.63–88.29)
• GGUF sha256: 9c3e4853af40205c11b5d1000dfcf08e96454adfd872e48b934cd67d59ec3097
• Sponsor disclosure: none

Chat template vs upstream

Matches: the chat template in the GGUF we tested is identical to the provider's upstream template after whitespace normalisation.

• GGUF template sha256 (normalised): 9449da7ef700cd6d252423355a41a1a680e6ff94948a92fb5c8a1a7bc3e4e446
• Upstream reference sha256 (normalised): 9449da7ef700cd6d252423355a41a1a680e6ff94948a92fb5c8a1a7bc3e4e446 · source
• Ollama renders this model with its built-in Go renderer qwen3-vl-thinking, not the GGUF Jinja template, so the prompt it actually sends is not covered by this hash.

Labels are only “matches / differs / no reference”, never safe/unsafe. Ollama may use a built-in Go template or renderer instead of the GGUF Jinja. Why templates matter: Chat-template backdoors. Not part of the score.