ornith-1.5:9b
Family: ornith-1.5 · 9B parameters · 1 run(s)
Fits in 8 GB VRAM · ollama 0.40.6 · Q4_K_M
Measured VRAM at 32K context (KV cache included, ollama ps on the RTX 3090): 6.7 GB · previous tier: 16 GB (results v0.1.1). Moved to the 8 GB tier in results v0.1.2 (2026-10-26) because it needs more than 16 GB at 32K context; score unchanged. Tier rule
92.0/100 · scoring v4
95% task-bootstrap CI: 87.7–95.9 · ≈ statistically tied with other models in tie group 1 of this tier · rank #3 in tier, rank range under weight sensitivity #3–#3 (5th–95th percentile; min #3, max #3) · v3: 82.2 · mention 48.9% (22/45) · how
Plain-language summary
Excellent at tool-using assistant work on this suite.
Good for
- returning clean JSON
- matching the requested format
- picking the right tool
- multi-step plans
- staying safe under prompt injection
Weak at
- —
Category breakdown
| JSON | Schema | Tool | Multi-step | Injection (useful&safe) | tok/s | p50 s/task | p95 s/task |
|---|---|---|---|---|---|---|---|
| 95 | 95 | 97 | 82 | 98 | — | 12.9 | 21.5 |
Prompt injection (15 tasks × 3 trials): useful and safe 97.8% (task completed and not compromised; counts in the score) · utility under attack —% (completed the user's task) · harmful-action attack success 0.0% (a real harmful tool call or canary leak). Mentions (quoting the attacker without acting): 48.9% (22/45); they do not lower the score. v3 useful and safe: 48.9%. v2.1 used injection resistance 51.1 × tool-call rate . Definitions
- Run:
20261112-ornith-1.5-9b-898500· 2026-11-12T08:35:57+00:00 - GPU: NVIDIA GeForce RTX 3090 (24 GB) · Linux 6.8.0-57-generic
- Host:
· kind not recorded · measured · hosts - Harness: hermes-agent Hermes Agent v0.22.2+2363.g9cb34e9 (2026.9.24) · stable · local 9cb34e9e285d · trials 3 · sampling: Ollama default (temperature and seed not set by harness) · context window: 32768 tokens as loaded by Ollama (the unused runner field records 8192; see Methodology)
- GGUF sha256:
852922174ee4f76621df26105333f1dfe2171cdfb60ebe5a4b013836681a8a77 - Sponsor disclosure: none
Chat template vs upstream
No reference
Labels are only “matches / differs / no reference”, never safe/unsafe. Ollama may use a built-in Go template or renderer instead of the GGUF Jinja. Why templates matter: Chat-template backdoors. Not part of the score.