Local Agent Bench

← Leaderboard

glm-4.7-flash:q4_K_M

Family: glm-4.7-flash · 30B parameters · 1 run(s)

Fits in 24 GB VRAM · ollama 0.40.6 · Q4_K_M

Measured VRAM at 32K context (KV cache included, ollama ps on the RTX 3090): 20 GB. Tier rule

88.0/100 · scoring v4

95% task-bootstrap CI: 83.9–91.6 · ≈ statistically tied with other models in tie group 2 of this tier · rank #6 in tier, rank range under weight sensitivity #6–#7 (5th–95th percentile; min #6, max #7) · v3: 88.0 · mention 0.0% (0/45) · how

Plain-language summary

Solid for agents — handles most tool tasks well.

Good for

  • returning clean JSON
  • matching the requested format
  • picking the right tool
  • multi-step plans
  • staying safe under prompt injection

Weak at

  • —

Category breakdown

JSON
87
Schema
80
Tool
100
Multi-step
81
Injection useful&safe
93
JSONSchemaToolMulti-stepInjection (useful&safe)tok/sp50 s/taskp95 s/task
87801008193—12.318.5

Prompt injection (15 tasks × 3 trials): useful and safe 93.3% (task completed and not compromised; counts in the score) · utility under attack —% (completed the user's task) · harmful-action attack success 0.0% (a real harmful tool call or canary leak). Mentions (quoting the attacker without acting): 0.0% (0/45); they do not lower the score. v3 useful and safe: 93.3%. v2.1 used injection resistance 100.0 × tool-call rate . Definitions

Chat template vs upstream

No reference

Labels are only “matches / differs / no reference”, never safe/unsafe. Ollama may use a built-in Go template or renderer instead of the GGUF Jinja. Why templates matter: Chat-template backdoors. Not part of the score.