Local Agent Bench

← Leaderboard

nemotron-3-nano:4b

Family: nemotron-3-nano · 4B parameters · 1 run(s)

Fits in 8 GB VRAM · ollama 0.40.6 · Q4_K_M

Measured VRAM at 32K context (KV cache included, ollama ps on the RTX 3090): 3.5 GB. Tier rule

79.0/100 · scoring v4

95% task-bootstrap CI: 73.6–84.0 · rank #4 in tier, rank range under weight sensitivity #4–#4 (5th–95th percentile; min #4, max #5) · v3: 78.1 · mention 4.4% (2/45) · how

Plain-language summary

Solid for agents — handles most tool tasks well.

Good for

  • returning clean JSON
  • matching the requested format
  • picking the right tool

Weak at

  • —

Category breakdown

JSON
100
Schema
91
Tool
96
Multi-step
61
Injection useful&safe
64
JSONSchemaToolMulti-stepInjection (useful&safe)tok/sp50 s/taskp95 s/task
10091966164—12.220.2

Prompt injection (15 tasks × 3 trials): useful and safe 64.4% (task completed and not compromised; counts in the score) · utility under attack —% (completed the user's task) · harmful-action attack success 0.0% (a real harmful tool call or canary leak). Mentions (quoting the attacker without acting): 4.4% (2/45); they do not lower the score. v3 useful and safe: 60.0%. v2.1 used injection resistance 95.6 × tool-call rate . Definitions

Chat template vs upstream

No reference

Labels are only “matches / differs / no reference”, never safe/unsafe. Ollama may use a built-in Go template or renderer instead of the GGUF Jinja. Why templates matter: Chat-template backdoors. Not part of the score.