Mistral Nemo — Benchmark Results
Mistral Overall Score: 48/100
Performance Scores
Overall
48
Rank #39 of 45 — Top 13%
SWE-bench
40
Rank #39 of 45 — Top 13%
LiveCodeBench
50
Rank #39 of 45 — Top 13%
HumanEval
70
Rank #39 of 45 — Top 13%
BigCodeBench
32
Rank #39 of 45 — Top 13%
Strengths & Weaknesses
Strengths
- Open weight
- Self-hostable
Weaknesses
- Basic coding ability
Compare with Similar-Priced Models
| Model | Overall Score | Input $/M |
|---|---|---|
| Mistral Nemo | 48 | $0.150 |
| Claude 3.5 Haiku | 52 | $0.800 |
| Claude 3 Haiku | 45 | $0.250 |
| GPT-4o mini | 58 | $0.150 |
| GPT-3.5 Turbo | 40 | $0.500 |
| OpenAI o1-mini | 70 | $1.10 |