Claude 3.5 Haiku — Benchmark Results
Anthropic Overall Score: 52/100
Performance Scores
Overall
52
Rank #35 of 45 — Top 22%
SWE-bench
45
Rank #35 of 45 — Top 22%
LiveCodeBench
55
Rank #35 of 45 — Top 22%
HumanEval
75
Rank #36 of 45 — Top 20%
BigCodeBench
38
Rank #35 of 45 — Top 22%
Strengths & Weaknesses
Strengths
- Fastest Claude model
- Cheapest option
- Good for code review
Weaknesses
- Struggles with complex tasks
- Limited reasoning depth
Compare with Similar-Priced Models
| Model | Overall Score | Input $/M |
|---|---|---|
| Claude 3.5 Haiku | 52 | $0.800 |
| Claude 3 Haiku | 45 | $0.250 |
| GPT-4o mini | 58 | $0.150 |
| GPT-3.5 Turbo | 40 | $0.500 |
| OpenAI o1-mini | 70 | $1.10 |
| OpenAI o3-mini | 80 | $1.10 |