Performance Scores

Overall

62
Rank #26 of 45 — Top 42%

SWE-bench

56
Rank #26 of 45 — Top 42%

LiveCodeBench

64
Rank #26 of 45 — Top 42%

HumanEval

82
Rank #27 of 45 — Top 40%

BigCodeBench

46
Rank #26 of 45 — Top 42%

Strengths & Weaknesses

Strengths

  • Large context
  • Established model

Weaknesses

  • Older generation

Compare with Similar-Priced Models

ModelOverall ScoreInput $/M
Gemini 1.5 Pro 62