This repository contains a benchmark for AI models on the Swedish university admissions test, Högskoleprovet.
| Model | Verbal | Verbal Points | Math | Math Points | Total |
|---|---|---|---|---|---|
| claude-3-5-sonnet-20241022 | 58/60 | 2.0 | 61/80 | 1.5 | 1.75 |
| gemini-2.0-flash-exp | 57/60 | 1.9 | 61/80 | 1.5 | 1.7 |
| claude-3-5-sonnet-20240620 | 58/60 | 2.0 | 59/80 | 1.4 | 1.7 |
| claude-3-5-haiku-20241022 | 58/60 | 2.0 | - | - | - |
| gpt-4o | 58/60 | 2.0 | 59/80 | 1.4 | 1.7 |
| gpt-turbo | 57/60 | 1.9 | 47/80 | 1.1 | 1.5 |
| gpt-4o-mini | 55/60 | 1.8 | 46/80 | 1.1 | 1.45 |
| claude-3-opus-20240229 | 58/60 | 2.0 | 39/80 | 0.8 | 1.4 |
| gemini-pro-vision | 56/60 | 1.9 | - | - | - |
| claude-3-sonnet-20240229 | 55/60 | 1.8 | - | - | - |
| gemini-1.5-flash-latest | 54/60 | 1.7 | - | - | - |
| claude-3-haiku-20240307 | 53/60 | 1.7 | - | - | - |
| gpt-3.5-turbo | 51/60 | 1.6 | - | - | - |
| llama3-70b | 50/60 | 1.5 | - | - | - |
| llama3-7b | 31/60 | 0.8 | - | - | - |
https://www.studera.nu/hogskoleprov/fpn/provfragor-och-facit-varen-2024/