LALA Benchmark Results

Read the paper (PDF)How to read these results

The model table below is available without JavaScript. Enable JavaScript for interactive breakdowns, or download all subcategory and keyword results and the complete results data.

Model rankings

Rank		Model group				0 · No credit	1 · Partial credit	2 · Full credit
1	Claude Fable 5	Proprietary	73.1%	1.462	236	34	59	143
2	Gemini 3.1 Pro Preview	Proprietary	65.5%	1.309	236	46	71	119
3	GPT-5.6 Sol Thinking	Proprietary	60.8%	1.216	236	53	79	104
4	Gemma 4 31B	Open post-trained	51.9%	1.038	236	66	95	75
5	DeepSeek R1	Open post-trained	49.6%	0.992	236	69	100	67
6	Qwen3 235B A22B Thinking 2507	Open post-trained	47.2%	0.945	236	82	85	69
7	Gemma 4 26B A4B	Open post-trained	46.4%	0.928	236	83	87	66
8	Qwen3 235B A22B 2507	Open post-trained	42.8%	0.856	236	85	100	51
9	DeepSeek V3	Open post-trained	42.2%	0.843	236	87	99	50
10	Qwen3 235B A22B	Open post-trained	41.7%	0.835	236	89	97	50
11	GPT-OSS 120B	Open post-trained	40.3%	0.805	236	94	94	48
12	Ministral 3 14B Instruct 2512	Open post-trained	35.2%	0.703	236	109	88	39
13	Llama 3.1 8B Instruct	Open post-trained	31.1%	0.623	236	123	79	34
14	Ministral 3 14B Reasoning 2512	Open post-trained	30.5%	0.610	236	119	90	27
15	Llama 3.2 3B Instruct	Open post-trained	22.5%	0.449	236	149	68	19
16	DeepSeek V3 Base	Base	2.5%	0.051	236	224	12	0
16	Ministral 3 14B Base 2512	Base	2.5%	0.051	236	224	12	0

Results by category

Model scope

				0 · No credit	1 · Partial credit	2 · Full credit
Loading decomposition scores…

Results by model group

Model group	Models	Members	Score (%)	Mean (0–2)	Responses	0 · No credit	1 · Partial credit	2 · Full credit
Loading grouped scores…

Results by parameter size

Loading parameter-size coverage…

Parameter band	Models	Members	Score (%)	Mean (0–2)	Responses	0 · No credit	1 · Partial credit	2 · Full credit
Loading parameter-size scores…

How to read these results

Rankings: Model rankings displayed on this page represent human scores as a percentage of the maximum score possible.

Model score: points earned divided by the maximum possible points, expressed as a percentage. Each response earns 0, 1 or 2 points. A 50% score means an average of 1 out of 2. Missing responses receive zero.

Model groups: Base contains 2 models; Open post-trained contains 12; Proprietary contains 3. Group scores pool responses. Equal unrounded scores share the same ranking.

Subcategories and keywords: answers scored 0 or 2 contribute that score for every associated keyword. For answers scored 1, the individual keyword scores are used. Categories and subcategories pool those keyword judgments, giving each judgment equal weight. Subcategories can overlap.

Parameter bands: published total parameters include all mixture-of-experts parameters. Bands include the lower bound and exclude the upper bound; empty bands are omitted. The 3 proprietary models have undisclosed sizes and are excluded.

Paper (PDF)Complete results (JSON)Model scores (CSV)