Leaderboard | Coding

Models

Choose model family

Claude by Anthropic

Mistral by Mistral AI

More filters

Show inactive models

Hide models that are no longer actively available on Yupp.

Turns

Filter model performance by the number of turns in a conversation.

All Single turn Multiple turns

Open license models

Filter the leaderboard to only show models that have an open license.

All selected Open license Proprietary license

935

DeepSeek Prover v2

931

Codestral

928

NVIDIA Llama 3.1 Nemotron 70B

925

Mistral Small 3.1

918

Open Mistral Nemo

910

Arcee AI Spotlight

910

OLMo 3 32B Think

896

Llama 3.1 405B Instruct Turbo

896

Arcee AI Virtuoso-Medium

892

GLM 4.6V Flash

889

Arcee AI Blitz

885

Llama 3.2 11B Instruct

880

Mistral Small 3 24B Instruct

875

Mistral Nemo

870

Qwen 2.5 7B Turbo

Last updated about 1 month ago

Rank	Overall	Name	VIBE Score	Confidence Interval	Votes	Downvote %	Abort %	Speed	Latency	Context	Cost (Input)	Cost (Output)
41	179	DeepSeek Prover v2	935	±10	1.3K	3.0%	5.2%	14 tps	1.3s	164K	$0.40	$1.56
42	189	Codestral	931	±19	855	6.0%	5.2%	151 tps	0.9s	262K	$0.15	$0.45
43	245	NVIDIA Llama 3.1 Nemotron 70B	928	±7	5.3K	2.0%	<0.1%	9 tps	0.1s	128K	$0.33	$0.39
44	189	Mistral Small 3.1	925	±14	2.4K	4.0%	7.4%	13 tps	2.6s	32K	$0.17	$0.28
45	189	Open Mistral Nemo	918	±21	1.7K	4.5%	1.5%	171 tps	0.5s	131K	$0.15	$0.15
46	264	Arcee AI Spotlight	910	±8	4.6K	4.3%	<0.1%	121 tps	0.4s	131K	$0.18	$0.18
47	264	OLMo 3 32B Think	910	±25	470	6.0%	<0.1%	84 tps	0.6s	66K	$0.15	$0.50
48	264	Llama 3.1 405B Instruct Turbo	896	±11	2K	3.9%	<0.1%	26 tps	0.8s	131K	$3.50	$3.50
49	264	Arcee AI Virtuoso-Medium	896	±12	2K	2.6%	<0.1%	3 tps	N/A	131K	$0.50	$0.80
50	201	GLM 4.6V Flash	892	±10	2.5K	7.6%	3.7%	64 tps	2.1s	128K	$0.04	$0.40
51	280	Arcee AI Blitz	889	±8	3K	2.1%	<0.1%	6 tps	N/A	33K	$0.45	$0.75
52	201	Llama 3.2 11B Instruct	885	±15	2.1K	4.1%	1.5%	152 tps	0.5s	8K	$0.16	$0.16
53	210	Mistral Small 3 24B Instruct	880	±10	1.7K	3.6%	2.6%	77 tps	0.6s	33K	$0.07	$0.14
54	210	Mistral Nemo	875	±15	915	2.7%	<0.1%	112 tps	0.4s	131K	$0.07	$0.13
55	210	Qwen 2.5 7B Turbo	870	±25	615	6.1%	0.5%	125 tps	0.4s	131K	$0.30	$0.30
56	210	Hermes 2 Pro Llama 3 8B	864	±21	1.8K	2.5%	<0.1%	76 tps	1.0s	131K	$0.08	$0.09
57	210	Mistral Small 24B Instruct	864	±16	1.5K	4.1%	1.5%	84 tps	0.4s	33K	$0.80	$0.80
58	293	Hermes 2 Mixtral 8x7B DPO	863	±17	1.2K	1.3%	<0.1%	1 tps	N/A	33K	$0.60	$0.60
59	312	Yi Large	858	±12	1.5K	<0.1%	<0.1%	34 tps	N/A	33K	$1.50	$1.50
60	312	Command Light	856	±16	1.1K	4.9%	<0.1%	23 tps	N/A	4K	$0.10	$0.20
61	210	Gemma 3 27B	856	±27	1.1K	6.9%	1.8%	35 tps	1.1s	66K	$0.06	$0.10
62	210	Mixtral 8x7B	855	±18	1.3K	5.1%	2.2%	142 tps	0.6s	33K	$0.23	$0.23
63	210	Mixtral 8x7B Instruct	854	±16	1.4K	4.4%	0.2%	79 tps	0.7s	33K	$0.23	$0.31
64	234	Gemma 3 27B IT	853	±10	2.3K	3.9%	2.0%	60 tps	0.8s	128K	$0.17	$0.29
65	234	Llama 3.3 70B Instruct Turbo	851	±19	1.2K	6.0%	2.0%	78 tps	1.0s	131K	$0.88	$0.88
66	234	Command R 7B	849	±15	3.3K	4.8%	1.1%	76 tps	0.4s	128K	$0.04	$0.15
67	240	DeepSeek-R1 Distill Llama 70B	835	±9	3.4K	5.2%	3.6%	27 tps	1.6s	32K	$0.73	$0.95
68	324	Typhoon 2 70B Instruct	835	±15	1.4K	4.0%	<0.1%	19 tps	0.1s	8K	$0.88	$0.88
69	324	OLMo 2 0425 1B Instruct	833	±21	560	2.6%	<0.1%	68 tps	0.0s	4K	$0	$0
70	240	Mixtral-8x7B Instruct v0.1	832	±23	1.3K	4.6%	1.3%	54 tps	0.4s	33K	$0.60	$0.60
71	240	Qwen 2.5 7B	831	±17	2K	5.1%	3.7%	40 tps	1.9s	131K	$0.08	$0.27
72	240	Sky T1 32B Preview	829	±14	2.4K	4.5%	7.8%	73 tps	0.6s	16K	$0.12	$0.18
73	240	Ministral 8B	825	±17	2.2K	5.5%	1.4%	177 tps	0.4s	128K	$0.14	$0.14
74	240	C4AI Aya Expanse 32B	821	±7	3.8K	4.0%	1.5%	43 tps	0.5s	128K	$0.50	$1.50
75	240	Gemma 2 27B	815	±17	1.5K	4.1%	1.4%	44 tps	1.4s	8K	$0.80	$0.80
76	252	Ministral 3B	806	±16	2.3K	5.1%	0.8%	248 tps	0.4s	131K	$0.08	$0.08
77	252	Gemma 3 1B	802	±11	2K	6.1%	0.6%	176 tps	1.0s	33K	$0.06	$0.10
78	252	Phi 4	798	±16	1.7K	3.4%	5.1%	28 tps	1.3s	128K	$0.10	$0.32
79	262	Command R	778	±18	2.2K	4.9%	5.8%	54 tps	0.6s	128K	$0.30	$0.99
80	262	Mistral Small	770	±12	1.2K	4.5%	1.7%	142 tps	0.6s	32K	$0.43	$1.30

2of3

View All (99 models)