Leaderboard | Coding

Models

Choose model family

Claude by Anthropic

Mistral by Mistral AI

More filters

Show inactive models

Hide models that are no longer actively available on Yupp.

Turns

Filter model performance by the number of turns in a conversation.

All Single turn Multiple turns

Open license models

Filter the leaderboard to only show models that have an open license.

All selected Open license Proprietary license

899

Moonshot V1 Auto

900

GPT-4o mini

903

Mistral Small 3.2 24B Instruct

904

Llama 3 8B

908

GPT-3.5 Turbo

908

Magistral Small 2506

912

Inception Mercury Coder Small Beta

913

Seed 1.6 Flash 250715

915

Rnj-1 Instruct

916

Devstral Small

917

Jamba 1.7 Large

918

Open Mistral Nemo

919

Llama 3.3 Swallow 70B Instruct

924

Jamba 1.6 Large

925

Mistral Small 3.1

Last updated about 1 month ago

Rank	Name	VIBE Score	Confidence Interval	Votes	Downvote %	Abort %	Speed	Latency	Context	Cost (Input)	Cost (Output)
81	Moonshot V1 Auto	899	±22	930	4.1%	1.2%	54 tps	1.5s	8K	$2.00	$5.00
82	GPT-4o mini	900	±14	2.8K	5.3%	2.1%	71 tps	1.7s	128K	$0.15	$0.60
83	Mistral Small 3.2 24B Instruct	903	±20	850	8.6%	1.9%	113 tps	1.1s	131K	$0.02	$0.08
84	Llama 3 8B	904	±10	3.2K	3.5%	6.0%	85 tps	0.7s	8K	$0.12	$0.16
85	GPT-3.5 Turbo	908	±15	1.2K	2.5%	1.3%	74 tps	0.9s	16K	$0.75	$1.75
86	Magistral Small 2506	908	±11	4.3K	3.1%	1.6%	156 tps	0.5s	40K	$0.37	$1.10
87	Inception Mercury Coder Small Beta	912	±20	610	3.2%	1.7%	270 tps	1.4s	32K	$0.25	$1.00
88	Seed 1.6 Flash 250715	913	±16	1.2K	5.7%	2.5%	108 tps	1.6s	256K	$0.07	$0.30
89	Rnj-1 Instruct	915	±21	970	6.7%	0.6%	103 tps	0.3s	33K	$0.15	$0.15
90	Devstral Small	916	±17	1.4K	5.0%	2.4%	180 tps	0.6s	131K	$0.10	$0.30
91	Jamba 1.7 Large	917	±16	975	8.0%	1.3%	58 tps	1.0s	256K	$1.33	$5.33
92	Open Mistral Nemo	918	±21	1.7K	4.5%	1.5%	171 tps	0.5s	131K	$0.15	$0.15
93	Llama 3.3 Swallow 70B Instruct	919	±8	3.5K	5.5%	1.4%	153 tps	1.3s	131K	$0.13	$0.39
94	Jamba 1.6 Large	924	±9	3.3K	3.8%	2.0%	59 tps	1.2s	256K	$1.33	$5.33
95	Mistral Small 3.1	925	±14	2.4K	4.0%	7.4%	13 tps	2.6s	32K	$0.17	$0.28
96	Grok 3 Mini	927	±6	9.9K	6.4%	1.2%	43 tps	0.5s	131K	$0.30	$0.50
97	Codestral	931	±19	855	6.0%	5.2%	151 tps	0.9s	262K	$0.15	$0.45
98	Llama 3.3 70B	933	±10	3K	6.6%	0.3%	500 tps	0.5s	8K	$0.48	$0.66
99	DeepSeek Prover v2	935	±10	1.3K	3.0%	5.2%	14 tps	1.3s	164K	$0.40	$1.56
100	DeepSeek-R1	939	±6	6.4K	4.3%	0.8%	133 tps	0.6s	64K	$0.91	$3.07
101	NVIDIA Llama 3.3 Nemotron Super 49B v1.5	941	±19	1.4K	6.9%	2.0%	50 tps	0.6s	131K	$0.09	$0.33
102	ERNIE 4.5 VL 424B A47B	942	±18	725	6.5%	4.9%	36 tps	3.5s	123K	$0.42	$1.25
103	ERNIE 4.5 21B A3B	943	±28	540	6.9%	2.3%	78 tps	1.5s	120K	$0.05	$0.19
104	Grok 3 Mini Fast	943	±7	9K	7.0%	1.6%	44 tps	0.5s	131K	$0.60	$4.00
105	Ministral 14B 3.0	948	±16	805	8.5%	2.0%	119 tps	0.5s	128K	$0.20	$0.20
106	Qwen3 8B	948	±9	4.2K	8.2%	2.4%	61 tps	1.4s	41K	$0.02	$0.07
107	Switchpoint Router	949	±10	2.7K	3.6%	1.7%	71 tps	4.9s	131K	$0.85	$3.40
108	Qwen3 30B A3B Thinking 2507	953	±10	3.5K	4.7%	0.5%	124 tps	1.2s	131K	$0.16	$1.70
109	Llama 3.1 8B Turbo	958	±14	2.2K	2.0%	2.1%	650 tps	0.5s	128K	$0.13	$0.14
110	Qwen 2.5 72B	960	±15	1.4K	4.5%	1.2%	96 tps	1.2s	131K	$0.14	$0.26
111	Qwen3 14B	962	±8	5.3K	8.4%	1.7%	109 tps	0.8s	41K	$0.04	$0.15
112	Devstral Medium	962	±11	3.5K	5.2%	1.5%	77 tps	0.6s	131K	$0.40	$2.00
113	Llama 4 Scout	965	±5	17.5K	5.3%	0.6%	88 tps	5.1s	131K	$0.18	$0.46
114	Qwen3 VL 30B A3B Thinking	967	±11	1.9K	8.9%	4.5%	84 tps	2.9s	127K	$0.20	$1.47
115	Pixtral Large	969	±14	3.5K	3.9%	2.5%	57 tps	1.3s	128K	$1.50	$4.50
116	Mistral Small 3.2 24B	970	±13	4.6K	4.9%	2.8%	141 tps	0.7s	33K	$0.02	$0.08
117	Llama 4 Maverick	971	±5	21K	5.0%	1.2%	88 tps	2.4s	1M	$0.23	$0.83
118	Qwen 2.5 32B Instruct	972	±7	4.1K	6.5%	2.5%	48 tps	1.0s	131K	$0.21	$0.25
119	Nemotron 3 Nano	974	±46	580	6.5%	1.3%	216 tps	0.8s	256K	$0.05	$4.94
120	DeepSeek V3.1 Thinking	976	±9	5.2K	9.5%	7.1%	18 tps	1.8s	131K	$0.23	$0.75

3of8

View All (286 models)