Qwen2.5-0.5B-Instruct

pack Qwen2.5-0.5B-Instruct@f499c4436fa8e370 (id f499c4436fa8e370), 15 result(s). Lower KLD is better. Click a column to sort.

KLD mean against file size. Bars are 95% intervals. Lower and further left is better.

Quant	Publisher	Size	KLD mean	95% interval	Worst 1%	Top-1	PPL ratio	Loops (q/ref)	JSON ok	Backend
Q8_0	bartowski	531.1 MB	0.0025	0.0023 to 0.0026	0.021	97.1%	1.003	27/23	7/8	llamacpp
Q6_K	bartowski	505.7 MB	0.0043	0.0039 to 0.0047	0.039	96.6%	1.003	22/23	8/8	llamacpp
Q5_K_M	bartowski	420.1 MB	0.0229	0.0205 to 0.0258	0.249	92.3%	1.023	24/23	8/8	llamacpp
Q8_0	Qwen	675.7 MB	0.0275	0.0251 to 0.0304	0.284	91.4%	1.035	27/23	8/8	llamacpp
Q4_K_M	bartowski	397.8 MB	0.0337	0.0303 to 0.0382	0.325	90.6%	1.035	29/23	8/8	llamacpp
Q4_K_S	bartowski	385.5 MB	0.0429	0.0384 to 0.0485	0.453	89.3%	1.042	22/23	8/8	llamacpp
Q5_K_M	Qwen	522.2 MB	0.0498	0.0451 to 0.0551	0.546	88.8%	1.057	24/23	8/8	llamacpp
Q4_K_M	Qwen	491.4 MB	0.0647	0.0587 to 0.0716	0.691	87.2%	1.066	31/23	8/8	llamacpp
Q3_K_M	bartowski	355.5 MB	0.0724	0.0648 to 0.0815	0.848	86.5%	1.072	27/23	8/8	llamacpp
IQ4_XS	bartowski	349.4 MB	0.0837	0.0748 to 0.0943	1.029	86.0%	1.076	27/23	7/8	llamacpp
Q3_K_M	Qwen	432.0 MB	0.1109	0.1002 to 0.1235	1.323	83.9%	1.115	26/23	8/8	llamacpp
IQ3_XS	bartowski	338.6 MB	0.1258	0.1140 to 0.1399	1.398	82.6%	1.132	31/23	8/8	llamacpp
Q2_K	bartowski	338.6 MB	0.1706	0.1561 to 0.1872	1.831	79.6%	1.197	34/23	8/8	llamacpp
Q2_K	Qwen	415.2 MB	0.1761	0.1605 to 0.1954	1.958	79.5%	1.214	29/23	8/8	llamacpp
IQ2_M	bartowski	328.6 MB	0.2604	0.2404 to 0.2829	2.453	74.9%	1.310	34/23	8/8	llamacpp

approx = the server only reported its own top-N, so some reference tokens were counted in the tail bucket. nd = the same text gave slightly different numbers on two runs. "Loops" shows quant / original. All results used the same probe set and pack.