Qwen2.5-1.5B-Instruct

pack Qwen2.5-1.5B-Instruct@e2cc88932f748bcb (id e2cc88932f748bcb), 8 result(s). Lower KLD is better. Click a column to sort.

KLD mean against file size. Bars are 95% intervals. Lower and further left is better.

Quant	Publisher	Size	KLD mean	95% interval	Worst 1%	Top-1	PPL ratio	Loops (q/ref)	JSON ok	Backend
Q8_0	Qwen	1.9 GB	0.0152	0.0137 to 0.0170	0.154	94.2%	1.010	28/16	8/8	llamacpp
Q6_K	Qwen	1.5 GB	0.0187	0.0170 to 0.0206	0.192	93.0%	1.012	25/16	8/8	llamacpp
Q5_K_M	Qwen	1.3 GB	0.0297	0.0268 to 0.0329	0.326	91.8%	1.023	28/16	8/8	llamacpp
IQ4_XS	bartowski	895.7 MB	0.0563	0.0493 to 0.0646	0.626	89.5%	1.042	17/16	8/8	llamacpp
Q4_K_M	Qwen	1.1 GB	0.0616	0.0546 to 0.0697	0.701	88.6%	1.056	26/16	8/8	llamacpp
IQ3_M	bartowski	776.7 MB	0.1494	0.1345 to 0.1661	1.731	82.6%	1.155	21/16	8/8	llamacpp
Q3_K_M	Qwen	924.5 MB	0.1592	0.1425 to 0.1778	1.894	82.4%	1.159	24/16	6/8	llamacpp
Q2_K	Qwen	752.9 MB	0.6087	0.5596 to 0.6602	4.707	64.9%	1.822	38/16	8/8	llamacpp

approx = the server only reported its own top-N, so some reference tokens were counted in the tail bucket. nd = the same text gave slightly different numbers on two runs. "Loops" shows quant / original. All results used the same probe set and pack.