is Imatrix better than the regular Quants?
i'm curious about it, I see two GGUF models one had the "i" for the Imatrix, but confused on which I should use.
Weighted/imatrix quants offer higher quality than static quants at the same model size and resource usage. If unsure always use weighted/imatrix quants. I recommend you consult the quality column at ouer download page linked in all ouer model cards. You can even select diffrent metric like KL divergence, Perplexity, Same token probablity and eval results to check what quant best fits your needs.
Weighted/imatrix quants offer higher quality than static quants at the same model size and resource usage. If unsure always use weighted/imatrix quants. I recommend you consult the quality column at ouer download page linked in all ouer model cards. You can even select diffrent metric like KL divergence, Perplexity, Same token probablity and eval results to check what quant best fits your needs.
Thank you so much for the info.
i'm curious about it, I see two GGUF models one had the "i" for the Imatrix, but confused on which I should use.
Also see these benchmarks https://huggingface.co/mradermacher/BabyHercules-4x150M-GGUF/discussions/2#674a7958ce9bc37b8e33cf55