InternVL3_5-14B-AWB
#5
by
thdeus12
- opened
I am using a quantized model of InternVL3_5-14B for it to work on my home PC since it cannot run the base model well. However, in VLMEvalKit and LMDeploy, they do not use GGUFs that I used on my Ollama and LM Studio. They need AWQs (I already tried cyankiwi/InternVL3_5-14B-AWQ-8bit) but they didn't work and showed error "AssertionError: unsupported quant_config ... 'quant_method': 'compressed-tensors'". When will the official AWQ be out?