Yi 1.5 34B Chat Int8 GPTQ
#8
by cduk - opened
Could you please add an Int8 GPTQ quantization of this Yi 1.5 34B Chat model for easy deployment on vLLM? Thanks.
cduk changed discussion title from Int8 GPTQ to Yi 1.5 34B Chat Int8 GPTQ
Hi cduk,
We don't plan to provide an official this time. ModelScope community has provided AWQ and GTPQ variants using their SWIFT framework, check here: https://www.modelscope.cn/models/tclf90/Yi-1.5-34B-Chat-16K-GPTQ-Int4/summary
lorinma changed discussion status to closed