Qwen3.5-9B-MXFP4A16

Quantized version of Qwen/Qwen3.5-9B using MXFP4A16 (4-bit floating point weights, 16-bit activations).

Quantization Details

Method: MXFP4A16 (weight-only, per group 16)
Tool: vllm-project/llm-compressor
Ignored modules: lm_head, visual encoder (re:.*visual.*), linear attention (re:.*linear_attn.*), MTP modules (re:.*mtp.*)

vllm serve 2imi9/Qwen3.5-9B-MXFP4A16

Safetensors

Model size

6B params

Tensor type

BF16

Base model

Finetuned

Quantized

(249)

this model