Feature Extraction
MLX
Safetensors
Transformers
kimi_k25
quantization
dq3
custom_code
4-bit precision
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("cs2764/Kimi-K2.6_dq3-mlx", trust_remote_code=True, dtype="auto")Quick Links
Kimi-K2.6_dq3
This model is a DQ3 quantized version of the original model moonshotai/Kimi-K2.6.
It was quantized locally using the mlx_lm library.
Quantization Methodology (DQ3)
This model was quantized using the dynamic DQ3 (3-bit / 4-bit / 8-bit mixed) approach, inspired by the methodology described in the mlx-community/Kimi-K2.5-mlx-DQ3_K_M-q8 repository.
The weights are mixed based on MLX layers:
- Expert layers (switch_mlp / mlp) are quantized to 3-bit.
- The first 5 layers are kept at higher quality (5-bit).
- Every 5th layer is medium quality (4-bit).
- All other layers (e.g. attention, normalization) remain at 8-bit to serve as the "8-bit brain".
- Downloads last month
- 1,065
Model size
1T params
Tensor type
BF16
路
U32 路
F32 路
Hardware compatibility
Log In to add your hardware
4-bit
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="cs2764/Kimi-K2.6_dq3-mlx", trust_remote_code=True)