Qwen3-8B DFly MLX

MLX conversion of AngelSlim/Qwen3-8B-DFly-Block8 for native DFly speculative decoding on Apple Silicon.

The conversion uses upstream revision 5712926, the last revision with the correct Qwen3-8B metadata. The current upstream config.json describes an incompatible 80-layer target, so dfly-mlx pins and validates the known-good revision.

Use

pip install dfly-mlx
hf download will702/Qwen3-8B-DFly-MLX \
  --local-dir ./models/qwen3-8b-dfly-mlx

dfly-mlx generate \
  --draft ./models/qwen3-8b-dfly-mlx \
  --prompt "Write a quicksort in Python." \
  --max-tokens 256

The target defaults to mlx-community/Qwen3-8B-4bit. Generation is greedy and uses Qwen3 no-thinking mode.

Verification status

Checkpoint conversion, loading, and generation were tested on Apple Silicon with MLX 0.32.0. Every emitted draft proposal is accepted by the target verifier. As with the underlying DFlash runtime, output can differ from pure autoregressive generation near tied logits because MLX dispatch differs between single-token and block verification.

Source, reproducible benchmark command, and attribution: will702/dfly-mlx.

License

Apache-2.0. The original AngelSlim model license is included as License_AngelSlim_model_and_dataset.txt.

Downloads last month: 9

Safetensors

Model size

1B params

Tensor type

BF16

MLX

Hardware compatibility

Quantized

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for will702/Qwen3-8B-DFly-MLX

Base model

Qwen/Qwen3-8B-Base

Finetuned

Qwen/Qwen3-8B

Finetuned

(1995)

this model