Convert in bf16 or fp16?
#2
by remember2015 - opened
I noticed https://github.com/ml-explore/mlx-lm/issues/193 which say : "For many models it's fine to use float16 so in your case you can convert the model using"
so I was wondering which dtype used to quant model
I converted this with mlx-lm 0.24.1 so it should have been inferred from the original model, so bf16.
I'd have to double check but that should be the case!
I converted this with mlx-lm 0.24.1 so it should have been inferred from the original model, so bf16.
I'd have to double check but that should be the case!
thank you for ur reply~
remember2015 changed discussion status to closed
remember2015 changed discussion status to open