GGUF failed - unsupported architecture?
#1
by Muzel - opened
Hiya, huge fan of the "old" Athene-70B running on ollama. I asked mradermacher to quant it into a GGUF, but apparently the architecture is not supported.
Is this a new architecture? And is it possible to create GGUFs of it?
Oh, and quick but exciting question: Did you run any benchmarks? I was (and still am) ever so intrigued by the really good ranking in the Arena-Leaderboard!
this is a reward model, you wouldn't typically run this as a GGUF model (I mean, you could, but yeah this is not a general chat model)