In partnership with Bench Labs

What is "FWKV"???
FWKV (call it Feed-forward Weighted Key Value, or Floored Weighted Key Value, its up to you) is a novel efficient language model architecture proposed on May 13, 2026 by me.

It started as a fun side project inspired by RWKV. Then I (We, FlameF0X) realised that for a 50 million parameter language model is fast, like, faster than any Transformers base Dense or MoE 50m model

Why should I use FWKV?
You dont use it since its still in its early stages and its very unstable. But its fast.

Models

How do we name the models?
Very simple actually!
FWKV-[model size or type or something uniquie]
They must stay simple so you can search easily.

Models:

• Language models:
  • FWKV-100M (being planned to enter in development in 2026-2027)
  • FWKV-Translation (being planned to enter in development in late 2026)
  • FWKV-ROSA
  • FWKV-50M
  • FWKV-29M
  • FKWV-Tinystories
• Vision models:
  • FWKV-Image
  • FWKV-CLIP (NAP - Not A Prioriy, you will see this very frequently)
• Other modalities:
  • FWKV-TTS (NAP)
  • FWKV-Bert (NAP)