Spaces:
Runtime error
Runtime error
A newer version of the Gradio SDK is available: 6.14.0
metadata
title: UncGPT-69 hybrid demo (stage 1, step 2000)
emoji: 🦉
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 4.44.1
app_file: app.py
pinned: false
UncGPT-69 hybrid demo
47M-param Jamba-style hybrid (10 Mamba-2 + 2 MQA-attn) + MoE (1 shared + 6 routed top-k=2) + BitNet 1.58b weights. Trained from scratch ~64 min on 4×L40 to lm_loss 1.88 (perplexity ≈ 6.55).