FINAL_Bench

A small gift for anyone building or studying foundation models.

Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing — so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.

What you can do with it:
🔁 Rebuild it from scratch, or fork the recipe for your own model
🔬 Study a real heterogeneous-attention MoE — 49 layers place 5 attention mechanisms on a 7×7 Latin square, arranged as a clean, attributable ablation
📈 Trace training dynamics across the released checkpoints (110k / 115k / 162k)

It's a modest 6.59B model, and an honest one — the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. 🤗

📖 Full write-up →
[blog] · https://huggingface.co/blog/FINAL-Bench/opensource-llm
📦 5 Attention Base · FINAL-Bench/Aether-7B-5Attn
🎯 5 Attention Instruct · FINAL-Bench/Aether-7B-5Attn-it
🚀 5 Attention Live demo · FINAL-Bench/Aether-Sovereign-AI
📦 7 Attention Base · https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
📦 11 Attention Base · FINAL-Bench/Aether-6B-11Attn-base
🧬 Collection · https://huggingface.co/collections/FINAL-Bench/aether-foundation-model

#opensource #LLM #MoE #reproducibility #Apache2

5 replies

SeaWolf-AI

published an article 2 days ago

Article

Aether-7B-5Attn: A 100% Open-Source Sovereign Foundation Model — and a Controlled Experiment in Heterogeneous Attention

FINAL-Bench

•

2 days ago

• 21

SeaWolf-AI

updated a collection 2 days ago

'Aether' Foundation Model

Collection

6 items • Updated 1 day ago • 31

SeaWolf-AI

published a dataset 2 days ago

FINAL-Bench/Aether-7B-5Attn-checkpoints

Updated 1 day ago • 11 • 20

AI & ML interests

Recent Activity

Papers

Articles

Aether-7B-5Attn: A 100% Open-Source Sovereign Foundation Model — and a Controlled Experiment in Heterogeneous Attention

VKUE: No GPU? Runs Anyway — a 34.7B Reasoner on a Laptop and on Bare CPU

Quantum Cryptanalysis on Real Hardware: Pushing Symmetric-Structure Key Recovery Beyond the Published Frontier

Adding a GPU Without Building One

Chitos: From Detection to Proof — An Autonomous Security AI That Actually Exploits

FINAL-Bench Quantum: An Open, Neutral Benchmark for Quantum-Computing Methods

Training-Free Reasoning at 88.89% on GPQA Diamond: How Darwin Family Hit Frontier Scores Without a Single Gradient Step

Darwin-TTS: We Gave a TTS Model 3% of an LLM's Brain — It Started Showing Emotion

"Darwin-27B-Opus: Surpassing the Foundation Model Without Training"

Darwin V6: Diagnostic-Guided Evolutionary Model Merging

"The Child That Surpassed Both Parents Through MRI-Guided Evolutionary Merge"

Introducing WM Bench: A Benchmark for Cognitive Intelligence in World Models

🏟️ Smol AI WorldCup: A 5-Axis Benchmark That Reveals What Small Language Models Can Really Do

MARL: Runtime Middleware That Reduces LLM Hallucination Without Fine-Tuning

Structural Problems in AI Benchmarking and the Case for a Unified Evaluation Framework

Do Bubbles Form When Tens of Thousands of AIs Simulate Capitalism?

FINAL Bench: The Real Bottleneck to AGI Is Self-Correction

Team members 1

FINAL-Bench's activity

FINAL-Bench Quantum Leaderboard

Aether-7B-5Attn — 100% Open-source Foundation Model: Sovereign AI

Aether-7B-5Attn: A 100% Open-Source Sovereign Foundation Model — and a Controlled Experiment in Heterogeneous Attention