SiteAgent - AI ์น ์ด์์คํดํธ
๐ค
22
์ด๋ค ์น ํ์ด์ง์์๋ ๋์ํ๋ AI ์ด์์คํดํธ.
ginigen-ai PRO
ginigen-ai
AI & ML interests
None yet
Recent Activity
updated
a Space about 5 hours ago
ginigen-ai/site-agent liked
a Space about 23 hours ago
ginigen-ai/site-agent published
a Space 2 days ago
ginigen-ai/site-agent Organizations
None yet
updated a Space about 5 hours ago
liked a Space about 23 hours ago
published a Space 2 days ago
updated a Space 3 days ago
Request for Benchmark Evaluation
4
#1 opened 5 days ago
by
ginigen-ai
liked a Space 4 days ago
reacted to mayafree's post with ๐๐ฅ 4 days ago
Post
5773
Leaderboard of Leaderboards โ A Real-Time Meta-Ranking of AI Benchmarks
MAYA-AI/all-leaderboard
Hundreds of AI leaderboards exist on HuggingFace. Knowing which ones the community actually trusts has never been easy โ until now.
Leaderboard of Leaderboards (LoL) ranks the leaderboards themselves, using live HuggingFace trending scores and cumulative likes as the signal. No editorial curation. No manual selection. Just what the global AI research community is actually visiting and endorsing, surfaced in real time.
Sort by trending to see what is capturing attention right now, or by likes to see what has built lasting credibility over time. Nine domain filters let you zero in on what matters most to your work, and every entry shows both its rank within this collection and its real-time global rank across all HuggingFace Spaces.
The collection spans well-established standards like Open LLM Leaderboard, Chatbot Arena, MTEB, and BigCodeBench alongside frameworks worth watching. FINAL Bench targets AGI-level evaluation across 100 tasks in 15 domains and recently reached the global top 5 in HuggingFace dataset rankings. Smol AI WorldCup runs tournament-format competitions for sub-8B models scored via FINAL Bench criteria. ALL Bench aggregates results across frameworks into a unified ranking that resists the overfitting risks of any single standard.
The deeper purpose is not convenience. It is transparency. How we measure AI matters as much as the AI we measure.
MAYA-AI/all-leaderboard
Hundreds of AI leaderboards exist on HuggingFace. Knowing which ones the community actually trusts has never been easy โ until now.
Leaderboard of Leaderboards (LoL) ranks the leaderboards themselves, using live HuggingFace trending scores and cumulative likes as the signal. No editorial curation. No manual selection. Just what the global AI research community is actually visiting and endorsing, surfaced in real time.
Sort by trending to see what is capturing attention right now, or by likes to see what has built lasting credibility over time. Nine domain filters let you zero in on what matters most to your work, and every entry shows both its rank within this collection and its real-time global rank across all HuggingFace Spaces.
The collection spans well-established standards like Open LLM Leaderboard, Chatbot Arena, MTEB, and BigCodeBench alongside frameworks worth watching. FINAL Bench targets AGI-level evaluation across 100 tasks in 15 domains and recently reached the global top 5 in HuggingFace dataset rankings. Smol AI WorldCup runs tournament-format competitions for sub-8B models scored via FINAL Bench criteria. ALL Bench aggregates results across frameworks into a unified ranking that resists the overfitting risks of any single standard.
The deeper purpose is not convenience. It is transparency. How we measure AI matters as much as the AI we measure.
- 5 replies
liked 2 Spaces 4 days ago
On the parameter counts and VRAM requirements
1
#2 opened 5 days ago
by
PierreLepagnol
upvoted 2 articles 5 days ago
Article
MARL: Runtime Middleware That Reduces LLM Hallucination Without Fine-Tuning
โข
15
Article
Structural Problems in AI Benchmarking and the Case for a Unified Evaluation Framework
โข
12
reacted to SeaWolf-AI's post with ๐ค๐๐๐ฅ 5 days ago
Post
6107
ALL Bench Leaderboard โ Structural Problems in AI Benchmarking and the Case for Unified Evaluation
FINAL-Bench/all-bench-leaderboard
The AI benchmark ecosystem has three structural problems. Major benchmarks like MMLU have surpassed 90%, losing discriminative power. Most leaderboards publish unverified self-reported scores โ our cross-verification found Claude Opus 4.6's ARC-AGI-2 listed as 37.6% (actual: 68.8%), Gemini 3.1 Pro as 88.1% (actual: 77.1%). OpenAI's own audit confirmed 59.4% of SWE-bench Verified tasks are defective, yet it remains widely used.
ALL Bench addresses this by comparing 91 models across 6 modalities (LLM ยท VLM ยท Agent ยท Image ยท Video ยท Music) with 3-tier confidence badges (โโ cross-verified ยท โ single-source ยท ~ self-reported). Composite scoring uses a 5-Axis Framework and replaces SWE-Verified with contamination-resistant LiveCodeBench.
Key finding: metacognition is the largest blind spot. FINAL Bench shows Error Recovery explains 94.8% of self-correction variance, yet only 9 of 42 models are even measured. The 9.2-point spread (Kimi K2.5: 68.71 โ rank 9: 59.5) is 3ร the GPQA top-model spread, suggesting metacognition may be the single biggest differentiator among frontier models today.
VLM cross-verification revealed rank reversals โ Claude Opus 4.6 leads MMMU-Pro (85.1%) while Gemini 3 Flash leads MMMU (87.6%), producing contradictory rankings between the two benchmarks.
๐ Article: https://huggingface.co/blog/FINAL-Bench/all-bench
๐ฆ Dataset: FINAL-Bench/ALL-Bench-Leaderboard
โก GitHub: https://github.com/final-bench/ALL-Bench-Leaderboard
๐ Leaderboard: FINAL-Bench/all-bench-leaderboard
๐งฌ FINAL Bench: FINAL-Bench/Metacognitive
FINAL-Bench/all-bench-leaderboard
The AI benchmark ecosystem has three structural problems. Major benchmarks like MMLU have surpassed 90%, losing discriminative power. Most leaderboards publish unverified self-reported scores โ our cross-verification found Claude Opus 4.6's ARC-AGI-2 listed as 37.6% (actual: 68.8%), Gemini 3.1 Pro as 88.1% (actual: 77.1%). OpenAI's own audit confirmed 59.4% of SWE-bench Verified tasks are defective, yet it remains widely used.
ALL Bench addresses this by comparing 91 models across 6 modalities (LLM ยท VLM ยท Agent ยท Image ยท Video ยท Music) with 3-tier confidence badges (โโ cross-verified ยท โ single-source ยท ~ self-reported). Composite scoring uses a 5-Axis Framework and replaces SWE-Verified with contamination-resistant LiveCodeBench.
Key finding: metacognition is the largest blind spot. FINAL Bench shows Error Recovery explains 94.8% of self-correction variance, yet only 9 of 42 models are even measured. The 9.2-point spread (Kimi K2.5: 68.71 โ rank 9: 59.5) is 3ร the GPQA top-model spread, suggesting metacognition may be the single biggest differentiator among frontier models today.
VLM cross-verification revealed rank reversals โ Claude Opus 4.6 leads MMMU-Pro (85.1%) while Gemini 3 Flash leads MMMU (87.6%), producing contradictory rankings between the two benchmarks.
๐ Article: https://huggingface.co/blog/FINAL-Bench/all-bench
๐ฆ Dataset: FINAL-Bench/ALL-Bench-Leaderboard
โก GitHub: https://github.com/final-bench/ALL-Bench-Leaderboard
๐ Leaderboard: FINAL-Bench/all-bench-leaderboard
๐งฌ FINAL Bench: FINAL-Bench/Metacognitive
reacted to SeaWolf-AI's post with ๐ฅ 5 days ago
Post
8134
๐ Introducing MARL โ Runtime Middleware That Reduces LLM Hallucination Without Fine-Tuning
Now available on PyPI ยท GitHub ยท ClawHub ยท HuggingFace
AI models sense they could be wrong, but they can't actually fix what's broken.
๐ค Live A/B test: VIDraft/MARL
We evaluated 9 SOTA models (GPT-5.2, Claude Opus 4.6, Gemini 3 Pro, etc.) across 1,800 assessments in FINAL Bench and found a 39.2%p gap between "recognizing potential errors (MA=0.694)" and "actually finding and fixing them (ER=0.302)."
MARL (Model-Agnostic Runtime Middleware for LLMs) was built to close this metacognitive gap. It decomposes a single LLM call into a 5-stage expert pipeline (Hypothesis โ Solver โ Auditor โ Adversarial Verifier โ Synthesizer), transforming "answer in one shot" into "think, doubt, correct, and rewrite."
No weight modification โ works instantly with GPT-5.4, Claude, Gemini, Llama, or any OpenAI API-compatible LLM by changing one line: base_url. Ships with 9 domain-specific emergence engines (invention, pharma, genomics, chemistry, ecology, law, and more โ 5,538 expert data items) activated by a simple tag like model="gpt-5.4::pharma".
pip install marl-middleware
MARL is also officially registered on ClawHub, the skill marketplace of OpenClaw โ an AI agent platform with 260K+ developers and 3,200+ skills. It's the first middleware in the Reasoning Enhancement category. One command โ clawhub install marl-middleware โ gives your AI agent a metacognition upgrade.
๐ Technical deep dive: https://huggingface.co/blog/FINAL-Bench/marl-middleware
๐ฆ PyPI: https://pypi.org/project/marl-middleware/
๐ GitHub: https://github.com/Vidraft/MARL
๐ฆ ClawHub: https://clawhub.ai/Cutechicken99/marl-middleware
#MARL #LLM #Hallucination #Metacognition #MultiAgent #AIMiddleware #FINALBench #OpenClaw #ClawHub #PyPI #AGI #HuggingFace #ReasoningAI #SelfCorrection #GlassBoxAI
Now available on PyPI ยท GitHub ยท ClawHub ยท HuggingFace
AI models sense they could be wrong, but they can't actually fix what's broken.
๐ค Live A/B test: VIDraft/MARL
We evaluated 9 SOTA models (GPT-5.2, Claude Opus 4.6, Gemini 3 Pro, etc.) across 1,800 assessments in FINAL Bench and found a 39.2%p gap between "recognizing potential errors (MA=0.694)" and "actually finding and fixing them (ER=0.302)."
MARL (Model-Agnostic Runtime Middleware for LLMs) was built to close this metacognitive gap. It decomposes a single LLM call into a 5-stage expert pipeline (Hypothesis โ Solver โ Auditor โ Adversarial Verifier โ Synthesizer), transforming "answer in one shot" into "think, doubt, correct, and rewrite."
No weight modification โ works instantly with GPT-5.4, Claude, Gemini, Llama, or any OpenAI API-compatible LLM by changing one line: base_url. Ships with 9 domain-specific emergence engines (invention, pharma, genomics, chemistry, ecology, law, and more โ 5,538 expert data items) activated by a simple tag like model="gpt-5.4::pharma".
pip install marl-middleware
MARL is also officially registered on ClawHub, the skill marketplace of OpenClaw โ an AI agent platform with 260K+ developers and 3,200+ skills. It's the first middleware in the Reasoning Enhancement category. One command โ clawhub install marl-middleware โ gives your AI agent a metacognition upgrade.
๐ Technical deep dive: https://huggingface.co/blog/FINAL-Bench/marl-middleware
๐ฆ PyPI: https://pypi.org/project/marl-middleware/
๐ GitHub: https://github.com/Vidraft/MARL
๐ฆ ClawHub: https://clawhub.ai/Cutechicken99/marl-middleware
#MARL #LLM #Hallucination #Metacognition #MultiAgent #AIMiddleware #FINALBench #OpenClaw #ClawHub #PyPI #AGI #HuggingFace #ReasoningAI #SelfCorrection #GlassBoxAI
reacted to SeaWolf-AI's post with โค๏ธ 5 days ago
Post
11048
๐๏ธ Smol AI WorldCup: A 4B Model Just Beat 8B โ Here's the Data
We evaluated 18 small language models from 12 makers on 125 questions across 7 languages. The results challenge the assumption that bigger is always better.
Community Article: https://huggingface.co/blog/FINAL-Bench/smol-worldcup
Live Leaderboard: ginigen-ai/smol-worldcup
Dataset: ginigen-ai/smol-worldcup
What we found:
โ Gemma-3n-E4B (4B, 2GB RAM) outscores Qwen3-8B (8B, 5.5GB). Doubling parameters gained only 0.4 points. RAM cost: 2.75x more.
โ GPT-OSS-20B fits in 1.5GB yet matches Champions-league dense models requiring 8.5GB. MoE architecture is the edge AI game-changer.
โ Thinking models hurt structured output. DeepSeek-R1-7B scores 8.7 points below same-size Qwen3-8B and runs 2.7x slower.
โ A 1.3B model fabricates confident fake content 80% of the time when prompted with nonexistent entities. Qwen3 family hits 100% trap detection across all sizes.
โ Qwen3-1.7B (1.2GB) outscores Mistral-7B, Llama-3.1-8B, and DeepSeek-R1-14B. Latest architecture at 1.7B beats older architecture at 14B.
What makes this benchmark different?
Most benchmarks ask "how smart?" โ we measure five axes simultaneously: Size, Honesty, Intelligence, Fast, Thrift (SHIFT). Our ranking metric WCS = sqrt(SHIFT x PIR_norm) rewards models that are both high-quality AND efficient. Smart but massive? Low rank. Tiny but poor? Also low.
Top 5 by WCS:
1. GPT-OSS-20B โ WCS 82.6 โ 1.5GB โ Raspberry Pi tier
2. Gemma-3n-E4B โ WCS 81.8 โ 2.0GB โ Smartphone tier
3. Llama-4-Scout โ WCS 79.3 โ 240 tok/s โ Fastest model
4. Qwen3-4B โ WCS 76.6 โ 2.8GB โ Smartphone tier
5. Qwen3-1.7B โ WCS 76.1 โ 1.2GB โ IoT tier
Built in collaboration with the FINAL Bench research team. Interoperable with ALL Bench Leaderboard for full small-to-large model comparison.
Dataset is open under Apache 2.0 (125 questions, 7 languages). We welcome new model submissions.
We evaluated 18 small language models from 12 makers on 125 questions across 7 languages. The results challenge the assumption that bigger is always better.
Community Article: https://huggingface.co/blog/FINAL-Bench/smol-worldcup
Live Leaderboard: ginigen-ai/smol-worldcup
Dataset: ginigen-ai/smol-worldcup
What we found:
โ Gemma-3n-E4B (4B, 2GB RAM) outscores Qwen3-8B (8B, 5.5GB). Doubling parameters gained only 0.4 points. RAM cost: 2.75x more.
โ GPT-OSS-20B fits in 1.5GB yet matches Champions-league dense models requiring 8.5GB. MoE architecture is the edge AI game-changer.
โ Thinking models hurt structured output. DeepSeek-R1-7B scores 8.7 points below same-size Qwen3-8B and runs 2.7x slower.
โ A 1.3B model fabricates confident fake content 80% of the time when prompted with nonexistent entities. Qwen3 family hits 100% trap detection across all sizes.
โ Qwen3-1.7B (1.2GB) outscores Mistral-7B, Llama-3.1-8B, and DeepSeek-R1-14B. Latest architecture at 1.7B beats older architecture at 14B.
What makes this benchmark different?
Most benchmarks ask "how smart?" โ we measure five axes simultaneously: Size, Honesty, Intelligence, Fast, Thrift (SHIFT). Our ranking metric WCS = sqrt(SHIFT x PIR_norm) rewards models that are both high-quality AND efficient. Smart but massive? Low rank. Tiny but poor? Also low.
Top 5 by WCS:
1. GPT-OSS-20B โ WCS 82.6 โ 1.5GB โ Raspberry Pi tier
2. Gemma-3n-E4B โ WCS 81.8 โ 2.0GB โ Smartphone tier
3. Llama-4-Scout โ WCS 79.3 โ 240 tok/s โ Fastest model
4. Qwen3-4B โ WCS 76.6 โ 2.8GB โ Smartphone tier
5. Qwen3-1.7B โ WCS 76.1 โ 1.2GB โ IoT tier
Built in collaboration with the FINAL Bench research team. Interoperable with ALL Bench Leaderboard for full small-to-large model comparison.
Dataset is open under Apache 2.0 (125 questions, 7 languages). We welcome new model submissions.
- 1 reply
reacted to SeaWolf-AI's post with โค๏ธ 6 days ago
Post
6107
ALL Bench Leaderboard โ Structural Problems in AI Benchmarking and the Case for Unified Evaluation
FINAL-Bench/all-bench-leaderboard
The AI benchmark ecosystem has three structural problems. Major benchmarks like MMLU have surpassed 90%, losing discriminative power. Most leaderboards publish unverified self-reported scores โ our cross-verification found Claude Opus 4.6's ARC-AGI-2 listed as 37.6% (actual: 68.8%), Gemini 3.1 Pro as 88.1% (actual: 77.1%). OpenAI's own audit confirmed 59.4% of SWE-bench Verified tasks are defective, yet it remains widely used.
ALL Bench addresses this by comparing 91 models across 6 modalities (LLM ยท VLM ยท Agent ยท Image ยท Video ยท Music) with 3-tier confidence badges (โโ cross-verified ยท โ single-source ยท ~ self-reported). Composite scoring uses a 5-Axis Framework and replaces SWE-Verified with contamination-resistant LiveCodeBench.
Key finding: metacognition is the largest blind spot. FINAL Bench shows Error Recovery explains 94.8% of self-correction variance, yet only 9 of 42 models are even measured. The 9.2-point spread (Kimi K2.5: 68.71 โ rank 9: 59.5) is 3ร the GPQA top-model spread, suggesting metacognition may be the single biggest differentiator among frontier models today.
VLM cross-verification revealed rank reversals โ Claude Opus 4.6 leads MMMU-Pro (85.1%) while Gemini 3 Flash leads MMMU (87.6%), producing contradictory rankings between the two benchmarks.
๐ Article: https://huggingface.co/blog/FINAL-Bench/all-bench
๐ฆ Dataset: FINAL-Bench/ALL-Bench-Leaderboard
โก GitHub: https://github.com/final-bench/ALL-Bench-Leaderboard
๐ Leaderboard: FINAL-Bench/all-bench-leaderboard
๐งฌ FINAL Bench: FINAL-Bench/Metacognitive
FINAL-Bench/all-bench-leaderboard
The AI benchmark ecosystem has three structural problems. Major benchmarks like MMLU have surpassed 90%, losing discriminative power. Most leaderboards publish unverified self-reported scores โ our cross-verification found Claude Opus 4.6's ARC-AGI-2 listed as 37.6% (actual: 68.8%), Gemini 3.1 Pro as 88.1% (actual: 77.1%). OpenAI's own audit confirmed 59.4% of SWE-bench Verified tasks are defective, yet it remains widely used.
ALL Bench addresses this by comparing 91 models across 6 modalities (LLM ยท VLM ยท Agent ยท Image ยท Video ยท Music) with 3-tier confidence badges (โโ cross-verified ยท โ single-source ยท ~ self-reported). Composite scoring uses a 5-Axis Framework and replaces SWE-Verified with contamination-resistant LiveCodeBench.
Key finding: metacognition is the largest blind spot. FINAL Bench shows Error Recovery explains 94.8% of self-correction variance, yet only 9 of 42 models are even measured. The 9.2-point spread (Kimi K2.5: 68.71 โ rank 9: 59.5) is 3ร the GPQA top-model spread, suggesting metacognition may be the single biggest differentiator among frontier models today.
VLM cross-verification revealed rank reversals โ Claude Opus 4.6 leads MMMU-Pro (85.1%) while Gemini 3 Flash leads MMMU (87.6%), producing contradictory rankings between the two benchmarks.
๐ Article: https://huggingface.co/blog/FINAL-Bench/all-bench
๐ฆ Dataset: FINAL-Bench/ALL-Bench-Leaderboard
โก GitHub: https://github.com/final-bench/ALL-Bench-Leaderboard
๐ Leaderboard: FINAL-Bench/all-bench-leaderboard
๐งฌ FINAL Bench: FINAL-Bench/Metacognitive