Byrne-Embed

85M sentence embedder. Maps text to 768-dim unit-norm vectors - similarity, retrieval, clustering, reranking.

Backbone is a SpikeWhale decoder (Byrne line). Mean-pool last hidden state, learned head to 768, unit-normalize. Cosine is a dot product.

Benchmark vs. EmbeddingGemma-300M

I ran Byrne-Embed against Google's EmbeddingGemma-300M on 4,000 held-out sentences - educational web, encyclopedic, instruction/chat. Geometry tracks EmbeddingGemma at about 1/3.5 the params:

Metric (Byrne-Embed vs EmbeddingGemma)	Result
Mean per-sentence cosine	0.9415 (median 0.945, p10 0.912)
Sentences within 0.90 cosine	94.7%
Similarity-structure agreement (Pearson)	0.9702
Similarity-structure agreement (Spearman)	0.9599
Per-anchor neighbour-ranking correlation	0.9494
Retrieval top-1 nearest-neighbour agreement	72.8%
Retrieval Recall@10 overlap	78.2%

The two that matter most - do they agree on which sentences are similar - are Pearson 0.97 / Spearman 0.96. When EmbeddingGemma says two sentences are similar, this one says the same thing. 94.7% of sentences sit within 0.90 cosine. Top-1 retrieval is lower and that is expected, not a quality hole: dense real sentences have a lot of near-ties (0.88 vs 0.87), so #1 flips between near-duplicates. That is why Recall@10 stays ~78% and neighbour-ranking correlation is 0.95. Same neighbourhood; they just swap rank 1 and 2 on near-identical candidates.

Reproduce with run_tests.py (loads both models, prints the full table).

MTEB English Benchmark - MTEB(eng, v2)

Evaluated with the official mteb library on the full MTEB(eng, v2) suite (41/41 tasks). Raw results are in mteb_results/; machine-readable scores are in the model-index metadata above.

Overall MTEB(eng, v2) mean: 50.79

Category	Mean	Tasks
STS	71.93	9
Classification	70.57	8
PairClassification	74.07	3
Clustering	37.32	8
Reranking	40.48	2
Retrieval	24.64	10
Summarization	22.39	1

STS

Task	Score
BIOSSES	75.56
SICK-R	69.08
STS12	64.88
STS13	72.08
STS14	67.76
STS15	77.13
STS17	83.23
STS22.v2	60.53
STSBenchmark	77.08

Classification

Task	Score
AmazonCounterfactualClassification	80.12
Banking77Classification	74.64
ImdbClassification	60.97
MTOPDomainClassification	92.29
MassiveIntentClassification	63.23
MassiveScenarioClassification	73.05
ToxicConversationsClassification	62.94
TweetSentimentExtractionClassification	57.29

PairClassification

Task	Score
SprintDuplicateQuestions	86.47
TwitterSemEval2015	53.19
TwitterURLCorpus	82.55

Clustering

Task	Score
ArXivHierarchicalClusteringP2P	53.15
ArXivHierarchicalClusteringS2S	50.39
BiorxivClusteringP2P.v2	33.73
MedrxivClusteringP2P.v2	32.70
MedrxivClusteringS2S.v2	29.04
StackExchangeClustering.v2	41.93
StackExchangeClusteringP2P.v2	35.22
TwentyNewsgroupsClustering.v2	22.39

Reranking

Task	Score
AskUbuntuDupQuestions	52.88
MindSmallReranking	28.07

Retrieval

Task	Score
ArguAna	37.67
CQADupstackGamingRetrieval	37.14
CQADupstackUnixRetrieval	23.48
ClimateFEVERHardNegatives	13.60
FEVERHardNegatives	28.70
FiQA2018	11.38
HotpotQAHardNegatives	30.47
SCIDOCS	10.15
TRECCOVID	29.30
Touche2020Retrieval.v3	24.50

Summarization

Task	Score
SummEvalSummarization.v2	22.39

Usage

Standard transformers via trust_remote_code (projection head is fused; one from_pretrained loads everything):

import torch
from transformers import AutoModel, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Quazim0t0/Byrne-Embed", trust_remote_code=True)
model = AutoModel.from_pretrained("Quazim0t0/Byrne-Embed", trust_remote_code=True).eval()

texts = ["The cat sat on the windowsill.", "A feline rested by the window."]
enc = tok(texts, return_tensors="pt", padding=True, truncation=True, max_length=128)
with torch.no_grad():
    emb = model(**enc).last_hidden_state          # (2, 768), L2-normalized

print(float(emb[0] @ emb[1]))                      # cosine similarity ~ 0.83

forward() returns L2-normalized 768-dim sentence embeddings. Cosine is a dot product.

Files

File	Purpose
model.safetensors, config.json	fused SpikeWhale backbone + projection head + config
modeling_byrne_embed.py	self-contained custom AutoModel class (SpikeWhale arch inlined; loaded via trust_remote_code)
tokenizer.json, tokenizer_config.json, spike_tokenizer.py	byte-level SpikeTokenizer + its code

Limitations

• English-centric evaluation; non-English is untested.
• Weak spot I saw: finance/economics paraphrase retrieval. General semantic similarity is strong.
• Custom architecture: load via the bundled byrne_embedder.py (local modeling code - no remote code execution).

Citation

If you use Byrne-Embed, please cite:

@misc{byrne2026byrneembed,
  title        = {Byrne-Embed: A Compact 85M Sentence-Embedding Model},
  author       = {Byrne, Dean},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Quazim0t0/Byrne-Embed}},
}

License

Apache-2.0.