failed09/bashkir-pair-scorer

Bashkir-Russian Pair Scorer

Compact ONNX models that score whether a Bashkir-Russian sentence pair is parallel, for fast corpus filtering.

Overview

Compact ONNX models for estimating whether a Bashkir-Russian sentence pair is parallel and suitable for corpus filtering. The release is designed for fast, repeatable scoring of large bilingual corpora and contains three model sizes: nano, mini and medium. This is an alignment-quality scorer, not a translation model: it returns a compatibility score for a pair of texts and can be used to build a filtering cascade.

At a glance
Task	Parallel-pair quality estimation
Default artifact	models/mini/model.onnx (recommended)
Source	Reviewed parallel Bashkir-Russian data
Version / license	v1 / Apache-2.0

Contents

Files and Configurations

Path	Purpose	Size
models/nano/model.onnx	Fastest first-pass scorer	9.1 MB
models/mini/model.onnx (recommended)	Balanced default for most batch jobs	15.0 MB
models/medium/model.onnx	Compact higher-quality verifier	23.2 MB
spm_bpe_16k.model	Shared SentencePiece tokenizer	—
config.json	Runtime contract and model metadata	—
benchmark_summary.json	Machine-readable evaluation results	—
META.json	Release passport and artifact hashes	—
SHA256SUMS	Release checksums	—

All configurations use the same SentencePiece vocabulary and the fast128 profile: FP16 weights, dynamic batch dimension and a maximum of 128 tokens per side. This is a deliberate production design for stable speed and predictable memory use; longer inputs are truncated before scoring.

Model Architecture

Property	Description
Task	Parallel-pair compatibility scoring
Directions	ba_ru, ru_ba
Profile	fast128 (FP16, dynamic batch, ≤128 tokens per side)
Output	score_logit (Sigmoid to 0.0–1.0)
Tokenizer	spm_bpe_16k.model (SentencePiece BPE)
Runtime	ONNX Runtime on CPU or GPU
Starting thresholds	accepted ≥ 0.80, review ≥ 0.50 (heuristics; calibrate per corpus)

Examples

Scores from the medium model on CPU (Sigmoid of score_logit):

Bashkir	Russian	Score
Бөгөн һауа бик йылы.	Сегодня очень тепло.	0.981
Башҡортостан Республикаһы — Рәсәй Федерацияһы субъекты.	Республика Башкортостан — субъект Российской Федерации.	0.984
Мин башҡорт телен яратам.	Квантовая механика описывает поведение микрочастиц и полей.	0.000

Method

reviewed parallel data + hard negatives + teacher signals → compact cross-encoder
    → ONNX export (fast128, FP16) → nano / mini / medium

The models were trained for Bashkir-Russian parallel-corpus quality estimation using reviewed parallel data, hard negative pairs and teacher/reference scoring signals. The release contains model weights and tokenizer assets only; the source corpus text is not included.

Benchmark

Comparison on a calibrated bilingual evaluation set (10,000 positive pairs and 20,000 deterministic global/local hard negatives, excluded from training). Full numbers are in benchmark_summary.json.

Model	ROC-AUC	Average Precision	RTX 4060 throughput
LaBSE	0.9944	0.9895	44.8 pairs/s
LaBSE + LASER	0.9939	0.9871	44.8 pairs/s
DevLake BERT	0.9896	0.9706	112.5 pairs/s
Medium	0.9633	0.9155	2,555.9 pairs/s
Mini	0.9590	0.9040	4,673.5 pairs/s
Nano	0.9498	0.8785	7,472.2 pairs/s

The compact scorers trade a small amount of discrimination for a large throughput gain. A practical cascade is nano/mini for the first pass, medium for uncertain pairs and LaBSE/LASER only for the final review layer.

Quality and Use

Scores are not calibrated human probabilities. The accepted, review and quarantine thresholds must be calibrated for a target corpus and a reviewed sample. The fast128 profile truncates long inputs; for long literary or document segments use a review layer with a longer-context scorer.

Limitations

• Alignment scores are model estimates, not human judgments.
• The benchmark is intended for comparison and does not guarantee production precision on every domain.
• fast128 truncates inputs beyond 128 tokens per side.
• Starting thresholds are heuristics and require per-corpus calibration.

Related Resources

• Bashkir-Russian Wikipedia Parallel Corpus — uses the medium model to produce its pair_score column, a concrete application of this scorer.

Usage

pip install onnxruntime sentencepiece numpy

import numpy as np
import onnxruntime as ort
import sentencepiece as spm

session = ort.InferenceSession(
    "models/mini/model.onnx",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
tokenizer = spm.SentencePieceProcessor(model_file="spm_bpe_16k.model")


def encode(ba, ru, max_len=128):
    limit = (max_len - 3) // 2
    ru_ids = tokenizer.encode(ru, out_type=int)[:limit]
    ba_ids = tokenizer.encode(ba, out_type=int)[:limit]
    ids = [1, *ru_ids, 2, *ba_ids, 2]
    ids += [tokenizer.pad_id()] * (max_len - len(ids))
    ids = np.asarray([ids], dtype=np.int64)
    return {"input_ids": ids, "attention_mask": ids != tokenizer.pad_id()}


logit = session.run(["score_logit"], encode("Сәләм донъя", "Привет мир"))[0][0]
score = float(1 / (1 + np.exp(-logit)))
print(score)

For corpus-scale jobs, use the direct batch runner to avoid HTTP/JSON overhead and stream Parquet, CSV/TSV or JSONL inputs.

License

Distributed under the Apache-2.0 license. This release contains derived alignment-scoring software and model assets, not source corpus text, scans or original page layouts.

Citation

@software{failed09_bashkir_pair_scorer_2026,
  title = {Bashkir-Russian Pair Scorer},
  author = {failed09},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/failed09/bashkir-pair-scorer},
  note = {Open-source compact alignment models for the Bashkir language}
}

Open Bashkir Data and Sources 🐝

This release is part of an open-source effort to support the development, preservation and practical use of the Bashkir language. Other related models, datasets and tools are available on the author's Hugging Face profile.

The author does not claim ownership or authorship of the source texts or other materials used to derive this release; rights and licensing remain with the original authors, publishers and dataset providers. Source texts are not redistributed in this repository, so users should follow the licenses and attribution requirements of the relevant upstream resources.