k47-XL

A from-scratch Turkish RNN-T speech recognizer. Not a Whisper fine-tune — the encoder, tokenizer and predictor are all trained from random initialization on Turkish audio.

"k47" names the encoder's depthwise convolution kernel size (47).

Architecture

ConformerV2 encoder + stateless predictor + RNN-T joiner, with an auxiliary CTC head (weight 0.3) during training.

Parameters	109.9M
Encoder	16 layers, dim 512, 8 heads, conv kernel 47, Macaron MatFormer FFN, Transformer-XL relative-position attention
Subsampling	4× (40 ms frames)
Tokenizer	SentencePiece, 4,096-token Turkish vocabulary (tokenizer/)
Decoding	RNN-T greedy (this checkpoint's reported numbers); beam + LODR external-LM fusion supported by the training repo

Model code is included under modeling/ (asr_model_v2.py, conformer_v2.py) so the checkpoint is loadable without depending on the private training repo. It is reused as-is, not repackaged into a transformers AutoModel class.

Training

Two-phase recipe on a from-scratch base, both phases on the same architecture:

1. Base pretraining (320k optimizer steps) on corpus_v2: 2,396.6 h / 1,527,976 clips, mixing four sources — a from-scratch Turkish aggregate (orhun_base, 1,356.6 h), Whisper-teacher pseudo-labels over a YODAS-TR harvest (yodas_king, 547.4 h), an in-house WorldSpeech-TR parliament/broadcast aggregation (284.6 h), and ISSAI + a smaller held-out-adjacent set (issai_tsc, 208.0 h).
2. Core-anneal continuation (this checkpoint, +50k steps, lr 1.5e-5, cosine, tokenizer frozen) restricted to the orhun_base slice only (713,143 clips / 1,178 h) — annealing on the cleanest-provenance subset measurably improved out-of-distribution WER over the base checkpoint without new data.

Single seed, single run. No checkpoint averaging.

Evaluation

RNN-T greedy decode, trnorm text normalization applied to both reference and hypothesis on both sides of every comparison, corpus-level WER.

Test set	k47-XL (this checkpoint)	Whisper large-v3 (unmodified, faster-whisper beam-5)	lv3-KING (author's Whisper LoRA fine-tune, faster-whisper beam-5)
WorldSpeech-TR (n=250)	7.64%	8.97%	8.91%
FLEURS-TR (n=715)	17.27%	—	—

The WorldSpeech-TR comparison is the one meaningful head-to-head this checkpoint has against named competitors on an identical test set. Two things temper it:

• Decode paradigms differ. k47-XL decodes RNN-T greedy; the Whisper comparators decode beam-5 on a separate, mature inference stack (faster-whisper/CTranslate2). This is a structural difference between architectures, not a controllable variable — it is disclosed rather than equalized.
• Single-seed, no confidence intervals. These are point estimates from one training run. Bootstrap intervals for this comparison have not been computed.

FLEURS-TR has no same-set Whisper number recorded for this checkpoint; treat it as an in-family reference point (the base checkpoint before annealing scored 22.84% on the same set), not a competitive claim.

Training data — provenance and licensing

This is a research checkpoint. Its training mixture carries the same provenance caveats documented for the author's other released models, and they are disclosed rather than smoothed over:

• The orhun_base component descends from an earlier corpus whose largest single source (a community-aggregated "merge" of YouTube/news audio, roughly half of that corpus by hours) has unverified licensing. It measurably helps WER and is retained for research use; it is not a basis for a licensing claim.
• yodas_king is pseudo-labeled audio harvested from YODAS-TR. YODAS-derived data in this project is scoped to research and publication use, not commercial redistribution.
• The remaining sources (ISSAI, Common Voice-derived, parliamentary transcripts) carry clean, identifiable licenses.

Given this mixture, the released weights are for research and reproduction, not for commercial deployment, until the training set is re-audited on a license-clean subset. This mirrors the disclosure already made for RsGoksel/RsGoksel_ITU_Mainframe.

Related

• Sibling leaderboard: TurkSpeak Bench
• Whisper-family track (different architecture, different corpus): RsGoksel/RsGoksel_ITU_Mainframe

Loading

import torch, json
from safetensors.torch import load_file

state_dict = load_file("model.safetensors")
config = json.load(open("config.json"))
# construct ConformerV2 / TurkSpeakASRV2 from modeling/asr_model_v2.py with `config`,
# then model.load_state_dict(state_dict)