Muno459

/

fastconformer-quran-streaming

like 11

elhamdle Allah the greatest <3
Allah subhanu my god guided me to export the onnx here to a 80 mb model and baked the cmvn tlog with this script on google colab and the raw output was 100% similar to the 128 q8 model masha'a ALLAH <3 ELHAMDLE ALLAH THE GREATEST

but for noisy it is not good so i removed the script ....
but 128 q8 worked with the baked better elhamdle Allah

Wa alaykum as-salam, jazak Allah khayr for the pointer. A mixed-precision export (keeping the sensitive layers higher precision and quantizing the rest) is a solid way to get q8-level accuracy at a smaller size, and your own test confirms it: clean matched q8, noisy degraded. For now q8 stays the safe default because of that noisy-audio drop, but I will look at a mixed q4/q8 export for the next release. Barakallah feek for testing so carefully.

Thanks for Kind Words my brother <3
a small question can you export the streaming model to accept lower chunks size ? the 112 chunk size is not a productive ready....as someone can recite a word then stop in that case the asr doesn't even output it ...or even any words less than the 112 time it won't have an output unless i be in a continous reciting
small words like muqattat won't be output
words in surahs then silence won't be output
the smallest possible chunk or it is limited to 112 for a something ?

Wa ʿalaykum as-salām, akhi, and thank you for the kind words. Sharp question. 🤝

On the chunk size: the [70, 13] preset (about a 1.12s chunk, ~1040 ms lookahead) is set deliberately. I tested smaller windows, and below this the accuracy drops too much for Qur'an. The model needs that right context to get both the word and the tajwīd right, and shrinking the lookahead trades away far more quality than it gains. So [70, 13] stays as the recommended, accuracy-first preset rather than weakening recognition for everyone.

You have pointed at a real limitation of this streaming model though (short isolated utterances and the muqaṭṭaʿāt are not its strength), and I would rather solve it properly than cripple the accuracy.

You already know I have had a new model in training. This is the first time I am describing it properly, so here is what it actually is, and why it will handle this far better. It is a brand new Zipformer hybrid (RNN-T plus CTC), built from scratch:

• A genuine Arabic foundation model, pretrained on nearly 2,000 hours of broad Arabic (conversational, broadcast, dialectal, and read speech). For context, the current FastConformer was built on a roughly 1,150 hour Arabic base, so this is a much larger, from-scratch foundation, and a general purpose Arabic ASR base model in its own right.
• Then specialized for Qur'an on a large curated recitation set, with a Qur'an balanced tokenizer built for the orthography and vocabulary of the muṣḥaf.
• Streaming native and multi latency by design, trained across several lookahead profiles from the ground up, so low latency and short utterance cases (exactly what you are describing) are handled far more gracefully than any single fixed chunk model can.
• Built to run on phones. It is engineered for efficient on device, real time inference and exports to CoreML (iPhone Neural Engine) and ONNX, so it runs locally with low latency and no server round trip.

Zipformer is a newer, more efficient encoder than FastConformer, and building the Arabic base from scratch means the Qur'an fine tune stands on a much stronger, purpose built foundation. It is training now. Stay tuned, and jazāk Allāhu khayran for pushing on this. 🌙

ربنا تقبل منا انك انت السميع العليم
Our Lord Accept You Are the Hearing The Knowing

Thats Reeaaaally Awesome my brother <3 , Allah subhanu gave you alot of knowledge masha'a Allah <3

and yo uare using it in the sake of Allah Elhamdle Allah

Wa alaykum as-salam akhi 🤍

This is a good question and I want to separate it into two problems, because I think the one that is actually biting you is not the chunk size.

1. The real bug: you are missing a tail flush.

"a word then silence never comes out", and muqatta'at never appearing, is almost certainly not latency. A cache-aware model holds audio inside its right-context buffer. Whatever is still in that buffer when the audio stops never gets emitted, because nothing pushes it out.

I hit this exact bug myself and it cost me hours. My verification harness was scoring 1 correct segment out of 10 and I was convinced the ONNX export was broken. It was not. My loop was dropping the final partial chunk.

The fix: when your VAD sees end of speech (or the user presses stop), run one more chunk of zeros through the model, carrying the cache across as normal. Feed at least the right-context worth, 13 encoder frames at 80 ms each, so roughly 1.05 s of silence. The buffered word then comes out.

Do this first. I am fairly confident it alone fixes the short-word and muqatta'at cases.

2. The lookahead itself: yes, exportable, but read the caveat.

Lowering it is one line before export:

m.encoder.set_default_att_context_size([70, N])   # N < 13

Roughly: N=13 is about 1.04 s of lookahead, N=6 about 0.48 s, N=1 about 0.08 s, N=0 none.

But I checked the config of the weights this model comes from before answering you, and I want to be honest rather than just say yes:

att_context_size:  [-1, -1]
att_context_style: regular

That means it was trained with full context. The streaming export only clamps the attention window at inference time; the model never saw limited context while training. That is exactly why streaming already costs accuracy (about 4% WER offline against 12% streaming). If you shrink the lookahead further that gap widens, and at N=0 I expect it to degrade badly.

So by all means try N=6 and measure it on your own recitations, it may be a trade you are happy with. Just do not expect it to be free.

The proper fix is retraining with att_context_style: chunked_limited and a list of context sizes, so one set of weights serves several latencies and you pick per device. That is the route I am taking on the zipformer side.

Start with the tail flush though. I think that is your actual bug. 🤍