TenzinGayche/tibetan-whisper-small

Whisper Small - tibetan v1.0.0

This model is a fine-tuned version of openai/whisper-small on the stt-test-v1.0.0 dataset. It achieves the following results on the evaluation set:

• Loss: 0.1309
• Cer: 38.5374

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Cer	Validation Loss
0.5037	0.1249	1000	48.0593	0.2382
0.3617	0.2499	2000	0.1749	41.4889
0.3172	0.3748	3000	0.1483	40.1784
0.2864	0.4998	4000	0.1374	39.0413
0.2645	0.6247	5000	0.1309	38.5374

Framework versions

• Transformers 5.6.2
• Pytorch 2.11.0+cu126
• Datasets 4.8.4
• Tokenizers 0.22.2