w2v-bert-2.0-lg-mental-health-luganda-v1

This model is a fine-tuned version of sulaimank/w2v-bert-2.0-lg-CV-Fleurs-300 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.1481
• Wer: 0.2782
• Cer: 0.1182

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Cer	Validation Loss	Wer
0.6873	1.0	297	0.1263	0.8923	0.3042
0.5491	2.0	594	0.1203	0.8985	0.2858
0.4482	3.0	891	0.1203	0.9618	0.2818
0.3689	4.0	1188	0.1190	1.0299	0.2772
0.3137	5.0	1485	0.1139	1.1969	0.2654
0.2402	6.0	1782	0.1168	1.1816	0.2751
1.9391	7.0	2079	0.1144	1.3093	0.2743
0.1829	8.0	2376	0.1182	1.2839	0.2776
0.1448	9.0	2673	0.1158	1.4365	0.2716
0.1230	10.0	2970	0.1193	1.5307	0.2797
0.1089	11.0	3267	0.1189	1.5912	0.2799
0.0892	12.0	3564	1.6067	0.2728	0.1169
0.0742	13.0	3861	1.8053	0.2725	0.1153
0.0638	14.0	4158	1.8934	0.2781	0.1173
0.0463	15.0	4455	1.8062	0.2884	0.1208
0.0367	16.0	4752	2.0412	0.2796	0.1175
0.0290	17.0	5049	2.1093	0.2803	0.1179
0.0218	18.0	5346	2.1808	0.2772	0.1187
0.0206	19.0	5643	2.1250	0.2869	0.1191
0.0178	20.0	5940	2.1182	0.2713	0.1152
0.0140	21.0	6237	2.1481	0.2782	0.1182

Framework versions

• Transformers 5.3.0
• Pytorch 2.10.0+cu130
• Datasets 3.0.0
• Tokenizers 0.22.2