gs-GreBerta

This model is a fine-tuned version of bowphs/GreBerta on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.3458
• Top1: 38.0
• Top5: 61.0
• Top10: 69.3333
• Top20: 72.3333
• Bertscore F1 Top1: 86.1297
• Bertscore F1 Top1 Mean: 86.1297
• Bertscore F1 Top5: 92.0578
• Bertscore F1 Top5 Mean: 81.3940
• Bertscore F1 Top10: 94.0554
• Bertscore F1 Top10 Mean: 80.0775
• Bertscore F1 Top20: 94.9076
• Bertscore F1 Top20 Mean: 78.9279
• Cos Sim Top1 Max: 71.9315
• Cos Sim Top1 Mean: 71.9315
• Cos Sim Top5 Max: 81.2391
• Cos Sim Top5 Mean: 64.2925
• Cos Sim Top10 Max: 83.9104
• Cos Sim Top10 Mean: 62.5278
• Cos Sim Top20 Max: 85.4491
• Cos Sim Top20 Mean: 61.1446
• Composite Score: 54.9657

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.22833541147989e-06
• train_batch_size: 128
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.1
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Top1	Top5	Top10	Top20	Bertscore F1 Top1	Bertscore F1 Top1 Mean	Bertscore F1 Top5	Bertscore F1 Top5 Mean	Bertscore F1 Top10	Bertscore F1 Top10 Mean	Bertscore F1 Top20	Bertscore F1 Top20 Mean	Cos Sim Top1 Max	Cos Sim Top1 Mean	Cos Sim Top5 Max	Cos Sim Top5 Mean	Cos Sim Top10 Max	Cos Sim Top10 Mean	Cos Sim Top20 Max	Cos Sim Top20 Mean	Composite Score
2.8080	1.0	441	2.5453	36.6667	61.0	66.6667	70.3333	85.8308	85.8308	92.1664	81.3928	93.7891	79.9716	94.7290	78.8533	71.0352	71.0352	80.7255	63.8912	83.5151	62.0287	85.1105	60.6243	53.8509
2.5855	2.0	882	2.3822	38.3333	61.0	68.6667	72.3333	86.0617	86.0617	92.0687	81.4034	93.9262	80.0525	94.9252	78.9209	71.6309	71.6309	81.2905	64.2855	83.8109	62.4660	85.3562	61.1656	54.9821
2.5899	3.0	1323	2.3488	38.0	61.0	69.3333	72.3333	86.1297	86.1297	92.0578	81.3940	94.0554	80.0775	94.9076	78.9279	71.9315	71.9315	81.2391	64.2925	83.9104	62.5278	85.4491	61.1446	54.9657

Framework versions

• Transformers 5.9.0
• Pytorch 2.12.0+cu126
• Datasets 3.6.0
• Tokenizers 0.22.2