viet-legal-4B

A Vietnamese Legal SLM fine-tuned from Qwen/Qwen3-4B on the VLegal-Bench dataset using QLoRA.

Benchmark Results

Metric	Value
loss (epoch 1)	0.7023
runtime (epoch 1)	610.3857
samples_per_second (epoch 1)	2.0070
steps_per_second (epoch 1)	0.5030
loss (epoch 2)	0.6648
runtime (epoch 2)	609.0622
samples_per_second (epoch 2)	2.0110
steps_per_second (epoch 2)	0.5040

Training Details

Parameter	Value
Base model	Qwen/Qwen3-4B
Dataset	legal-combined (11025 train / 1225 eval)
LoRA rank	16
LoRA alpha	32
Learning rate	0.0002
Epochs	3
Batch size	1 x 8 (effective: 8)
Precision	fp16
Max seq length	1024
Training time	N/A
Best epoch	2
Hardware	Kaggle T4 (16GB VRAM)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "datht/viet-legal-4B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

messages = [
    {"role": "system", "content": "Bạn là một trợ lý pháp luật Việt Nam."},
    {"role": "user", "content": "Hành vi trộm cắp tài sản trị giá 5 triệu đồng bị xử lý như thế nào?"},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Dataset

VLegal-Bench — A cognitively grounded benchmark for Vietnamese legal reasoning, comprising 10,467 samples across 22 tasks organized into 5 categories:

1. Recognition & Recall (5 tasks) — Entity recognition, topic classification, concept recall
2. Understanding & Structuring (5 tasks) — Relation extraction, legal graph structuring
3. Reasoning & Inference (5 tasks) — Article prediction, court decision prediction
4. Interpretation & Generation (3 tasks) — Summarization, judicial reasoning
5. Ethics, Fairness & Bias (4 tasks) — Bias detection, privacy, ethical assessment

Training Framework

Trained with nlp-trainer using Unsloth + TRL SFTTrainer with QLoRA on Kaggle T4 GPU.

Citation

@misc{dong2025vlegalbench,
    title={VLegal-Bench: Cognitively Grounded Benchmark for Vietnamese Legal Reasoning of Large Language Models},
    author={Nguyen Tien Dong and others},
    year={2025},
    eprint={2512.14554},
    archivePrefix={arXiv},
}