| --- |
| license: llama3.2 |
| datasets: |
| - CarrotAI/Magpie-Ko-Pro-AIR |
| - CarrotAI/Carrot |
| - CarrotAI/ko-instruction-dataset |
| language: |
| - ko |
| - en |
| base_model: |
| - meta-llama/Llama-3.2-3B-Instruct |
| pipeline_tag: text-generation |
| new_version: CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412 |
| --- |
| |
|  |
|
|
| ## Model Description |
|
|
| ### Model Details |
| - **Name**: Carrot Llama-3.2 Rabbit Ko |
| - **Version**: 3B Instruct |
| - **Base Model**: CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct |
| - **Languages**: Korean, English |
| - **Model Type**: Large Language Model (Instruction-tuned) |
|
|
| ### Training Process |
| ๋ณธ ๋ชจ๋ธ์ ๋ค์๊ณผ ๊ฐ์ ์ฃผ์ ํ๋ จ ๋จ๊ณ๋ฅผ ๊ฑฐ์ณค์ต๋๋ค: |
|
|
| 1. **SFT (Supervised Fine-Tuning)** |
| - ๊ณ ํ์ง ํ๊ตญ์ด ๋ฐ ์์ด ๋ฐ์ดํฐ์
์ ์ฌ์ฉํ์ฌ ๊ธฐ๋ณธ ๋ชจ๋ธ์ ์ธ๋ถ ์กฐ์ |
|
|
| ### Limitations |
| - 3B ํ๋ผ๋ฏธํฐ ๊ท๋ชจ๋ก ์ธํ ๋ณต์กํ ์์
์์์ ์ ํ์ ์ฑ๋ฅ |
| - ํน์ ๋๋ฉ์ธ์ ๋ํ ๊น์ด ์๋ ์ ๋ฌธ์ฑ ๋ถ์กฑ |
| - ํธํฅ์ฑ ๋ฐ ํ๊ฐ ๊ฐ๋ฅ์ฑ |
|
|
| ### Ethics Statement |
| ๋ชจ๋ธ ๊ฐ๋ฐ ๊ณผ์ ์์ ์ค๋ฆฌ์ ๊ณ ๋ ค์ฌํญ์ ์ต๋ํ ๋ฐ์ํ์์ผ๋, ์ฌ์ฉ์๋ ํญ์ ๊ฒฐ๊ณผ๋ฅผ ๋นํ์ ์ผ๋ก ๊ฒํ ํด์ผ ํฉ๋๋ค. |
|
|
| ### How to Use |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct") |
| tokenizer = AutoTokenizer.from_pretrained("CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct") |
| ``` |
|
|
| ## Score |
|
|
| |Tasks|Version| Filter |n-shot| Metric | |Value | |Stderr| |
| |-----|------:|----------------|-----:|-----------|---|-----:|---|-----:| |
| |gsm8k| 3|flexible-extract| 5|exact_match|โ |0.6490|ยฑ |0.0131| |
| | | |strict-match | 5|exact_match|โ |0.0023|ยฑ |0.0013| |
| |gsm8k-ko| 3|flexible-extract| 5|exact_match|โ |0.3275|ยฑ |0.0134| |
| | | |strict-match | 5|exact_match|โ |0.2737|ยฑ |0.0134| |
| |ifeval| 4|none | 5|inst_level_loose_acc |โ |0.8058|ยฑ | N/A| |
| | | |none | 5|inst_level_strict_acc |โ |0.7686|ยฑ | N/A| |
| | | |none | 5|prompt_level_loose_acc |โ |0.7320|ยฑ |0.0191| |
| | | |none | 5|prompt_level_strict_acc|โ |0.6858|ยฑ |0.0200| |
|
|
| | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr| |
| |-------------------------------|------:|------|-----:|--------|---|-----:|---|-----:| |
| |haerae | 1|none | |acc |โ |0.4180|ยฑ |0.0148| |
| | | |none | |acc_norm|โ |0.4180|ยฑ |0.0148| |
| | - haerae_general_knowledge | 1|none | 5|acc |โ |0.3125|ยฑ |0.0350| |
| | | |none | 5|acc_norm|โ |0.3125|ยฑ |0.0350| |
| | - haerae_history | 1|none | 5|acc |โ |0.3404|ยฑ |0.0347| |
| | | |none | 5|acc_norm|โ |0.3404|ยฑ |0.0347| |
| | - haerae_loan_word | 1|none | 5|acc |โ |0.4083|ยฑ |0.0379| |
| | | |none | 5|acc_norm|โ |0.4083|ยฑ |0.0379| |
| | - haerae_rare_word | 1|none | 5|acc |โ |0.4815|ยฑ |0.0249| |
| | | |none | 5|acc_norm|โ |0.4815|ยฑ |0.0249| |
| | - haerae_standard_nomenclature| 1|none | 5|acc |โ |0.4771|ยฑ |0.0405| |
| | | |none | 5|acc_norm|โ |0.4771|ยฑ |0.0405| |
| |
| | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr| |
| |----------------|------:|------|-----:|--------|---|-----:|---|------| |
| |kobest_boolq | 1|none | 5|acc |โ |0.7664|ยฑ |0.0113| |
| | | |none | 5|f1 |โ |0.7662|ยฑ | N/A| |
| |kobest_copa | 1|none | 5|acc |โ |0.5620|ยฑ |0.0157| |
| | | |none | 5|f1 |โ |0.5612|ยฑ | N/A| |
| |kobest_hellaswag| 1|none | 5|acc |โ |0.3840|ยฑ |0.0218| |
| | | |none | 5|acc_norm|โ |0.4900|ยฑ |0.0224| |
| | | |none | 5|f1 |โ |0.3807|ยฑ | N/A| |
| |kobest_sentineg | 1|none | 5|acc |โ |0.5869|ยฑ |0.0247| |
| | | |none | 5|f1 |โ |0.5545|ยฑ | N/A| |
| |kobest_wic | 1|none | 5|acc |โ |0.4952|ยฑ |0.0141| |
| | | |none | 5|f1 |โ |0.4000|ยฑ | N/A| |
| |
