| --- |
| base_model: |
| - google/gemma-3-4b-pt |
| - google/gemma-3-4b-it |
| library_name: peft |
| tags: |
| - mergekit |
| - peft |
|
|
| --- |
| # google_gemma-3-4b-it-lora |
| |
| This is a LoRA extracted from a language model. It was extracted using [mergekit](https://github.com/arcee-ai/mergekit). |
| |
| ## LoRA Details |
| |
| This LoRA adapter was extracted from [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it) and uses [google/gemma-3-4b-pt](https://huggingface.co/google/gemma-3-4b-pt) as a base. |
| |
| ### Parameters |
| |
| The following command was used to extract this LoRA adapter: |
| |
| ```sh |
| mergekit-extract-lora --model google/gemma-3-4b-it --base-model google/gemma-3-4b-pt --out-path ./google_gemma-3-4b-it-lora --lora-merge-dtype float32 --cuda --max-rank=64 --skip-undecomposable --include-regex model.language_model.layers.[\d]+.(mlp|cross_attn|self_attn).(up|down|gate|q|k|v|o)_proj --exclude-regex vision_tower|multi_modal_projector|lm_head|embed_tokens |
| ``` |
| |
