metadata
base_model:
- google/gemma-3-4b-pt
- google/gemma-3-4b-it
library_name: peft
tags:
- mergekit
- peft
google_gemma-3-4b-it-lora
This is a LoRA extracted from a language model. It was extracted using mergekit.
LoRA Details
This LoRA adapter was extracted from google/gemma-3-4b-it and uses google/gemma-3-4b-pt as a base.
Parameters
The following command was used to extract this LoRA adapter:
mergekit-extract-lora --model google/gemma-3-4b-it --base-model google/gemma-3-4b-pt --out-path ./google_gemma-3-4b-it-lora --lora-merge-dtype float32 --cuda --max-rank=64 --skip-undecomposable --include-regex model.language_model.layers.[\d]+.(mlp|cross_attn|self_attn).(up|down|gate|q|k|v|o)_proj --exclude-regex vision_tower|multi_modal_projector|lm_head|embed_tokens