Error during training with LoRA & Sentence Transformers
#33
by cc4718 - opened
trainer.train() throws the following error:NVEmbedModel.forward() got an unexpected keyword argument 'inputs_embeds'
The error appears when using LoRA.
model = SentenceTransformer(model_id, device=device, trust_remote_code=True, model_kwargs=kwargs)
peft_config = LoraConfig(
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
task_type=TaskType.FEATURE_EXTRACTION,
inference_mode=False,
r=8,
lora_alpha=32,
lora_dropout=0.1,
)
model._modules["0"].auto_model = get_peft_model(
model._modules["0"].auto_model, peft_config
)
sentence-transformers==3.1.1
transformers==4.45.2
cc4718 changed discussion title from Error during training with Sentence Transformers to Error during training with LoRA & Sentence Transformers