TypeError: from_pretrained() missing 1 required positional argument: 'model_id'

[jayantkhannadocplix1]

Hello @hkonsti @justusmattern27

I've successfully fine-tuned the Llama2-13b-chat-hf model using Llamatune. The fine-tuning process went well, and I was able to fine-tune my model. However, when attempting to run the fine-tuned model using PEFT, I encountered the following error:

/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (2.0.4) or chardet (3.0.4) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
Traceback (most recent call last):
  File "gradio.py", line 7, in <module>
    peft_model = PeftModel.from_pretrained("my_finetunedmodel_path_on_huggingface", use_auth_token="xxxxx")
TypeError: from_pretrained() missing 1 required positional argument: 'model_id'

Here is my code that i used to run the model:

config = PeftConfig.from_pretrained("my_finetunedmodel_path_on_huggingface")
peft_model = PeftModel.from_pretrained("my_finetunedmodel_path_on_huggingface", use_auth_token="xxxxx">
lm_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-13b-chat-hf", use_auth_token="xxxxx")
# Define the function that combines the models
def generate_response(text):
    peft_output = peft_model.generate(text)
    peft_response = peft_output[0]['generated_text']

    lm_input = peft_response + " "
    lm_output = lm_model.generate(lm_input, max_length=50, num_return_sequences=1)
    lm_response = lm_output[0]['generated_text']

    return lm_response

It appears that I'm encountering an issue with the from_pretrained function in the PeftModel class, where it's expecting a 'model_id' argument. I'm uncertain about how to resolve this issue. Any insights or suggestions would be greatly appreciated.

[justusmattern27]

When using PeftModel.from_pretrained, you need to specify two things: the base model on which you want to put the adapters, as well as the weights of the adapters. In this case (assuming that you fine-tuned the model with lora), you are only specifying the adapter weights, not the base model. See here for the documentation: https://huggingface.co/docs/peft/package_reference/peft_model#peft.PeftModel.from_pretrained