Fine-tuned Weights Doesn't Work

[baotruyenthach]

Hi,

I am trying to reproduce some of the results from your paper using the provided fine-tuned model. I followed the instructions to download the weight diff from https://huggingface.co/epfml/landmark-attention-llama7b-wdiff, and then I recovered the fine-tuned weights.

However, when I ran the run_test.py script in llama_legacy, I couldn't reproduce your results. The base model achieves the correct accuracy, while the fine-tuned model has an accuracy close to zero. I suspect there might be an issue with the uploaded weight diff.

Could you please check the weight diff?

Thank you!