Code for training the SFT_MODEL=Eurus-2-7B-SFT

[RiverTre]

Hello, Thanks for the good job.

Could you release the code for training the sft start point model 'Eurus-2-7B-SFT'？ If that provided, it will be easier to reproduce the results of your work since settings and paramaters would be exactly the same.

Thanks!

[wanghanbinpanda]

Thanks for following! We use LLaMA-Factory for sft, you can refer to their repo. The config we use is as follows, we use 16 × A800 80G.

### model
model_name_or_path: Qwen/Qwen2.5-Math-7B-Instruct

### method
stage: sft
do_train: true
finetuning_type: full
deepspeed: examples/deepspeed/ds_z3_config.json

### dataset
dataset: <Process our sft data into the corresponding format>
### dataformat: https://github.com/hiyouga/LLaMA-Factory/tree/main/data#supervised-fine-tuning-dataset-1


template: qwen
cutoff_len: 6144
max_samples: 1000000
overwrite_cache: true
preprocessing_num_workers: 16

### output
output_dir: <your output dir>


logging_steps: 10
save_steps: 100
plot_loss: true
overwrite_output_dir: true

### train
per_device_train_batch_size: 4
gradient_accumulation_steps: 2
learning_rate: 1.0e-5
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000

### eval
val_size: 0.001
per_device_eval_batch_size: 1
eval_strategy: steps
eval_steps: 500

[RiverTre]

Thanks! It helps a lot. 😀

[RiverTre]

Hi friend,
I have replicate the SFT-model training with the hparams you provided.

The cmd is

torchrun --nproc_per_node=8     src/train.py     --model_name_or_path /models/Qwen2.5-Math-7B     --resume_from_checkpoint /eurus/checkpoint-9200/     --stage sft     --do_train     --finetuning_type full     --deepspeed examples/deepspeed/ds_z3_config.json     --dataset Eurus-2-SFT-Data     --template qwen     --cutoff_len 6144     --max_samples 1000000     --overwrite_cache     --preprocessing_num_workers 16     --output_dir /eurus     --logging_steps 10     --save_steps 100     --plot_loss     --overwrite_output_dir     --per_device_train_batch_size 2     --gradient_accumulation_steps 4     --learning_rate 1.0e-5     --num_train_epochs 3.0     --lr_scheduler_type cosine     --warmup_ratio 0.1     --bf16     --ddp_timeout 180000000     --val_size 0.001     --per_device_eval_batch_size 1     --eval_strategy steps     --eval_steps 500 

However, it seems that there is twice loss sharp down in the training loss png figure.
I assume that maybe it is because that the dataset is not shuffle. But it seems that By default, llamafactory will shuffle the datasets globally.

Have you seen the same plot in your training process? What is your insights about the reasons for the sharp down of the loss figures? @wanghanbinpanda Thanks!