LLM model fine-tuning pipeline

For (s)LLM model fine-tuning

Dataset

HuggingFace Korean dataset(preprocessed as instruction, input, and response)

Quick setup

# clone project
git clone https://github.com/DimensionSTP/llm-fine-tune.git
cd llm-fine-tune

# [OPTIONAL] create conda environment
conda create -n myenv python=3.12 -y
conda activate myenv

# install requirements
pip install -r requirements.txt

.env file setting

PROJECT_DIR={PROJECT_DIR}
CONNECTED_DIR={CONNECTED_DIR}
DEVICES={DEVICES}
HF_HOME={HF_HOME}
USER_NAME={USER_NAME}

Model Hyper-Parameters Tuning

• end-to-end

python main.py mode=tune is_tuned=untuned num_trials={num_trials}

Training

• end-to-end

python main.py mode=train is_tuned={tuned or untuned} num_trials={num_trials}

Test

• end-to-end

python main.py mode=test is_tuned={tuned or untuned} num_trials={num_trials} epoch={ckpt epoch}

Prediction

• end-to-end

python main.py mode=predict is_tuned={tuned or untuned} num_trials={num_trials} epoch={ckpt epoch}
python merge_predictions.py is_tuned={tuned or untuned} num_trials={num_trials} epoch={ckpt epoch}
python decode_predictions.py is_tuned={tuned or untuned} num_trials={num_trials} epoch={ckpt epoch}

Examples of shell scipts

• full preprocessing

bash scripts/preprocessing/preprocess.sh

• dataset preprocessing

bash scripts/preprocessing/preprocess_dataset.sh

• train

bash scripts/train/train.sh

• predict

bash scripts/test/predict.sh

Additional Options

• pure decoder based LLM QLoRA 4-bit quantization option

quantization_type={origin or quantization} 

• pure decoder based LLM LoRA or QLoRA PEFT option

peft_type={origin or lora}

• for LLM full fine-tuning(Continued Pretraining) in multi-GPU, recommended

strategy={deepspeed_stage_2 or deepspeed_stage_2_offload or deepspeed_stage_3 or deepspeed_stage_3_offload}

• for LLM DPO fine-tuning in multi-GPU, recommended

strategy={deepspeed_stage_2 or deepspeed_stage_2_offload}

• for LLM DPO fine-tuning in multi-GPU, avoid using the following strategies to prevent errors!

strategy={deepspeed_stage_3 or deepspeed_stage_3_offload or fsdp}

• upload user name and model name at HuggingFace Model card

upload_user={upload_user} 
model_type={model_type}

• Set data and target max length for model training and generation

data_max_length={data_max_length} 
target_max_length={target_max_length} 

If you want to change main config, use --config-name={config_name}.

Also, you can use --multirun option.

You can set additional arguments through the command line.