swift训练

数据集进行一次监督微调。配置中使用了 LoRA 作为参数高效微调方法，设置了常见的学习率、梯度累积、日志记录和检查点保存等参数。：训练时每个进程的全局批大小为 1。训练、微调和推理大语言模型的工具。以上命令使用了两块 GPU，基于。模型，表示一个 7B 参数。训练的进程数，这里设置为。GPU 各分配一个进程。（一个高效的推理引擎）。预训练模型进行微调。

AI生成曾小健

1819人浏览 · 2024-11-25 20:59:13

AI生成曾小健 · 2024-11-25 20:59:13 发布

CUDA_VISIBLE_DEVICES=0,1 \
> NPROC_PER_NODE=2 \
> swift sft \
>     --model_id_or_path qwen/Qwen-7B-Chat \
>     --dataset AI-ModelScope/blossom-math-v2 \
>     --output_dir output \
> 

run sh: `/home/opc/anaconda3/envs/swfit/bin/python3.1 -m torch.distributed.run --nproc_per_node 2 /home/opc/anaconda3/envs/swfit/lib/python3.11/site-packages/swift/cli/sft.py --model_id_or_path qwen/Qwen-7B-Chat --dataset AI-ModelScope/blossom-math-v2 --output_dir output`
WARNING:__main__:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
[INFO:swift] Successfully registered `/home/opc/anaconda3/envs/swfit/lib/python3.11/site-packages/swift/llm/data/dataset_info.json`
[INFO:swift] No vLLM installed, if you are using vLLM, you will get `ImportError: cannot import name 'get_vllm_engine' from 'swift.llm'`
[INFO:swift] No LMDeploy installed, if you are using LMDeploy, you will get `ImportError: cannot import name 'prepare_lmdeploy_engine_template' from 'swift.llm'`
[INFO:swift] Start time of running main: 2024-11-23 10:44:06.868211
[INFO:swift] Setting args.model_type: qwen-7b-chat
[INFO:swift] Setting template_type: qwen
[INFO:swift] Setting args.lazy_tokenize: False
[INFO:swift] Setting args.dataloader_num_workers: 1
device_count: 2
rank: 1, local_rank: 1, world_size: 2, local_world_size: 2
[INFO:swift] output_dir: /home/opc/ms-swift/output/qwen-7b-chat/v0-20241123-104407
[INFO:swift] args: SftArguments(model_type='qwen-7b-chat', model_id_or_path='qwen/Qwen-7B-Chat', model_revision='master', full_determinism=False, sft_type='lora', freeze_parameters=[], freeze_vit=False, freeze_parameters_ratio=0.0, additional_trainable_parameters=[], tuner_backend='peft', template_type='qwen', output_dir='/home/opc/ms-swift/output/qwen-7b-chat/v0-20241123-104407', add_output_dir_suffix=True, ddp_backend='nccl', ddp_find_unused_parameters=None, ddp_broadcast_buffers=None, ddp_timeout=1800, seed=42, resume_from_checkpoint=None, resume_only_model=False, ignore_data_skip=False, dtype='bf16', packing=False, train_backend='transformers', tp=1, pp=1, min_lr=None, sequence_parallel=False, model_kwargs={}, loss_name=None, dataset=['AI-ModelScope/blossom-math-v2'], val_dataset=[], dataset_seed=42, dataset_test_ratio=0.01, use_loss_scale=False, loss_scale_config_path='/home/opc/anaconda3/envs/swfit/lib/python3.11/site-packages/swift/llm/agent/default_loss_scale_config.json', system=None, tools_prompt='react_en', max_length=2048, truncation_strategy='delete', check_dataset_strategy='none', streaming=False, streaming_val_size=0, streaming_buffer_size=16384, model_name=[None, None], model_author=[None, None], quant_method=None, quantization_bit=0, hqq_axis=0, hqq_dynamic_config_path=None, bnb_4bit_comp_dtype='bf16', bnb_4bit_quant_type='nf4', bnb_4bit_use_double_quant=True, bnb_4bit_quant_storage=None, rescale_image=-1, target_modules=['c_attn'], target_regex=None, modules_to_save=[], lora_rank=8, lora_alpha=32, lora_dropout=0.05, lora_bias_trainable='none', lora_dtype='AUTO', lora_lr_ratio=None, use_rslora=False, use_dora=False, init_lora_weights='true', fourier_n_frequency=2000, fourier_scaling=300.0, rope_scaling=None, boft_block_size=4, boft_block_num=0, boft_n_butterfly_factor=1, boft_dropout=0.0, vera_rank=256, vera_projection_prng_key=0, vera_dropout=0.0, vera_d_initial=0.1, adapter_act='gelu', adapter_length=128, use_galore=False, galore_target_modules=None, galore_rank=128, galore_update_proj_gap=50, galore_scale=1.0, galore_proj_type='std', galore_optim_per_parameter=False, galore_with_embedding=False, galore_quantization=False, galore_proj_quant=False, galore_proj_bits=4, galore_proj_group_size=256, galore_cos_threshold=0.4, galore_gamma_proj=2, galore_queue_size=5, adalora_target_r=8, adalora_init_r=12, adalora_tinit=0, adalora_tfinal=0, adalora_deltaT=1, adalora_beta1=0.85, adalora_beta2=0.85, adalora_orth_reg_weight=0.5, ia3_feedforward_modules=[], llamapro_num_new_blocks=4, llamapro_num_groups=None, neftune_noise_alpha=None, neftune_backend='transformers', lisa_activated_layers=0, lisa_step_interval=20, reft_layer_key=None, reft_layers=None, reft_rank=4, reft_intervention_type='LoreftIntervention', reft_args=None, use_liger=False, gradient_checkpointing=True, vit_use_gc=True, deepspeed=None, batch_size=1, eval_batch_size=1, auto_find_batch_size=False, num_train_epochs=1, max_steps=-1, optim='adamw_torch', adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, learning_rate=0.0001, weight_decay=0.1, gradient_accumulation_steps=8, max_grad_norm=1, predict_with_generate=False, lr_scheduler_type='cosine', lr_scheduler_kwargs={}, warmup_ratio=0.05, warmup_steps=0, eval_steps=50, save_steps=50, save_only_model=False, save_total_limit=2, logging_steps=5, acc_steps=1, dataloader_num_workers=1, dataloader_pin_memory=True, dataloader_drop_last=False, push_to_hub=False, hub_model_id=None, hub_token=None, hub_private_repo=False, hub_strategy='every_save', test_oom_error=False, disable_tqdm=False, lazy_tokenize=False, preprocess_num_proc=1, use_flash_attn=None, ignore_args_error=False, check_model_is_latest=True, logging_dir='/home/opc/ms-swift/output/qwen-7b-chat/v0-20241123-104407/runs', report_to=['tensorboard'], acc_strategy='token', save_on_each_node=False, evaluation_strategy='steps', save_strategy='steps', save_safetensors=True, gpu_memory_fraction=None, include_num_input_tokens_seen=False, local_repo_path=None, custom_register_path=None, custom_dataset_info=None, device_map_config=None, device_max_memory=[], max_new_tokens=2048, do_sample=None, temperature=None, top_k=None, top_p=None, repetition_penalty=None, num_beams=1, fsdp='', fsdp_config=None, sequence_parallel_size=1, model_layer_cls_name=None, metric_warmup_step=0, fsdp_num=1, per_device_train_batch_size=None, per_device_eval_batch_size=None, eval_strategy=None, self_cognition_sample=0, train_dataset_mix_ratio=0.0, train_dataset_mix_ds=['ms-bench'], train_dataset_sample=-1, val_dataset_sample=None, safe_serialization=None, only_save_model=None, neftune_alpha=None, deepspeed_config_path=None, model_cache_dir=None, lora_dropout_p=None, lora_target_modules=[], lora_target_regex=None, lora_modules_to_save=[], boft_target_modules=[], boft_modules_to_save=[], vera_target_modules=[], vera_modules_to_save=[], ia3_target_modules=[], ia3_modules_to_save=[], custom_train_dataset_path=[], custom_val_dataset_path=[], device_map_config_path=None, push_hub_strategy=None)
[INFO:swift] Global seed set to 42
device_count: 2
rank: 0, local_rank: 0, world_size: 2, local_world_size: 2
[INFO:swift] Downloading the model from ModelScope Hub, model_id: qwen/Qwen-7B-Chat
Downloading Model to directory: /home/opc/.cache/modelscope/hub/qwen/Qwen-7B-Chat
[WARNING:modelscope] Using branch: master as version is unstable, use with caution
Downloading [cache_autogptq_cuda_256.cpp]: 100%|██| 8.21k/8.21k [00:01<00:00, 7.04kB/s]
Downloading [cache_autogptq_cuda_kernel_256.cu]: 100%|█| 50.8k/50.8k [00:01<00:00, 35.8
Downloading [assets/code_interpreter_showcase_001.jpg]: 100%|█| 135k/135k [00:05<00:00,
Downloading [config.json]: 100%|████████████████████████| 911/911 [00:01<00:00, 803B/s]
Downloading [configuration.json]: 100%|██████████████| 77.0/77.0 [00:02<00:00, 32.8B/s]
Downloading [configuration_qwen.py]: 100%|████████| 2.29k/2.29k [00:01<00:00, 1.35kB/s]
Downloading [cpp_kernels.py]: 100%|███████████████| 1.88k/1.88k [00:01<00:00, 1.71kB/s]
Downloading [generation_config.json]: 100%|█████████████| 273/273 [00:01<00:00, 265B/s]
Downloading [LICENSE]: 100%|██████████████████████| 6.73k/6.73k [00:02<00:00, 2.46kB/s]
Downloading [assets/logo.jpg]: 100%|██████████████| 80.8k/80.8k [00:02<00:00, 29.1kB/s]
Downloading [model-00001-of-00008.safetensors]: 100%|█| 1.83G/1.83G [01:21<00:00, 24.1M
Downloading [model-00002-of-00008.safetensors]: 100%|█| 1.88G/1.88G [01:26<00:00, 23.3M
Downloading [model-00003-of-00008.safetensors]: 100%|█| 1.88G/1.88G [01:25<00:00, 23.7M
Downloading [model-00004-of-00008.safetensors]: 100%|█| 1.88G/1.88G [01:25<00:00, 23.8M
Downloading [model-00005-of-00008.safetensors]: 100%|█| 1.88G/1.88G [01:12<00:00, 28.1M

T-4o

Poe

以下是逐步解析每个训练参数及其含义的详解：

1. 环境变量

`CUDA_VISIBLE_DEVICES=0,1`

指定使用哪些 GPU 设备进行训练。
这里设置为 GPU 设备 0 和 1，表示启用两块 GPU。

`NPROC_PER_NODE=2`

每个节点上用于分布式训练的进程数，这里设置为 2。
每个进程通常对应一块 GPU，因此这里两块 GPU 各分配一个进程。

2. 命令行工具

`swift sft`

swift 是一个用于训练、微调和推理大语言模型的工具。
sft 表示 “Supervised Fine-Tuning”（监督微调），用于对预训练模型进行微调。

3. 命令行参数

`--model_id_or_path qwen/Qwen-7B-Chat`

指定要加载的预训练模型的标识符或路径。
这里使用的是 ModelScope 平台上的 Qwen-7B-Chat 模型，表示一个 7B 参数规模的对话模型。

`--dataset AI-ModelScope/blossom-math-v2`

指定用于微调的训练数据集。
数据集 blossom-math-v2 来源于 AI-ModelScope，可能包含高质量的数学相关任务数据。

`--output_dir output`

指定模型训练后的输出目录。
微调后的模型以及训练日志将保存到 output 文件夹中。

4. 执行命令

实际运行的命令是：

bash

复制

/home/opc/anaconda3/envs/swfit/bin/python3.1 -m torch.distributed.run --nproc_per_node 2 /home/opc/anaconda3/envs/swfit/lib/python3.11/site-packages/swift/cli/sft.py --model_id_or_path qwen/Qwen-7B-Chat --dataset AI-ModelScope/blossom-math-v2 --output_dir output