boost wrapped 多个 model #5595
-
|
假如有两个模型都比较大，每个模型单独退推理单卡都不够，训练的时候只微调其中一个，想问一下要如何使用boost函数wrappe了，是需要合并成一个大的模型吗？原始boost函数的模型输入是一个nn.Module类型，不支持模型列表传入 |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 4 replies
-
|
比如蒸馏模型这种情况用tensor并行要怎么设置呢 |
Beta Was this translation helpful? Give feedback.
-
|
I have the same question. Have you solved it? |
Beta Was this translation helpful? Give feedback.
-
|
no.. |
Beta Was this translation helpful? Give feedback.
-
|
Have you tried calling booster.boost twice or creating another booster? |
Beta Was this translation helpful? Give feedback.
-
|
为啥不支持一下list of nn.Module传入… |
Beta Was this translation helpful? Give feedback.
-
|
You can call boost() on multiple models, but returning a list of list (model, optimizer, criterion, dataloader, lr_scheduler) is convoluted logic and not elegant. |
Beta Was this translation helpful? Give feedback.
You can call boost() on multiple models, but returning a list of list (model, optimizer, criterion, dataloader, lr_scheduler) is convoluted logic and not elegant.