RedditPP Mascot

RedditPP

r/Oobabooga • … • u/Iory1998

How to Offload Specific number of MoE layers to the CPU Instead of All Layers?

As mentioned in the title, activating CPU-MOE load all layers to CPU, which is not ideal. I'd like to use my 2GPUs too.

It would really be helpful to just use a slider to help load a specific number of layers a la LM Studio where 0 means turning CPU-MOE off.

![img](001ei4gwx5tg1)

— 0