How to Offload Specific number of MoE layers to the CPU Instead of All Layers?
As mentioned in the title, activating CPU-MOE load all layers to CPU, which is not ideal. I'd like to use my 2GPUs too.
It would really be helpful to just use a slider to help load a specific number of layers a la LM Studio where 0 means turning CPU-MOE off.
