For those of you who want to test larger MoE models on Macs with lower configs

I made a PR that introduces MoE streaming to oMLX. I allows you to run Deepseek, GLM 5.3 and Qwen 3.8 Flash Next using a lower memory footprint. Of course it's a lot slower than having the whole model loaded in memory.

Here are the results on my M4 Pro 48GB

|model|RAM after load|session peak|TTFT, short prompt|decode| |:-|:-|:-|:-|:-| |GLM-5.3-Flash-JANG-MTP|10.52 GiB|14.68 GiB|12.36 s|**1.82 tok/s**| |Qwen3.8-JANG 4S|7.04 GiB|11.19 GiB|8.56 s|**3.63 tok/s**| |Qwen3.8-JANG 4M|7.05 GiB|11.11 GiB|11.23 s|**3.19 tok/s**| |DeepSeek-V4-Flash-0731-JANG|8.25 GiB|16.87 GiB|6.52 s|**2.71 tok/s**|

[https://github.com/jundot/omlx/pull/3468](https://github.com/jundot/omlx/pull/3468)