Solid improvement moving to vLLM. Took about 50 minutes for hermes to setup WSL w/vLLM etc etc from scratch and start serving. I did not touch CLI once. LMstudio replaced.
| Metric | LM Studio (5090) | vLLM, NVFP4 + MTP | Gain |...
Started running it as soon as it dropped. Insane performance so far compared to 3.6. Using LM Studio, but atm in process of moving it to vllm with unsloth NVFP4, which should 1.5-2x improve performance.
Very accurate model, moving my primary...
Anyone trying Qwen3.8-27B? I am running it right now Q8_0 on a M4 Max Mac Studio with 128GB ram.
I sent this prompt:
"Write a powershell script that tries to stop a windows service, if it does not stop normally in 60 seconds force it to stop."...
Mac Studio M4 Max w/128GB here. I was going to go for one of the 395+ machines but this mac ended up being cheaper.
Currently playing with Laguna S 2.1 and Bonsai 27B but have run all sorts of models to try them out. GPT-OSS-120B, Qwen3.6...