![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Didn't know where to put this post, but it started with wanting to test drive some smaller models before shelling out for my own hardware so I'll put it here...
Stumbled on Qwen 3.8 27b being served on Cerebras hardware, and wow it is FAST. Got 500tk/s in my testing via a 3rd party reseller, Cerebras themselves claim 1,500 - 2,500 depending on model. The responses are instant. It is definitely a novel feeling.
Even if it takes a few years for this tech to trickle down to the consumer, keep in mind things will be changing soon as hardware starts catching up. We might soon be reminiscing about the old days running a gaming GPU and watching our queries type out line by line...
Stumbled on Qwen 3.8 27b being served on Cerebras hardware, and wow it is FAST. Got 500tk/s in my testing via a 3rd party reseller, Cerebras themselves claim 1,500 - 2,500 depending on model. The responses are instant. It is definitely a novel feeling.
Even if it takes a few years for this tech to trickle down to the consumer, keep in mind things will be changing soon as hardware starts catching up. We might soon be reminiscing about the old days running a gaming GPU and watching our queries type out line by line...