What LLM/Model you using?

Jump on any major AI page and ask away at questions. I use Comet browser with perplexity for general free tier usage, about half my questions go to their built in chat vs basic web searches and I use their summarizer to summarize youtubes quickly before watching the video. If you want to tie an LLM into an agent that can do things with/for you, try something like Hermes Agent and a free tier model via Nous (which can added directly in settings>providers) to get you going.
 
antok86 said:
Where's a good starting point to lean what LLm is all about? I've got a bit of catching up to do.
Click to expand...
The nice thing about learning to use LLMs is you can just ask them. They know a lot about their own capabilities and limitations. I’m a few weeks into my journey of properly learning to use local models, having previously only dabbled, and I’ve surprised myself with how quickly I’m getting the hang of it. The biggest hurdle is just staring at that first chat console wondering “what do I even say to this thing?”

If you’re running local models on midrange gaming hardware (RX 9070 16GB, Ryzen 7700, 32 GB DDR5 in my case), then Strata and Unsloth Desktop will perform miracles on command. I’m not the best person to ask about the cloud-based AI that most people use, as I just don’t feel comfortable about handing over all my thoughts and data to the Zucks of the world.

So far, local LLMs have made a huge difference in my daily productivity. It’s good enough at summarizing web searches that I don’t bother to fight the algos anymore to get the info I need, I just ask something like “what is the community sentiment on places like Reddit and Github around [product x]? Do users feel it lives up to its marketing claims?” Then I would prompt it to put emphasis on comments that don’t use marketing speak, aren’t low-effort, don’t use similar words and phrases that the company’s website uses (the bot actually figured that one out on its own), etc. The output of a good model like Qwen 3.8 Flash-Next saves me a lot of time and mental effort, even when I don’t agree with its conclusions. You can ask it to summarize community sentiment around a product, criticisms of it, and figure out quickly what part of the praises and complaints applies to your use case and why because Qwen shows its work. If it’s a complex question or task, I just walk away and go do something else for a while. It’s already made a big difference in my life, and I’ve only just started.

I’m not much of a coder or hard-core software tinkerer, but I’ve managed to get it to write a script to automatically change my Mac’s resolution to 1080P when the TV connects via Sunshine + Artemis and then switch back to ultra-wide on disconnect. I’ve had it generate custom telemetry / ad / tracking blocklists for devices on my network that I don’t trust. I’ve had it figure out why software installs fail, or how to install tools that allow me to code and don’t phone home using Open WebUI. It’s not perfect, but it enables me to do things I wouldn’t have even tried before. My bigger “someday” projects feel very possible now, even if the jury’s out on whether the current tech and my efforts will be enough to get them to the finish line. I haven’t properly explored the reverse-engineering and emerging software-ownership possibilities yet, but I’m already planning how to get my old printer to print from my Mac and my phones, how to rip out the spyware from my smart TVs and devices, etc.

If I can do all of that with local models on modest hardware after a couple of weeks, then apply a multiplier for people who actually know what they are doing and can access frontier models or beefy hardware. I don’t want to get utopian (and there’s certainly a dystopian counter-argument), but I can easily see this technology creating a huge swell in entrepreneurship and cottage industries as motivated people rip and replace enshittified corporate moats with purpose-built tools. That vision may not be with us just yet (case in point, the sentiment around the Adobe clones from people who have actually tried to use them is dismal), but the genie is out of the bottle, and there’s no putting it back.
 
You must log in or register to reply here.
Back
Top