I am asking about local AI a functional home tool that won't call Jeff Bezos if I shop anywhere but Amazon. Not looking for a unit to crack quantum physics, or the meaning of life. I want something I can train on my own data to answer questions based on my home needs like document search, image manipulation, etc. Any chance I will be able to run it based in RAM and not have to have a $2000.00 vid card just to say hello?
![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Navigation
Install the app
How to install the app on iOS
Follow along with the video below to see how to install our site as a web app on your home screen.
Note: This feature may not be available in some browsers.
More options
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Where are we on local AI ?
- Thread starter narsbars
- Start date
We have a relatively new sub forum here:
https://hardforum.com/forums/artificial-intelligence-general-discussion.137/
Kyle is leading a lot of discussion there. That being said, local ai is pretty capable and it sounds like your needs are fairly basic. What are you system specs? You can run different guis to try different ai models, like ollama, lmstudio, lemonade, or unsloth desktop. I'm personally running unsloth desktop which has different sections for text, image or video gen, etc. Lemonade has "omni" models which basically merge several models into one chat session, so you can get image edit /generation functions and chat bot functions in 1 window, but the more models you fire up at once, the bigger ram hit.
An aside, I wouldn't mind seeing hardocp type blogs for local ai stuff, how to guides, deep dives, etc. So many people are anti ai, but showcasing utility and usefulness on the local level might help ease people into it, and while everything is under the "ai" umbrella, there's a huge difference between local ai and mega corporation greed world domination ai, both in practice and implication.
https://hardforum.com/forums/artificial-intelligence-general-discussion.137/
Kyle is leading a lot of discussion there. That being said, local ai is pretty capable and it sounds like your needs are fairly basic. What are you system specs? You can run different guis to try different ai models, like ollama, lmstudio, lemonade, or unsloth desktop. I'm personally running unsloth desktop which has different sections for text, image or video gen, etc. Lemonade has "omni" models which basically merge several models into one chat session, so you can get image edit /generation functions and chat bot functions in 1 window, but the more models you fire up at once, the bigger ram hit.
An aside, I wouldn't mind seeing hardocp type blogs for local ai stuff, how to guides, deep dives, etc. So many people are anti ai, but showcasing utility and usefulness on the local level might help ease people into it, and while everything is under the "ai" umbrella, there's a huge difference between local ai and mega corporation greed world domination ai, both in practice and implication.
X570, 5800X, 64 gig DDR 4 3600, 3070 vid card. Thanks.We have a relatively new sub forum here:
https://hardforum.com/forums/artificial-intelligence-general-discussion.137/
Kyle is leading a lot of discussion there. That being said, local ai is pretty capable and it sounds like your needs are fairly basic. What are you system specs? You can run different guis to try different ai models, like ollama, lmstudio, lemonade, or unsloth desktop. I'm personally running unsloth desktop which has different sections for text, image or video gen, etc. Lemonade has "omni" models which basically merge several models into one chat session, so you can get image edit /generation functions and chat bot functions in 1 window, but the more models you fire up at once, the bigger ram hit.
An aside, I wouldn't mind seeing hardocp type blogs for local ai stuff, how to guides, deep dives, etc. So many people are anti ai, but showcasing utility and usefulness on the local level might help ease people into it, and while everything is under the "ai" umbrella, there's a huge difference between local ai and mega corporation greed world domination ai, both in practice and implication.
I just tried it for development on my AMD CPU 7950x, 48 gigs of RAM and 16 gb 7900GRE video card. I did ollama, and OpenCode I think it was called (open source equiv of claude code)... I will say I didn't have enough horsepower. IN fact, it uses all my system resources so hard that my computer crashed hard to the point I had to unplug it from the wall in order to get it to recover. So I ended up subscribing to Claude Code instead.