How AI will fail

[rinaldo00]

I thought that this was an interesting take from a non tech guy.

I hope that it is considered on topic if not then delete it.


View: https://youtu.be/YTLnnoZPALI

 

[LukeTbk]

Not sure how novel or special is take that local AI usage will eat distributed/cloud one.

As of now and what seem a long future, expensive dedicated large amount of silicon seem to be able to do it much faster and much lower cost than local DIY alternative and upcoming cerebras/groq type (SRAM cut compute-energy cost in exchange of large silicon usage) could accelerate that trend not reduce it.

Large data/cannot rely on the Internet edge ai (like cars, some robots) will continue to be popular, but my guess is that large computer will continue to be popular for inference (not just training) for a long time.

 

[Aurelius]

I'm prepared for AI to burst... but I can see the cloud being useful regardless of how good local models become. Even if you get Star Trek-level wisdom, why not use the cloud to amplify it? So long as there are guardrails on it, of course.

 

[rinaldo00]

But the question is will it be cheap enough and good enough for people to just stop paying for something better?

Early Windoze PC clones sucked compared to Apple computers but they were good enough and cheap enough to get huge market share.

 

[rinaldo00]

Not sure how novel or special is take that local AI usage will eat distributed/cloud one.

As of now and what seem a long future, expensive dedicated large amount of silicon seem to be able to do it much faster and much lower cost than local DIY alternative and upcoming cerebras/groq type (SRAM cut compute-energy cost in exchange of large silicon usage) could accelerate that trend not reduce it.

Large data/cannot rely on the Internet edge ai (like cars, some robots) will continue to be popular, but my guess is that large computer will continue to be popular for inference (not just training) for a long time.

What I thought was different was that this came from someone not in the tech industry, meaning that the word of the way to separate yourself from the big models by doing it yourself is spreading to the masses.

 

[kram182]

But the question is will it be cheap enough and good enough for people to just stop paying for something better?

Early Windoze PC clones sucked compared to Apple computers but they were good enough and cheap enough to get huge market share.

The masses aren't going to set up local AI for themselves and companies will always want something more powerful than their competitors (which eventually without fail becomes needing more compute than you can house locally)

 

[LukeTbk]

But the question is will it be cheap enough and good enough for people to just stop paying for something better?

will it ever be cheaper to start with ? that one of the element, will people be ready to pay more to run AI locally instead of continu to use the cheaper cloud alternative.

specially low/moderate volume users, giant users now that math change, but are they in the good enough market ?

at $0.20 per millions tokens from cloud providers, electricity-amortisation-everything included, you need a giant amount of tokens at good electricity cost to beat that to pay for the edge device up front cost.

gemini flash type of model is hard to beat in good enough and cheaper to making your own setup.

 

[FrgMstr]

The masses aren't going to set up local AI for themselves and companies will always want something more powerful than their competitors (which eventually without fail becomes needing more compute than you can house locally)

Not unless you can just buy a device that deploys as easily as plugging it in and putting connecting it to you WiFi.

 

[kram182]

Not unless you can just buy a device that is deploys as easily as plugging it in and putting connecting it to you WiFi.

'Becky' and 'Jonathan' still aren't going to do that - just as for all the talk of 'DVDs and vinyl etc are making a comeback!' Netflix and Spotify still exist and aren't going anywhere

Edit: And again, for companies, that only works until your competitor has 2, so you buy 3, then they have 4, and then what I said about needing more compute than you can house eventually again always comes into play

 

[FrgMstr]

'Becky' and 'Jonathan' still aren't going to do that - just as for all the talk of 'DVDs and vinyl etc are making a comeback!' Netflix and Spotify still exist and aren't going anywhere

I remember when people said that people would never have their own personal computers in their house as well. Honestly, I do not understand the train of thought behind your argument.

 

[kram182]

I remember when people said that people would never have their own personal computers in their house as well. Honestly, I do not understand the train of thought behind your argument.

Did that kill datacenters? Geforce Now gaming still exists too.

Local does not negate cloud/datacenters nor, in this case, the more power/capability that comes with.

No matter how powerful or efficient or effective local gets - non-local will always have more and there will always be a customer for more.

 

[LukeTbk]

your phone will have good capacity in an ease to use package and they will sell your alexa type box with local capability would be my guess (you car will have it)

But cheaper altearnative from the cloud will probably stay the most popular (free with ads being one way we can see happen) and like now extremelly cheap api/monthly pro tier.

That one big difference, in it example something cheap you could buy an use replacing something expensive, an GPU will tend to be way more expensive to buy to use for AI than simply using cloud provider for most people.

What google let you do 100% free on gemini is already a lot, very few need more than that.

 

[FrgMstr]

Did that kill datacenters? Geforce gaming still exists too.

Local does not negate cloud/datacenters nor, in this case, the more power/capability that comes with.

No matter how powerful or efficient or effective local gets - non-local will always have more and there will always be a customer for more.

Once price and availability change for local LLM, there will be a huge shift. You will see a lot of corporations that are huge frontier AI customers now, they will begin moving away from frontier to bring AI into a much more private ecosystem.

 

[kram182]

Once price and availability change for local LLM, there will be a huge shift. You will see a lot of corporations that are huge frontier AI customers now, they will begin moving away from frontier to bring AI into a much more private ecosystem.

And by then there will be even more powerful things/models/etc enabled by infrastructure/size of cloud

Edit: And again, for companies, that only works until your competitor has 2, so you buy 3, then they have 4, and then what I said about needing more compute than you can house eventually again always comes into play

Tech does not stay stagnate, the goalposts/capabilities - whether that of local or non-local - are always shifting and moving further and further

 

[LukeTbk]

And by then there will be even more powerful things/models/etc enabled by infrastructure/size of cloud

but with custom silicon just for you (hard or soft baked) you can make it worth it, once you reach giant volume (and upfront money capacity) or have strict regularity goal we will see it.

The existance of both model, seem until a surprising and massive change a bit certain (and for both to be large).

 

[FrgMstr]

And by then there will be even more powerful things/models/etc enabled by infrastructure/size of cloud

I would suggest you are not seeing the true friction point to many companies and individuals when it comes to actually using AI, and that is privacy. Protecting your personal information, protecting your company's IP, etc.

I am not suggesting frontier is going away. I am suggesting there will be a big pull back from frontier. I said this a few months ago: "Once we get lightning fast inexpensive inference at home, on our PCs, our tablets, our phones, the hybrid computing experience leveraging Frontier and Local will be the norm. Dirt cheap tokens are the future. "

 

[1.1.2.3.5...]

But the question is will it be cheap enough and good enough for people to just stop paying for something better?

Early Windoze PC clones sucked compared to Apple computers but they were good enough and cheap enough to get huge market share.

I think yes. Mixture of experts models will let you run extremely powerful models locally with smaller vram by offloading a lot of the model to host ram. Basically then the MOE will load the expert models locally into vram and only those weights are active at any given time. So pcie 5.0 and dual memory controllers coming along will speed that up.

The new deepseek model is really good.

But the keys are always the tools you have around your local ai.

 

[kram182]

I would suggest you are not seeing the true friction point to many companies and individuals when it comes to actually using AI, and that is privacy. Protecting your personal information, protecting your company's IP, etc.

I am not suggesting frontier is going away. I am suggesting there will be a big pull back from frontier. I said this a few months ago: "Once we get lightning fast inexpensive inference at home, on our PCs, our tablets, our phones, the hybrid computing experience leveraging Frontier and Local will be the norm. Dirt cheap tokens are the future. "

I'm not saying there won't be anything on prem either just as on prem is an existing thing now before even your 'cheap AI revolution' - but the cloud AI and datacenters that provide them etc aren't going away at all was my point

Server/client mainframe/terminal local/non-local is all essentially the same and part of computing from basically the onset

 

[FrgMstr]

before even your 'cheap AI revolution'

People are working hard on exactly that.

 

[Randall Stephens]

So long as there are guardrails on it, of course.

el oh el

 

[M76]

The masses aren't going to set up local AI for themselves and companies will always want something more powerful than their competitors (which eventually without fail becomes needing more compute than you can house locally)

Most companies aren't training their own model. And inference can be run on everyday HW. I don't even think the bigger the model the better. Sometimes the "dumbed down" version can give better results. I've been experimenting with local LLMs and generative AI for about two years and they are evolving at a rapid pace.

 

[Aurelius]

el oh el

I think there will be guardrails... the question is how effective they'll be. I'm not expecting a Skynet-style apocalypse, but current AI shows how far we have to go (see: people learning how to do horrible things through ChatGPT).

 

[kram182]

People are working hard on exactly that.

I know they are. As I said above the goalposts/capabilities of both local and non-local and just tech in general are ever shifting/progressing. But as long as companies want to keep outcompeting their competitors, 'more' (as in more than can be powered/housed/processed/financed/etc locally) will always matter and be appealing to someone(s), no matter how much has shifted to local/on-prem and no matter for how cheap. 'Good enough' is only good enough and does not actually beat 'better'. Never has, never will.

A modern cheap smart phone has more compute than the Apollo missions, so technically would have been good enough and cheap enough for the Artemis mission. Yet we didn't launch Artemis on just a modern smart phone or for something just as cheap as, we launched it on something that has more compute than and cost a lot more than.

Most companies aren't training their own model. And inference can be run on everyday HW. I don't even think the bigger the model the better. Sometimes the "dumbed down" version can give better results. I've been experimenting with local LLMs and generative AI for about two years and they are evolving at a rapid pace.

But more and bigger models are always going to be coming out/developed/trained until AGI is (ever) achieved. 'But we can do inference locally' only also matters so long as you think it's going to become stagnate and plateau and no one is going to try to push it further, at the level your local inference HW is capable of. Inference will always need more/more powerful compute as well, even if not to the degree of training. Which even the latter part of your statement highlights.

And to counter the OPs/video's claims, the music industry is actually doing well, wasn't killed, and is currently setting record revenues - no matter how much production gear one can purchase on their own/'locally'/for cheaper or even if just now being produced 'on a simple desktop computer'.

 

[1.1.2.3.5...]

I think there will be guardrails... the question is how effective they'll be. I'm not expecting a Skynet-style apocalypse, but current AI shows how far we have to go (see: people learning how to do horrible things through ChatGPT).

How do you put guard rails on? You can effectively delete any safety put on any model, and huggingface is full of examples of this.

As this evolves, especially for local instances, there's not much you can do. I mean, you can make a law, but you can't enforce it without violating fourth amendment protections.

Cloud providers already have model safety.

Other countries we can't regulate.

 

[kram182]

How do you put guard rails on? You can effectively delete any safety put on any model, and huggingface is full of examples of this.

...

Cloud providers already have model safety.

And this is actually a good example of instances where 'local' can be better than 'cloud/non-local', the same as FrgMstr was highlighting with data/security concerns of companies, but still won't negate the need for non-local, nor the performance of when competing like-for-like

You're/companies are just going to have on-prem AI that does some AI/some things, locally.

Other countries we can't regulate.

And just as I mentioned with companies wanting to outcompete each other, we have to outcompete these other countries too.

 

[rinaldo00]

I know they are. As I said above the goalposts/capabilities of both local and non-local and just tech in general are ever shifting/progressing. But as long as companies want to keep outcompeting their competitors, 'more' (as in more than can be powered/housed/processed/financed/etc locally) will always matter and be appealing to someone(s), no matter how much has shifted to local/on-prem and no matter for how cheap. 'Good enough' is only good enough and does not actually beat 'better'. Never has, never will.

Here is the problem with 'more' Data centers as they are NOW are pushing the envelope of what is physically possible due to money, hardware, land space, energy, water, local infrastructure, etc. Communities are already pushing back. How much 'more' is possible?

 

[kram182]

Here is the problem with 'more' Data centers as they are NOW are pushing the envelope of what is physically possible due to money, hardware, land space, energy, water, local infrastructure, etc. Communities are already pushing back.

Hence why we're looking into nuclear power (again), space, etc

How much 'more' is possible?

Stay tuned and find out. I've always been amazed by tech to see and find out 'what's next'

 

[FrgMstr]

Here is the problem with 'more' Data centers as they are NOW are pushing the envelope of what is physically possible due to money, hardware, land space, energy, water, local infrastructure, etc. Communities are already pushing back. How much 'more' is possible?

Let's see how many planned actually get built as the landscape and ecosystem around AI changes very rapidly. Every day we are see huge leaps forward in software on the AI side. I would expect people are working on remedies to the current market constraints in hardware as well.

 

[DukenukemX]

Define failure because AI can fail, but also not fail? Running AI on you local computer doesn't mean AI failed. It just means AI as a service will fail.

 

[That_Sound_Guy]

Let's see how many planned actually get built as the landscape and ecosystem around AI changes very rapidly. Every day we are see huge leaps forward in software on the AI side. I would expect people are working on remedies to the current market constraints in hardware as well.

You are well aware of this of course: Software or code resources being fairly infinite is one thing. Hardware physical material restraint plus resource consumption is an entirely different scenario. With oil, power, water and land all nearing record cost levels, which dam will break first? I'm assuming the people dam will go first as they can't afford life anymore.

Double sword : If a company doesn't AI innovate, they will become irrelevant and die. If they do and let their employees go and ravage the local community to power AI, they lose customers and they die. Meanwhile while they battle for either outcome, they mass roll out what they think is the future in hopes it doesn't backfire on them. Thus the AI bubble and when it pops like the dot com era, the financial implications on the market will be a catastrophe.

 

[1.1.2.3.5...]

And this is actually a good example of instances where 'local' can be better than 'cloud/non-local', the same as FrgMstr was highlighting with data/security concerns of companies, but still won't negate the need for non-local, nor the performance of when competing like-for-like

You're/companies are just going to have on-prem AI that does some AI/some things, locally.



And just as I mentioned with companies wanting to outcompete each other, we have to outcompete these other countries too.

It was cheaper for our company to buy a cluster of h200s than to spend on cloud compute for the models we make. To avoid outing the company I work at I won't say more (but if someone really wanted know it's not too hard to infer).

 

[kram182]

If they do and let their employees go and ravage the local community to power AI, they lose customers and they die.

Will they? Nestle is still around and a multi-billion dollar company for all the horrible things we can list they did/still currently do.

 

[philb2]

The masses aren't going to set up local AI for themselves

Dell, Lenovo, HP, Microsoft, and Apple will surely try to develop "affordable" hardware for home and small business usage. Will any of them succeed, and when? That question is above my boss's pay grade.

 

[FrgMstr]

Others thinking along the path I having been talking about for months. Lots of news on this in the last 48 hours.

 

[kram182]

Dell, Lenovo, HP, Microsoft, and Apple will surely try to develop "affordable" hardware for home and small business usage. Will any of them succeed, and when? That questions is above my boss's pay grade.

Like I mentioned elsewhere, Netflix ain't going anywhere, even with local media available/possible, even for all the articles of 'DVDs making a comeback!' and for all the price increases Netflix imposes. This is the same thing IMO. You'll have some done and some who do local. But cloud-based ain't going anywhere cause there will be more than that who just don't/can't do local/can't do entirely local.

It was cheaper for our company to buy a cluster of h200s than to spend on cloud compute for the models we make. To avoid outing the company I work at I won't say more (but if someone really wanted know it's not too hard to infer).

Does your company still use/do/perform any function(s) that still rely on outside datacenters that still at the very least subsidizes the AI also done at those datacenters though, if not also using AI directly from them (just asking Gemini something in a browser for example)? And even if not let's say - surely you understand not everyone is in the position to do so, even for just obtaining a cluster of H200s.

 

[1.1.2.3.5...]

Does your company still use/do/perform any function(s) that still rely on outside datacenters that still at the very least subsidizes the AI also done at those datacenters though, if not also using AI directly from them (just asking Gemini something in a browser for example)? And even if not let's say - surely you understand not everyone is in the position to do so, even for just obtaining a cluster of H200s.

Those were bought for model training, which we do a lot of but I don't want to get into details for reasons. For AI tools we us as devs we use IT approved and blessed providers.

 

[kram182]

Those were bought for model training, which we do a lot of but I don't want to get into details for reasons. For AI tools we us as devs we use IT approved and blessed providers.

Right I'm just pointing out - even with your local instance/setup of AI you still at least subsidize to at most still use, cloud/outside/datacenter services/AI - meaning it's (non-local/cloud-based/datacenter-based AI) not going anywhere as the title/video/people here argue.

Like I've said it's not like I think local AI isn't going to exist (already does) or 'get killed' either - it's just the terminal/client/local equivalent to the mainframe/server/cloud model that's been a part of computing since its inception essentially.

 

[rinaldo00]

Hence why we're looking into nuclear power (again), space, etc

I am thinking people's patience will run out soon, everyone is already complaining about electricity rates.

 

[rinaldo00]

Right I'm just pointing out - even with your local instance/setup of AI you still at least subsidize to at most still use, cloud/outside/datacenter services/AI - meaning it's (non-local/cloud-based/datacenter-based AI) not going anywhere as the title/video/people here argue.

We have to decide what constitutes 'going away' They still make typewriters and buggy whips

 

[kram182]

I am thinking people's patience will run out soon, everyone is already complaining about electricity rates.

Gonna tongue-in-cheekily use AI to reply to this

Technology is like time - it marches forward and comes for one and all - whether they want it to or not.