• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ZLUDA Allows CUDA Binaries to Run on AMD GPUs

AMD and Intel don’t want more development on CUDA libraries.
There’s an absolute shitload of CUDA libraries out there to do virtually anything and over the decades they have been meticulously optimized and curated.
It’s to the point where CUDA can do some things 2-3x faster than their non CUDA counterparts, but it locks you to Nvidia. But it lets you get more done with less.

Projects like Zulda that work with native CUDA only spur more CUDA development, because why translate the library to anything else? But even with Zulda it still runs better on Nvidia than it does on AMD or Intel. It becomes an "I’ll use this card for now and when I can I’ll replace it with an Nvidia card later for max performance" situation.

That’s obviously something AMD and Intel don’t want.
So they develop tools that translate CUDA into something more optimized for their platforms, and that’s where the Intel and AMD tools differ, the resulting code does to some degree work on other platforms but performance varies.

So where CUDA is essentially a hard lock to Nvidia hardware, the Intel and AMD solutions are a soft lock. You could move to another platform, but you would need to evaluate and re-optimize your code libraries, that’s expensive and complex so why do it unless you really need to, but you could….
 
Last edited:
This reinforces that anything you can do in CUDA, you can do in one of the Open equivalents.
Somebody just needs to build those libraries, and I find it very funny that a LOT of effort is going into making those CUDA libraries work on non-NVIDIA GPUs, instead of just replacing them entirely.

All that does is reinforce the Nvidia moat, because the takeaway is "you can't replace those CUDA libraries." You can only work to make them work for you.

And Nvidia's big selling point for CUDA and their hardware is, "Why do that work when we've done it for you?"
 
Lakados said:
This reinforces that anything you can do in CUDA, you can do in one of the Open equivalents.
Click to expand...

There's two issues with that:

1) AMD has been hot garbage about supporting any sort of open compute shit. They don't have good tools, and half the time they don't even support things until way after nVidia does. OpenCL is a great example, OpenCL 2 was really problematic and OpenCL 3 fixed a lot of that. nVidia supported it almost right away, AMD only got to supporting it recently. Some software moved off OpenCL because of issues, particularly on AMD cards. Vegas Pro used to use OpenCL for acceleration, they switched to CUDA.

2) There's lots of CUDA code out there now, because it became the defacto way of doing things, that people don't wanna port.


I do with AMD was less shit about the GPGPU situation, but they have been for a long time, and nVidia is now REALLY entrenched.
 
Sycraft said:
There's two issues with that:

1) AMD has been hot garbage about supporting any sort of open compute shit. They don't have good tools, and half the time they don't even support things until way after nVidia does. OpenCL is a great example, OpenCL 2 was really problematic and OpenCL 3 fixed a lot of that. nVidia supported it almost right away, AMD only got to supporting it recently. Some software moved off OpenCL because of issues, particularly on AMD cards. Vegas Pro used to use OpenCL for acceleration, they switched to CUDA.

2) There's lots of CUDA code out there now, because it became the defacto way of doing things, that people don't wanna port.


I do with AMD was less shit about the GPGPU situation, but they have been for a long time, and nVidia is now REALLY entrenched.
Click to expand...
ROCm/HIP which ZLUDA is using is open source and OpenCL comes with ROCm. The more AMD improves ROCm, the more they improve OpenCL. AMD is still terrible at supporting any kind of compute, but this does give hope that Nvidia's CUDA advantage could come to an end.
 
DukenukemX said:
ROCm/HIP which ZLUDA is using is open source and OpenCL comes with ROCm. The more AMD improves ROCm, the more they improve OpenCL. AMD is still terrible at supporting any kind of compute, but this does give hope that Nvidia's CUDA advantage could come to an end.
Click to expand...
It's a hard road because nVidia has already built such an installed base and AMD has built a bad reputation. I'm hopeful too, but I'm not optimistic.
 
DukenukemX said:
ROCm/HIP which ZLUDA is using is open source and OpenCL comes with ROCm. The more AMD improves ROCm, the more they improve OpenCL. AMD is still terrible at supporting any kind of compute, but this does give hope that Nvidia's CUDA advantage could come to an end.
Click to expand...
I mean yes….
But the other side of it is, how bad is AMD’s library set if you can see significant performance increases translating Nvidia’s libraries to work on AMD rather than just using AMDs.

Sure in some cases it’s because there is no AMD equivalent library. But still it’s a sad state of things.

I really think that AMD and Intel need to burry their respective hatchets and work on a joint plan because unless something dramatically crashes in a spectacularly catastrophic way, I only see Nvidia’s moat growing at this rate and I don’t want that.

I really dislike Monopolies regardless of which company it is.
 
Lakados said:
I mean yes….
But the other side of it is, how bad is AMD’s library set if you can see significant performance increases translating Nvidia’s libraries to work on AMD rather than just using AMDs.

Sure in some cases it’s because there is no AMD equivalent library. But still it’s a sad state of things.

I really think that AMD and Intel need to burry their respective hatchets and work on a joint plan because unless something dramatically crashes in a spectacularly catastrophic way, I only see Nvidia’s moat growing at this rate and I don’t want that.

I really dislike Monopolies regardless of which company it is.
Click to expand...
couldn’t they use AI to go through their code base and optimize it?
 
I would think Cuda will eventually become irrelevant as AI is used internally, like in Google and Microsoft to write TPU/TCU kernels internally. Cuda is just a method to hide the architecture behaviors behind an API/Programming model so Nvidia keeps an advantage or keep secrets. AI can generate all of the libraries, just they will need exact behaviors, port mapping, latencies, microcode and so on. AI can't do this now since the hardware is hidden to it. Nvidia hand toon, 20 years' worth of libraries would be replicated and probably improved upon if AI had access to the hardware models.
 
This thread sometimes mixes up source compatibility (you can recompile CUDA source code for AMD) and binary compatibility.
 
  • Like
Reactions: erek
like this
noko said:
I would think Cuda will eventually become irrelevant as AI is used internally, like in Google and Microsoft to write TPU/TCU kernels internally. Cuda is just a method to hide the architecture behaviors behind an API/Programming model so Nvidia keeps an advantage or keep secrets. AI can generate all of the libraries, just they will need exact behaviors, port mapping, latencies, microcode and so on. AI can't do this now since the hardware is hidden to it. Nvidia hand toon, 20 years' worth of libraries would be replicated and probably improved upon if AI had access to the hardware models.
Click to expand...
Nvidia does use AI to tune their own libraries too.
And they also patent formulas and algorithms associated with specific aspects.
Nvidia holds more than 600 patents for Ray Tracing alone, most of which are mathematical in nature, and not hand wavey concepts.
 
Lakados said:
Nvidia does use AI to tune their own libraries too.
And they also patent formulas and algorithms associated with specific aspects.
Nvidia holds more than 600 patents for Ray Tracing alone, most of which are mathematical in nature, and not hand wavey concepts.
Click to expand...
Yes, your right, one reason why Nvidia libraries can be 2x to 3x faster. AMD could do the same with ROCm, while AMD for some reason thinks it has to be totally open and not Open ROCm software with AMD AI optimized libraries for AMD hardware.
 
noko said:
Yes, you’re right, one reason why Nvidia libraries can be 2x to 3x faster. AMD could do the same with ROCm, while AMD for some reason thinks it has to be totally open and not Open ROCm software with AMD AI optimized libraries for AMD hardware.
Click to expand...
I really don’t understand AMD’s position on so many things….
I can’t tell if they just like to half ass everything or they have 2 major camps internally one who loves open source and another that hates it and they both need to fight on weekends for next weeks budget.
 
  • Like
Reactions: noko
like this
KazeoHin said:
I give this 30 days until AMD shuts it down.
Click to expand...
they never did for the way more relevant Linux version.... this is just letting people not having to use WSL on windows (and in very limited way, many of the most important libs do not work).

noko said:
AMD could do the same with ROCm, while AMD for some reason thinks it has to be totally open and not Open ROCm software with AMD AI optimized libraries for AMD hardware.
Click to expand...
From my limited understanding of that AMD side of things

ROCm is open source but incredibly optimised for AMD hardware (and vice-versa), open source does not have to be hardware agnostic.

ROCm does not run on Nvidia gpu at all for example and made with only AMD hardware in mind, it is open so people can see, compile, make change to make for AMD cards they have themselve, etc...

The library they write (rocBLAS) are also heavily optimised for the physical layout, cache, wavefront size of AMD hardware with complete disregard of other hardware.

They have a very limited list of target they support:
https://rocm.docs.amd.com/projects/HIP/en/latest/understand/compilers.html
The ROCm compilers only build the x86 and AMDGPU targets. Functionality and options that are relevant to other targets are not included within the ROCm compilers.

It rely a lot on:
https://en.wikipedia.org/wiki/AMDgpu_(Linux_kernel_module)

Which target only amd hardware, the compiler they make (amd clang, amd LLVM) are also for AMD target only.

AMD do optimise his libraries for AMD hardware a lot, Nvidia is just incredibly great at it and can afford the driver stack people and hardware people to sit in the same room for years before a new major generation come out and have been doing it at high level for more than a decade now.

Trying to compete head to head with anything Nvidia take seriously is one of the hardest thing to do in the world.
 
Last edited:
Can't enforce laws to other countries, at best, you set up agreements or threaten them one way or another. How much a country is going to enforce an international company's proprietary asset is also another matter.

As far as AMD not supporting it, is a mixed bag. I'm sure they don't want to make their graphics cards basically NVIDIA cards. "Hey, consumers, don't get AMD cards, we have nothing special but pricing. We're basically NVIDIA-lite!" However, I'm sure they do want some of the NVIDIA market share for AI and other markets. I would think AMD would want people saying, "Look, it's AMD, they have this that NVIDIA can't do!"
 
Marees said:
I believe AMD has focussed on inference as a strength to be developed. while nvidia is king of training they are kind of vulnerable to ATI/AMD always being on the more advanced process node with more advanced packaging capabilities

on the software side for inference AMD has made advancement with the SGLang package

https://x.com/SemiAnalysis_/status/2100027190205095977?s=20
Click to expand...
Nvidia are certainly way more vulnerable on the inference side of things... but still Rubin gap with Blackwell on inference is apparently giant:
https://newsletter.semianalysis.com/p/vera-rubin-nvl72-agentic-inference
Vera Rubin is a substantial improvement over GB300 in terms of performance per dollar. At 170 TPS, On Apples to Apples TRTLLM NVFP4 Dense, Vera Rubin NVL72 delivers ~67x the total throughput per TCO of GB300 Dynamo TRTLLM under the owning cost assumptions. Furthermore, at the part of the frontier where most providers would actually serve this model (60-100 TPS), Vera Rubin achieves between 1.4x and 3x the throughput per TCO compared to the latest and greatest GB300 TRTLLM configuration.

When you look at the same graph but for GB300 or Rubin it is not close
s%2F05f44b34-7b40-409d-ad6a-015be7cdadbe_2048x1336.jpg


It is still going back to Nvidia can have over 75% margin and its competition is still more expensive if they give it to you for free type of territory.

And during a generation, the stack get better every month and often end up twice as good than before launch, so that gap between Rubin and previous Nvidia gen should just grow massively between now and when it is mature around spring 2028.

And we will have to see what the Nvidia rack made for inference only that are coming will perform:
https://developer.nvidia.com/blog/h...interactivity-inference-on-nvidia-vera-rubin/

early, limited test are quite impressive, if it all fit in sram in that rack, many 1,000 of tokens per second even for relativily large context window should be possible.
 
Last edited:
El_Capitan said:
Can't enforce laws to other countries, at best, you set up agreements or threaten them one way or another. How much a country is going to enforce an international company's proprietary asset is also another matter.
Click to expand...
Maybe there is some gray area, but as of now ZLUDA and translation projects in general are 100% legal in the USA (that why a serious company like AMD started it originaly).

In the recen past Google copying all of oracal JAVA api structure was said to be fair use: https://www.supremecourt.gov/opinions/20pdf/18-956_d18f.pdf

If the project never used anything leaked, just the public API, it could pass and I imagine Nvidia do not want to bring anti-trust attention too much in this subject here to sue AMD/the volountary programmer still maintaining Zluda or its users.
 
You must log in or register to reply here.
Back
Top