AMD Publishes Initial openSIL Open-Source CPU Silicon Initialization Code

[erek]

Nice, at least giving the appearance of contributing to the community of sorts

“Today that very early proof-of-concept code for running on AMD EPYC Genoa processors has been posted! openSIL/openSIL on GitHub is where the code is now being developed in the open.

"The AMD open Silicon Initialization Library (openSIL) is a collection of C libraries which can be integrated into an x86 host firmware, by directly compiling source or by linking with static libraries.

AMD openSIL consists of three statically linked libraries; xSIM (x86 Silicon Initialization Libraries), xPRF (x86 Platform Reference Library), and xUSL (x86 Utilities & Services Library). These libraries can be statically linked to a host firmware during compile/link time.

Source for the libraries resides under xSIM, xPRF, and xUSL."

The code is still in its early phase and is MIT licensed. For now the initial motherboard target is the AMD Onyx CRB reference board for 4th Gen EPYC. Sadly, I don't have the AMD Onyx CRB but rather the 2P Titanite, but it's going to take some time anyhow before AMD openSIL will be a viable alternative with Coreboot to running the proprietary BIOS.

With 4th Gen EPYC is also the exciting point where AMD's reference boards now use OpenBMC rather than a proprietary BMC stack.”

Source: https://www.phoronix.com/news/AMD-openSIL-Published

 

[Lakados]

Yes….
AMD has next to no documentation on their AGESA builds and support for them is garbage and they have as many bugs as you would expect from an AMD driver team.
So let the OEM’s build their own and sit back and reap the rewards.

 

[erek]

Yes….
AMD has next to no documentation on their AGESA builds and support for them is garbage and they have as many bugs as you would expect from an AMD driver team.
So let the OEM’s build their own and sit back and reap the rewards.

Sorry a bit drunk rn but you glad they’re eventually transitioning away from AGESA?

AMD Plans to Transition from AGESA to Open Source OpenSIL Firmware by 2026 for Enhanced Security & Simplicity. During the OCP Regional Summit, AMD revealed an unexpected plan to shift from AGESA to the open-source openSIL firmware by 2026. This decision stems from concerns over AGESA's vulnerability to cyberattacks.May 8, 2023

https://www.guru3d.com › news-story

AMD Plans to Transition from AGESA to Open Source OpenSIL Firmware ...​

 

[DukenukemX]

Nice, at least giving the appearance of contributing to the community of sorts

How is this not contributing to the community? It's open source is it not?

 

[erek]

How is this not contributing to the community? It's open source is it not?

It’s contributing, apologies for the convolution or whatever you wanna cal it / the proper term

 

[Lakados]

Sorry a bit drunk rn but you glad they’re eventually transitioning away from AGESA?

AMD Plans to Transition from AGESA to Open Source OpenSIL Firmware by 2026 for Enhanced Security & Simplicity. During the OCP Regional Summit, AMD revealed an unexpected plan to shift from AGESA to the open-source openSIL firmware by 2026. This decision stems from concerns over AGESA's vulnerability to cyberattacks.May 8, 2023

https://www.guru3d.com › news-story

AMD Plans to Transition from AGESA to Open Source OpenSIL Firmware ...​

Yes, AGESA would be perfectly fine if AMD had kept up on documentation and support aspects for it but in the past 6 years they have let too much slip and they don't seem to be putting any effort into correcting their ways so between the way things are now or an opensource alternative created by the OEMs and SIs that have to deal with them I will take the Open version.

 

[Lakados]

How is this not contributing to the community? It's open source is it not?

It's open source, but AMD is notorious for creating open-source projects and then doing the absolute bare minimum when it comes to supporting or contributing to those projects.
So in this case AGESA is a known clusterfuck and AMD themselves do not know how much of it functions as they themselves do not have much of the documentation surrounding it.
Rather than document and bug fix their own BIOS code they are instead creating an open-source framework, which if history repeats they will do little more than add last-minute updates to for support for soon-to-be-release products that they haven't yet officially announced.
The bulk of the work will be left to the likes of MSI, Gigabyte, Acer, and Asus to code and document the project, and likely a number of major universities that use AMD hardware for their various research projects.

There is a big difference between creating an open-source project to be a contributor and releasing one to leach off the labor of the community who need the project to succeed because they have work to do.

 

[Gideon]

I would think they are doing this more for system integrators to make changes they feel are needed for custom builds needed by their customers or business like Amazon, Google and such. I don't see them doing this because they don't feel like dealing with the hardware they built, more for allowing others to have better access at the hardware level in the firmware. Being flexible is part of the reason why companies like to work with AMD over other vendors.

 

[cdabc123]

I feel non proprietary firmware and low level access posses a security risk.

Open source is not always ideal, if the engineering resourses behind cpus were public it would be a huge security issue.

 

[Lakados]

I would think they are doing this more for system integrators to make changes they feel are needed for custom builds needed by their customers or business like Amazon, Google and such. I don't see them doing this because they don't feel like dealing with the hardware they built, more for allowing others to have better access at the hardware level in the firmware. Being flexible is part of the reason why companies like to work with AMD over other vendors.

They could do all of that by cleaning up their AGESA implementation and documenting what already exists, something all those parties have been asking them to do for years.
To me this screams “fine you want it you do it, here!”

 

[Gideon]

They could do all of that by cleaning up their AGESA implementation and documenting what already exists, something all those parties have been asking them to do for years.
To me this screams “fine you want it you do it, here!”

I honestly don't see it that way, nor have I heard of this being a issue. Now if you got proof of these complaints from vendors and AMD blowing them off, well then thats a different story. I can only go by what my IT team has said about our AMD servers at work.

 

[schoolslave]

I feel non proprietary firmware and low level access posses a security risk.

Open source is not always ideal, if the engineering resourses behind cpus were public it would be a huge security issue.

Why? I hope you realize Spectre/Meltdown were most likely exploited years before public disclosure and that disclosure was only due to researchers (ie. young students like Google's Jann Horn) reverse engineering and then finding issues. Imagine if they had complete open access - skipping the tiresome RE process altogether and going straight to finding and patching security holes. The proprietary nature of CPU (micro)arch did not protect anything. In fact, proprietary/closed source low level blobs and firmware pose a greater security risk since nobody but the outsourced bumbling engineers who wrote it can perform any audits. Then you are beholden to them actually pushing out properly working and verified security patches. Good luck. I work at one of the tech companies whose hardware many people here run and let me tell you, the people who write this low level platform code and/or firmware are generally not what you'd consider top of their field. Often times this code is rushed out the door to meet arbitrary deadlines.

Consider also that IBM open sourced their entire embedded/low level initialization firmware with POWER9. POWER CPUs (well systems really since nobody but IBM and RaptorCS even manufacture compatible motherboards) are purchased and used by big banks, hospitals, governments etc.

What should make any firmware secure instead is signed images and trust-based verification based on sound cryptographic algorithms and attestation.

 

[Lakados]

Why? I hope you realize Spectre/Meltdown were most likely exploited years before public disclosure and that disclosure was only due to researchers (ie. young students like Google's Jann Horn) reverse engineering and then finding issues. Imagine if they had complete open access - skipping the tiresome RE process altogether and going straight to finding and patching security holes. The proprietary nature of CPU (micro)arch did not protect anything. In fact, proprietary/closed source low level blobs and firmware pose a greater security risk since nobody but the outsourced bumbling engineers who wrote it can perform any audits. Then you are beholden to them actually pushing out properly working and verified security patches. Good luck. I work at one of the tech companies whose hardware many people here run and let me tell you, the people who write this low level platform code and/or firmware are generally not what you'd consider top of their field. Often times this code is rushed out the door to meet arbitrary deadlines.

Consider also that IBM open sourced their entire embedded/low level initialization firmware with POWER9. POWER CPUs (well systems really since nobody but IBM and RaptorCS even manufacture compatible motherboards) are purchased and used by big banks, hospitals, governments etc.

What should make any firmware secure instead is signed images and trust-based verification based on sound cryptographic algorithms and attestation.

In theory this is true, but in practice? There are lots of large open-source projects out there that end up having massive security holes that go unnoticed for years before getting patched.
This will be better, more secure, and far more stable than AGESA, but it will do so in spite of AMD, not because of them.

 

[ElementDave]

Good. Hopefully OpenSIL proves useful to coreboot and other open-source boot projects in the future.

AMD's developer documentation of its processors was pretty good the one time that I needed to refer to it. I wasn't able to find guides for the current generation of processors, but fortunately the older documentation was still relevant to what I needed it for. AMD's documentation and support for its firmware and processors seemed to fall apart around the time that Zen was released.

Yes….
AMD has next to no documentation on their AGESA builds and support for them is garbage and they have as many bugs as you would expect from an AMD driver team.
So let the OEM’s build their own and sit back and reap the rewards.

It's open source, but AMD is notorious for creating open-source projects and then doing the absolute bare minimum when it comes to supporting or contributing to those projects.

So in this case AGESA is a known clusterfuck and AMD themselves do not know how much of it functions as they themselves do not have much of the documentation surrounding it.

AGESA may be a mess, but unless you have an NDA contract with AMD, you wouldn't even have access to the documentation you're referring to. That's how Google was able to develop coreboot ports for their Zen-based Chromebooks.

Rather than document and bug fix their own BIOS code they are instead creating an open-source framework, which if history repeats they will do little more than add last-minute updates to for support for soon-to-be-release products that they haven't yet officially announced.
The bulk of the work will be left to the likes of MSI, Gigabyte, Acer, and Asus to code and document the project, and likely a number of major universities that use AMD hardware for their various research projects.

There is a big difference between creating an open-source project to be a contributor and releasing one to leach off the labor of the community who need the project to succeed because they have work to do.

Most of these claims are overgeneralized and exaggerations, if not outright made up. It's not a bad idea to remain skeptical, but I don't understand the attitude here. I would expect "enthusiasts" to generally be in favor of open sourcing code for hardware.

 

[cdabc123]

Why? I hope you realize Spectre/Meltdown were most likely exploited years before public disclosure and that disclosure was only due to researchers (ie. young students like Google's Jann Horn) reverse engineering and then finding issues. Imagine if they had complete open access - skipping the tiresome RE process altogether and going straight to finding and patching security holes. The proprietary nature of CPU (micro)arch did not protect anything. In fact, proprietary/closed source low level blobs and firmware pose a greater security risk since nobody but the outsourced bumbling engineers who wrote it can perform any audits. Then you are beholden to them actually pushing out properly working and verified security patches. Good luck. I work at one of the tech companies whose hardware many people here run and let me tell you, the people who write this low level platform code and/or firmware are generally not what you'd consider top of their field. Often times this code is rushed out the door to meet arbitrary deadlines.

Consider also that IBM open sourced their entire embedded/low level initialization firmware with POWER9. POWER CPUs (well systems really since nobody but IBM and RaptorCS even manufacture compatible motherboards) are purchased and used by big banks, hospitals, governments etc.

What should make any firmware secure instead is signed images and trust-based verification based on sound cryptographic algorithms and attestation.

I feel Spectre/Meltdown would have been publicly known before had the propitiatory, convoluted arch and firmware been well known. The messy, performance based nature of x86 and its development structure created bugs in the device. Im not necessarily convinced a fully open, mathematically secure alternative exists for modern x86 archs.

I agree POWER is a good execution of this concept, and provides very cool devices that allow for control and sight of this level of processing. The design to define software that effectively utilizes very complex custom ASICs like a modern cpu is not a task to take lightly. It will directly impact performance and will be fairly ridged depending on the structure of the underlaying silicon. This can make "patches" far more complex or infeasible of a task. Power has the advantage of a relatively sound underlaying arch (built on more open concepts then x86).

With only a handful of cpu manufacturers, many being government sanctioned and altered in some ways. A single discovered exploit can be hugely detrimental to the security of devices. Likewise, perhaps all the major exploiters are so large that being propitiatory doesn't protect industry secrets at all.

I suppose AMDs move shows a promise to development confidence and good practices. It would be very cool to see an open source x86 base allow for similar low level control as a IBM POWER setup.

 

[RanceJustice]

It's open source, but AMD is notorious for creating open-source projects and then doing the absolute bare minimum when it comes to supporting or contributing to those projects.
So in this case AGESA is a known clusterfuck and AMD themselves do not know how much of it functions as they themselves do not have much of the documentation surrounding it.
Rather than document and bug fix their own BIOS code they are instead creating an open-source framework, which if history repeats they will do little more than add last-minute updates to for support for soon-to-be-release products that they haven't yet officially announced.
The bulk of the work will be left to the likes of MSI, Gigabyte, Acer, and Asus to code and document the project, and likely a number of major universities that use AMD hardware for their various research projects.

There is a big difference between creating an open-source project to be a contributor and releasing one to leach off the labor of the community who need the project to succeed because they have work to do.

Even if this is the case I'd still rather it see it open then proprietary. I'm not familiar with how some of their niche FOSS projects have developed (Ie I hear they do well, but it very well could be how you say where the community, including OEMs and the like, is doing the majority of the work to get to that positive final outcome) but do they not contribute significantly to their Linux GPU drivers? I do see what you mean sometimes that they often put it up to the community, but like with FreeSync / VESA ActiveSync this can still work out. Then again, there are sometimes that AMD drops the ball - I've written at length about the frustration with the lack of open, platform independent, software/firmware components to allow the Ryzen 79xx 3D Cache CPUs to have a good chance of scheduling and core pinning properly; even were it mostly up to the OEMs and community to support it, it would have been much better to have FOSS heuristic/benchmarking, profilers with manual setting capability and more, then leaving it up to Xbox Game Bar on Windows only to make a "IF GAME BAR ON, THEN CACHE CORES" switch.

In any case, this is good to see but I'd be interested to get them moving on consumer/enthusiast CPUs ASAP as opposed to just EPYC. There are a lot of devs who operate in a space that will affect both, and lots of user interest within FOSS adjacent communities (Linux, right to repair, privacy/security, anonymity, hactivist etc) that ,if well implemented would be a big reason to choose AMD. This also intersects with eliminating opaque "blackbox, yet required to run/init" functionality in the Intel ME / AMD PSP, Pluton and the like; this tech does have potential benefit to the users but when there are questions that it is or could be put to work on behalf of another entity besides its owner and remains proprietary and opaque, its hard to feel secure that its truly "your" device (yes, there are similar concerns with some Mobile / ARM chips etc). We've already seen that System76 and the like are using Coreboot and their own custom open firmware, so its growing and a company as big of AMD making libre solutions the default like OpenSIL / OpenBMC will add greatly to the momentum if done correctly.

 

[Lakados]

I honestly don't see it that way, nor have I heard of this being a issue. Now if you got proof of these complaints from vendors and AMD blowing them off, well then thats a different story. I can only go by what my IT team has said about our AMD servers at work.

OEM, vendors, and SI's have not been shy over the past 5 years about complaining about AMD's lack of documentation and support for the AGESA builds, nor have we been silent when we have to tiptoe through every BIOS update that updates its versions.
I run EPYC servers, and every critical security issue that has been fixed that includes a new AGESA version has also brought with it a completely new personality for those machines, be it fixes for USB ports that cause the network cards to hiccup, fixes for network cards that result in unstable power delivery and soft locks, or the myriad of issues the AM5 platforms AGESA updates are bringing.

Jim Keller built and designed AGESA, the story goes that only he really understood how and why it does much of what it does, and he left excellent documentation for it, that was lost some time after he left.
AMD also supposedly pays him some sort of fee for every device they ship that uses it, so that could also be a strong incentive to move away from it.

 

[Lakados]

Good. Hopefully OpenSIL proves useful to coreboot and other open-source boot projects in the future.

AMD's developer documentation of its processors was pretty good the one time that I needed to refer to it. I wasn't able to find guides for the current generation of processors, but fortunately the older documentation was still relevant to what I needed it for. AMD's documentation and support for its firmware and processors seemed to fall apart around the time that Zen was released.



AGESA may be a mess, but unless you have an NDA contract with AMD, you wouldn't even have access to the documentation you're referring to. That's how Google was able to develop coreboot ports for their Zen-based Chromebooks.

Most of these claims are overgeneralized and exaggerations, if not outright made up. It's not a bad idea to remain skeptical, but I don't understand the attitude here. I would expect "enthusiasts" to generally be in favor of open sourcing code for hardware.

I am in favor of anything that does away with AGESA, and Opensource away, because it can only get better.
Jim Keller developed the AGESA framework, only he understood how it all worked together, and after he left it was all downhill from there, and it was reported that AMD lost the documentation that he did provide on it a year or so after his departure. (AKA the Zen release)

And Google ended up resorting to a cleanroom reverse engineering of the bootloader for their Zen-based Chromebooks because AMD would not or could not provide them with the documentation to do it. Google did a big presentation on it for OSFC2020.

 

[Lakados]

Even if this is the case I'd still rather it see it open then proprietary. I'm not familiar with how some of their niche FOSS projects have developed (Ie I hear they do well, but it very well could be how you say where the community, including OEMs and the like, is doing the majority of the work to get to that positive final outcome) but do they not contribute significantly to their Linux GPU drivers? I do see what you mean sometimes that they often put it up to the community, but like with FreeSync / VESA ActiveSync this can still work out. Then again, there are sometimes that AMD drops the ball - I've written at length about the frustration with the lack of open, platform independent, software/firmware components to allow the Ryzen 79xx 3D Cache CPUs to have a good chance of scheduling and core pinning properly; even were it mostly up to the OEMs and community to support it, it would have been much better to have FOSS heuristic/benchmarking, profilers with manual setting capability and more, then leaving it up to Xbox Game Bar on Windows only to make a "IF GAME BAR ON, THEN CACHE CORES" switch.

In any case, this is good to see but I'd be interested to get them moving on consumer/enthusiast CPUs ASAP as opposed to just EPYC. There are a lot of devs who operate in a space that will affect both, and lots of user interest within FOSS adjacent communities (Linux, right to repair, privacy/security, anonymity, hactivist etc) that ,if well implemented would be a big reason to choose AMD. This also intersects with eliminating opaque "blackbox, yet required to run/init" functionality in the Intel ME / AMD PSP, Pluton and the like; this tech does have potential benefit to the users but when there are questions that it is or could be put to work on behalf of another entity besides its owner and remains proprietary and opaque, its hard to feel secure that its truly "your" device (yes, there are similar concerns with some Mobile / ARM chips etc). We've already seen that System76 and the like are using Coreboot and their own custom open firmware, so its growing and a company as big of AMD making libre solutions the default like OpenSIL / OpenBMC will add greatly to the momentum if done correctly.

I am not against this move at all, I welcome it, and I hope they have it rolled out and deployable for 2027 because that is when my VM stack gets replaced and I would hate to be buying new servers that are on the last AGESA BIOS build, and then getting the OpenSIL version as an afterthought because that is just a scenario I don't want to potentially deal with. If that is the case it would be a very strong motivator to either hold off on replacing them an extra year (which has its own problems) or evaluating how difficult it would be to migrate the VM's from an AMD platform to an Intel one. Most of the machines could be manually moved over, but I have a bunch that I know do not move that way because of how their licenses bind, and if I move them and it triggers their protections and they implode, can't even change the NIC brand let alone the MAC address on them without it shutting down, and as they don't support our version any more our only option is to pay to migrate it to their new platform or shut it down and they want a cool 1/4 mil for that job so...

 

[Gideon]

OEM, vendors, and SI's have not been shy over the past 5 years about complaining about AMD's lack of documentation and support for the AGESA builds, nor have we been silent when we have to tiptoe through every BIOS update that updates its versions.
I run EPYC servers, and every critical security issue that has been fixed that includes a new AGESA version has also brought with it a completely new personality for those machines, be it fixes for USB ports that cause the network cards to hiccup, fixes for network cards that result in unstable power delivery and soft locks, or the myriad of issues the AM5 platforms AGESA updates are bringing.

Jim Keller built and designed AGESA, the story goes that only he really understood how and why it does much of what it does, and he left excellent documentation for it, that was lost some time after he left.
AMD also supposedly pays him some sort of fee for every device they ship that uses it, so that could also be a strong incentive to move away from it.

AM5 and AM4 both had issues, but some of that comes at the cost of being bleeding edge and Zen was even a brand new architecture, so that came with higher chance of issues. I know my first gen Zen processor went through quite a few AGESA updates for various minor issues, but ram compatibility was the main issue. AMD addressed them pretty quick but I also expected some of these issues as well. Now my later gen board and 5800X3D on the other hand have caused me no issues at all and has been smooth. Bleeding edge can be painful at times, but some of the issues on AM5 have been caused by the motherboard companies pushing the limits as well.

We have not had much of any issues with our servers, but were at the Government and we move much slower in updating, so maybe that is the difference in experience. I just have not read of OEM's and SI's being upset over AGESA issues. Would figure that would be news worthy of mention in the tech sites. I just think it's a good think if AMD is willing to work more closely with other companies and will reserve judgement based on the results. Also if they are paying a fee to Keller then yeah that is a huge motivation to move away from it.

 

[ElementDave]

I am in favor of anything that does away with AGESA, and Opensource away, because it can only get better.

Not directly related to AGESA, but I noticed that the developer section of AMD's web site has been reorganized since my last visit, which was admittedly quite some time ago. Apparently the old BIOS and Kernel Developer’s Guide (BKDG) has been superseded by the Processor Programming Reference (PPR). The former were much better organized and comprehensive, and also accessible to a broader audience, though some of what's missing may be scattered among supplementary documents; I haven't taken the time to look. Note that I'm referring only to the publicly available documentation here.

I apologize for the tone of my earlier reply. It was a bit harsh and unfair, and (upon rereading it for context) did not read the way it was intended. The main point I took issue with was the following:

Jim Keller developed the AGESA framework, only he understood how it all worked together, and after he left it was all downhill from there, and it was reported that AMD lost the documentation that he did provide on it a year or so after his departure. (AKA the Zen release)

Jim Keller built and designed AGESA, the story goes that only he really understood how and why it does much of what it does, and he left excellent documentation for it, that was lost some time after he left.

I haven't heard that story before: the case of the "missing manual" — the lost documentation. That doesn't mean it isn't true, of course, but I was expecting a reference to support the claim. Or is that one of those things that "everyone" knows? You don't have to answer that; it doesn't matter anyway at this point. Perhaps one day a movie will be made about the incident. It sounds like a monumental blunder if true, to put it mildly.

In any case, this is good to see but I'd be interested to get them moving on consumer/enthusiast CPUs ASAP as opposed to just EPYC.

AMD has announced its intention to phase out AGESA entirely, with openSIL eventually replacing it on both client and server platforms. Unfortunately, according to the roadmap it isn't going to be released for production until 2026, and I'm not sure whether that figure includes both platforms or just server. In any case, it's difficult to get super excited about it at this point in time, but I'm pleased they're moving in that direction nonetheless. Maybe this will motivate Intel to release the source code of its FSPs. Or maybe not...

The coreboot project maintains links to external resources, some of which may be of interest to those curious about firmware or the boot process in general: https://doc.coreboot.org/external_docs.html
They're worth checking out regardless of one's interest in coreboot.

Edit: typo/thinko

 

[Lakados]

I haven't heard that story before: the case of the "missing manual" — the lost documentation. That doesn't mean it isn't true, of course, but I was expecting a reference to support the claim. Or is that one of those things that "everyone" knows? You don't have to answer that; it doesn't matter anyway at this point. Perhaps one day a movie will be made about the incident. It sounds like a monumental blunder if true, to put it mildly.

More one of those rumours that came about some time in 2018 when AIBs and OEMs were struggling with getting basic features stable on the platform and everyone was getting mad that each BIOS update fixed one thing but broke two more.
It matched up with many of the stories from “insiders” who were explaining why we don’t see AMD in more OEM Desktops and Laptops and how AMD just did not provide anywhere near the same level of documentation or support for integrating their tech into the systems like Intel does.
It was further spread when Google engineers claimed AMD wouldn’t provide them with the requested information so they could create their own boot-loaders for Chrome devices which lead to them reverse engineering them for Project-X and they speculated that AMD didn’t actually have the documents which matched up with the earlier rumours.

Now could it all be a chain of coincidence and AMD simply has an internal policy dictated why they can’t share that information. And the OEM integration stories could be just stories and just simple inaccurate answers for a more complicated reasoning.

But the stories do manage to follow pretty closely to reality, so while they may not be 100% accurate they are at least accurately adjacent to what ever is actually going on.

 

[ElementDave]

More one of those rumours that came about some time in 2018 when AIBs and OEMs were struggling with getting basic features stable on the platform and everyone was getting mad that each BIOS update fixed one thing but broke two more.
It matched up with many of the stories from “insiders” who were explaining why we don’t see AMD in more OEM Desktops and Laptops and how AMD just did not provide anywhere near the same level of documentation or support for integrating their tech into the systems like Intel does.
It was further spread when Google engineers claimed AMD wouldn’t provide them with the requested information so they could create their own boot-loaders for Chrome devices which lead to them reverse engineering them for Project-X and they speculated that AMD didn’t actually have the documents which matched up with the earlier rumours.

Now could it all be a chain of coincidence and AMD simply has an internal policy dictated why they can’t share that information. And the OEM integration stories could be just stories and just simple inaccurate answers for a more complicated reasoning.

But the stories do manage to follow pretty closely to reality, so while they may not be 100% accurate they are at least accurately adjacent to what ever is actually going on.

Ah, I see. That would explain why I couldn't find any individual articles or references related to the alleged missing documentation. I don't tend to pay much attention to tech rumors, but it sounds like they didn't have their shit together, whatever the reason.

Thanks for letting me know.

 

[schoolslave]

Ah, I see. That would explain why I couldn't find any individual articles or references related to the alleged missing documentation. I don't tend to pay much attention to tech rumors, but it sounds like they didn't have their shit together, whatever the reason.

Thanks for letting me know.

2018 matches up with the last time AMD released a BKDG (BIOS and Kernel Developer's Guide) for their platforms. The last one in 2018 covers Bulldozer iirc. If Lakados' rumors are true and nothing is properly documented internally that could explain why there's been no external BKDGs since.

FWIW, while I am pretty much an open-source zealot, I'm not sure OpenSIL will improve much of anything. Dumping 1000s of hardcoded, magic number register writes into a code repository is basically worthless compared to proper documentation (ie. a BKDG). This honestly just smells like the usual AMD marketers trying to capitalize on "open source" wording buzz. As Lakados also mentioned - AMD is generally absolutely terrible at interfacing with the wider open source community(ies). Source dumps are still a good first step I suppose but I'd prefer documentation.

 

[kac77]

It's open source, but AMD is notorious for creating open-source projects and then doing the absolute bare minimum when it comes to supporting or contributing to those projects.

Yes, in a world where people don't open source their crap at all. The entire reason Steam Deck even exists is because AMD was first to open source their drivers. You think they wouldn't have gone with nVidia if they could have? That's doubtful. AMD might not be able to give great documentation everywhere but it also outsources quite a bit more than most.

There is a big difference between creating an open-source project to be a contributor and releasing one to leach off the labor of the community who need the project to succeed because they have work to do.

This is also not even remotely correct. You think it's the "open source community" contributing to its microcode? That's really nuts. All of the compiler commits come from AMD not someone else. Why don't you take a test drive of the Nouveau drivers and get back to me on how those are?

 

[Deleted member 162929]

AMD views open source as "Do this for me for free because I tried and I can't", it's 'pawning off' for them.

 

[Lakados]

Yes, in a world where people don't open source their crap at all. The entire reason Steam Deck even exists is because AMD was first to open source their drivers. You think they wouldn't have gone with nVidia if they could have? That's doubtful. AMD might not be able to give great documentation everywhere but it also outsources quite a bit more than most.

This is also not even remotely correct. You think it's the "open source community" contributing to its microcode? That's really nuts. All of the compiler commits come from AMD not someone else. Why don't you take a test drive of the Nouveau drivers and get back to me on how those are?

Everything I can find says that AMD does not release an opensource Microcode, they are also quite famous for incredibly minor updates to their opensource platforms, the majority of those are usually just a few lines here or there to support unreleased products
Like this, which they submitted in Nov 2022.

And the entire reason the Steam Deck exists is because some people at Valve were tired of playing with their Emulators on a laptop.

 

[kac77]

Everything I can find says that AMD does not release an opensource Microcode, they are also quite famous for incredibly minor updates to their opensource platforms, the majority of those are usually just a few lines here or there to support unreleased products
Like this, which they submitted in Nov 2022.
View attachment 577475

Dude they have kernel commits pretty regularly. No one else creates the drivers for their products but them. If you want to say their enterprise software in Linux could use some work that's fine. But painting it as though everyone else is creating their drivers/platform support is utter nonsense.

And the entire reason the Steam Deck exists is because some people at Valve were tired of playing with their Emulators on a laptop.

Nope. The reason it exists is that one of the two GPU manufacturers had open source drivers that worked while the other one didn't. Who was that BTW?

 

[Lakados]

Dude they have kernel commits pretty regularly. No one else creates the drivers for their products but them. If you want to say their enterprise software in Linux could use some work that's fine. But painting it as though everyone else is creating their drivers/platform support is utter nonsense.

Nope. The reason it exists is that one of the two GPU manufacturers had open source drivers that worked while the other one didn't. Who was that BTW?

AMD was the only choice for the Steam Deck, drivers have nothing to do with it. Intel did not have the GPU capabilities and Nvidia does not have an x86 license and the cost associated with using an Intel/Nvidia combination are too high to make a viable product at that price point. And we can all agree that the performance of ARM and any x86 emulator/translator would be insufficient for the end goal. That leaves AMD as the only viable partner for the project.
Nvidia though would have likely turned down such a partnership request because the Nintendo contract is worth more to them than what Valve could have offered and you can bet that had Nvidia and Valve released the Deck together Nintendo would be all over that.

And I will admit that AMD’s recent string of hires for their opensource GPU drivers has had positive results. But that is one of many and generally the exception to the rule.

 

[Lakados]

2018 matches up with the last time AMD released a BKDG (BIOS and Kernel Developer's Guide) for their platforms. The last one in 2018 covers Bulldozer iirc. If Lakados' rumors are true and nothing is properly documented internally that could explain why there's been no external BKDGs since.

FWIW, while I am pretty much an open-source zealot, I'm not sure OpenSIL will improve much of anything. Dumping 1000s of hardcoded, magic number register writes into a code repository is basically worthless compared to proper documentation (ie. a BKDG). This honestly just smells like the usual AMD marketers trying to capitalize on "open source" wording buzz. As Lakados also mentioned - AMD is generally absolutely terrible at interfacing with the wider open source community(ies). Source dumps are still a good first step I suppose but I'd prefer documentation.

I am 100% sure AMD will deliver a solid framework that is documented when they upload the v0.9 to Github. Still, I also suspect that the documentation will be very barebones and the contributors will need to document many of the intricacies that a good design guide would typically include.
The result after a year or so will probably be excellent, Dell, HP, and Lenovo are going to love it but I expect growing pains, but a stronger platform when all is said and done.

AMD is more than welcome to surprise me on this one, but I expect them to deliver the bare minimum.

 

[GoodBoy]

Why? I hope you realize Spectre/Meltdown were most likely exploited years before public disclosure and that disclosure was only due to researchers (ie. young students like Google's Jann Horn) reverse engineering and then finding issues. Imagine if they had complete open access - skipping the tiresome RE process altogether and going straight to finding and patching security holes. The proprietary nature of CPU (micro)arch did not protect anything. In fact, proprietary/closed source low level blobs and firmware pose a greater security risk since nobody but the outsourced bumbling engineers who wrote it can perform any audits...

I don't think so. Those elements in the microarchitecture first appeared in the i7-920 family, that was what 2008? We got 9 to 10 years of security (ignorance is bliss) before the security holes were found because it was a closed development system.

Aa far as why it was engineered the way that it was, it wouldn't surprise me if a software development tool that 'solves for the simplest solution' (Sorry for shit phrasing) came up with those bits of the design. If someone purposefully chose it, I doubt they could foresee the ways it could be exploited. If you ever read how the data can be 'felt up' out of those registers, it's pretty crazy someone figured out a way to do it. For pc users I don't know that any exploits ever found foothold in the wild. The real dangers of spectre/meltdown was to businesses running virtual environments, where 1 VM could potentially siphon information out of another VM that was running on the same processor. And that was quickly mitigated with side-channel aware schedulers in ESXi, and the other hypervisors as well. What sucked for us, was that the fixed Bios/microcode had about a 10% performance hit.

I'm not sure if spectre/meltdown type vulnerabilities still exist in the last few generations of Intel processors, if so that is pretty shit.

 

[kac77]

AMD was the only choice for the Steam Deck, drivers have nothing to do with it.

Nope open source drivers had quite a bit to do with it. The ability to add features/change things was paramount to that project because it's on Linux.

Intel did not have the GPU capabilities and Nvidia does not have an x86 license and the cost associated with using an Intel/Nvidia combination are too high to make a viable product at that price point. And we can all agree that the performance of ARM and any x86 emulator/translator would be insufficient for the end goal. That leaves AMD as the only viable partner for the project.

You don't need an x86 license and hello Nintendo Switch. A lot of games run via emulation on that device. However Proton isn't an emulator. It's a translation library which runs much faster than an emulator. They are not the same thing at all. The main reason Proton can come out with fixes as fast as it does is because the driver is open source. The developers themselves have said as much.

Nvidia though would have likely turned down such a partnership request because the Nintendo contract is worth more to them than what Valve could have offered and you can bet that had Nvidia and Valve released the Deck together Nintendo would be all over that.

No company is going to turn down money and most certainly not Nvidia.

And I will admit that AMD’s recent string of hires for their opensource GPU drivers has had positive results. But that is one of many and generally the exception to the rule.

AMD has been on an open source hiring spree ever since the Deck and the Tesla contract.

Still didn't answer my previous question though. How is Nouveau?

 

[Lakados]

Nope open source drivers had quite a bit to do with it. The ability to add features/change things was paramount to that project because it's on Linux.

You don't need an x86 license and hello Nintendo Switch. A lot of games run via emulation on that device. However Proton isn't an emulator. It's a translation library which runs much faster than an emulator. They are not the same thing at all. The main reason Proton can come out with fixes as fast as it does is because the driver is open source. The developers themselves have said as much.

No company is going to turn down money and most certainly not Nvidia.

AMD has been on an open source hiring spree ever since the Deck and the Tesla contract.

Still didn't answer my previous question though. How is Nouveau?

Nouveau is a failed endeavor and it will never pan out. It's too complicated for a small team to hope to ever reverse engineer Nvidia's entire graphics driver with no documentation or support. And honestly, after the Nvidia driver code leaked the project basically died because they will have to constantly work to prove they didn't look at it.
The MESA project is paramount to the project the graphics drivers themselves are mostly irrelevant to the process. But AMD having open drivers is proving to be a benefit as it lets them make tweaks in-house on the fly which is why Nvidia has started making that transition, with their Open GPU Kernel, but the state of the drivers is irrelevant when they are the only option as a hardware supplier, even if Nvidia had 100% open drivers they could not supply Valve with adequate hardware. x86 is 100% required to make the Steam Deck work, Proton is working out great and I am a huge fan, but trying to add an ARM to x86 translation layer while delivering something that could give a decent performance on existing games while running on a battery, no way.
Nvidia wouldn't turn away new money, but they certainly won't burn a bridge made of money for a suitcase of it. Building an SoC for Valve would jeopardize their Nintendo relationship and Nvidia wouldn't risk it, the entire production run for the Steam Deck would not come close to what Nintendo is likely paying them for the next Switch.
AMD getting more contracts for things like Tesla will only accelerate their hiring of Opensource developers, because its sort of one of their selling features at this stage, I wonder when they will open up their Windows drivers or when the timeframe for the Linux drivers surpassing the Windows ones is looking like.
But for the past many years, AMD's contribution to the numerous Opensource projects they create has been little more than lip service with no follow-through or backup and it will take at least an equal amount of them actively participating in those projects before I change my mind. So if it was the 2021 contract with Tesla that jump-started their hiring spree, let's see what the state of AMD's opensource project stack looks like sometime after 2025 because they are going to have to work to move past their existing reputation for how they deal with their Opensource initiatives.

 

[kac77]

Nouveau is a failed endeavor and it will never pan out. It's too complicated for a small team to hope to ever reverse engineer AMD's entire graphics driver with no documentation or support. And honestly after the AMD driver code leaked the project basically died because they will have to constantly work to prove they didn't look at it.

Um sir, Nouveau is an open source nVida driver not AMD.

 

[Lakados]

Um sir, Nouveau is an open source nVida driver not AMD.

So used to using the word AMD I just made a typo so I can correct it if it makes you feel better, doesn't change the fact it's dead after the leaks.
Reverse engineering the driver set is a waste of time, more so since Nvidia has started open-sourcing things too.

 

[kac77]

So used to using the word AMD I just made a typo so I can correct it if it makes you feel better, doesn't change the fact it's dead after the leaks.
Reverse engineering the driver set is a waste of time, more so since Nvidia has started open-sourcing things too.

Yeah a few months ago. So why did they need to reverse engineer it anyway?

 

[Lakados]

Yeah a few months ago. So why did they need to reverse engineer it anyway?

Well back in 2005 they needed to patch some functionality into the NVidia “NV” Linux driver and it just sort of evolved from there.

 

[kac77]

Well back in 2005 they needed to patch some functionality into the NVidia “NV” Linux driver and it just sort of evolved from there.

Nope. The reason why I ask is because nVidia in particular has been way worse than AMD ever could be. I mean give me a break cause this is Linus.....

Linus Torvalds Calls NVIDIA The Worst Company Ever​

The ENTIRE benefit of open sourcing software is the development community. It's literally about crowd sourcing solutions from produced open specifications (aka open source).
If anything its the complete opposite of your claims. nVidia has been milking Red Hat in particular to do a lot of work with Wayland. Good lord, if you want a good read about what a manufacturer milking opensource looks like check out the history of Wayland and nVidia. Really good stuff (GBM in particular).
The answer to the question to "So why did they need to reverse engineer it anyway?" is because nVidia didn't have an open source driver so a community of people got together to reverse engineer their driver so that its video cards would work right out of the box without downloading and installing proprietary drivers. That info you pulled isn't going to suffice for people who have been using Linux the entire time and know exactly what it's been like to deal with nVidia proprietary drivers.

If you're on Linux Nouveau is what gets loaded so that you get a visual when first installing Linux on 2D desktops while also providing basic 3D acceleration. The basic 3D capability helped initially when compositors were all the rage. These included Beryl, Compiz, and like one other that escapes me they all rode ontop of xorg. Those were replaced by an upgrade to xorg to Wayland, which replaces xorg with a fully 3d accelerated workspace.

Back in the day, if you wanted to enjoy 3D accelerated desktops the only way to get it was nVidia, which I used. The problem with this approach was that it's closed nature caused all sorts of weird issues. However, nVidia cards were primarily what everyone used and AMD (then ATI) just didn't have anything in the space at that time (they had drivers but they were meant for CAD solutions not much else).

In 2014(5) AMD starts working on a new unified Linux driver that would become fully open source in 2019. That documentation that you think AMD didn't provide is what kicks off Proton, the Deck, assists with GBM development, and quite a few other projects because finally there was a major GPU provider with open source documented drivers.

If you have a preference that's fine, but lets not rewrite history.

 

[Lakados]

N

Nope.

"nouveau was started in 2005 by Stéphane Marchesin as a series of patches upon the "nv" driver,[19] and officially announced in February 2006 at a FOSDEM event"

"nouveau (/nuːˈvoʊ/) is a free and open-source graphics device driver for Nvidia video cards and the Tegra family of SoCs written by independent software engineers, with minor help from Nvidia employees."

https://en.wikipedia.org/wiki/Nouveau_(software)


I am not sure why you are trying to drag Nvidia into an AMD announcement of a product that Nvidia does not make nor compete with.


You keep going on about AMD's drivers, which is one of the literal hundreds of open-source projects they have started, and one of the few that they actually contribute to in a meaningful way.

AMD has a very long history of dumping things they can't figure out but need to compete as Open Source, so the community at large can solve it for them. They seem to be making inroads to correcting that behavior but it does not change their history of doing this and they will need to prove they are going to continue to actually support the projects they do create.

 

[kac77]

N

"nouveau was started in 2005 by Stéphane Marchesin as a series of patches upon the "nv" driver,[19] and officially announced in February 2006 at a FOSDEM event"

"nouveau (/nuːˈvoʊ/) is a free and open-source graphics device driver for Nvidia video cards and the Tegra family of SoCs written by independent software engineers, with minor help from Nvidia employees."

https://en.wikipedia.org/wiki/Nouveau_(software)

Yup and why was it? Because Nvidia didn't have open source drivers / documentation (you know that part you left out).

I am not sure why you are trying to drag Nvidia into an AMD announcement of a product that Nvidia does not make nor compete with.


You keep going on about AMD's drivers, which is one of the literal hundreds of open-source projects they have started, and one of the few that they actually contribute to in a meaningful way.

AMD has a very long history of dumping things they can't figure out but need to compete as Open Source, so the community at large can solve it for them. They seem to be making inroads to correcting that behavior but it does not change their history of doing this and they will need to prove they are going to continue to actually support the projects they do create.

Dude compared to Nvidia they are saints that's why I'm bringing them into this. Nvidia's history is far far worse. So if you don't have a problem with nVidia and their horrible record you can't get your soap box to talk about a company that's done far more for open source than it's rival.