Linux Steam beta update preparing to distribute shaders.

Status
Not open for further replies.
So it looks like this could have a significant improvement on Linux gaming, considering shaders will no longer be compiled on the fly but will be pre compiled depending on GPU and driver used and stored in a separate folder. Very interesting, going to be watching this development closely.
 
BulletDust said:
So it looks like this could have a significant improvement on Linux gaming, considering shaders will no longer be compiled on the fly but will be pre compiled depending on GPU and driver used and stored in a separate folder. Very interesting, going to be watching this development closely.
Click to expand...

At the very least I could see this helping Steam under wine out if it doesn't cause issues anyway.
 
BulletDust said:
~/.nv looks like a cache of some sort, not too sure what it's storing though? Are they shaders or something else?
Click to expand...
It was 1st added with the 290 series of nvidia drivers (Oct 2011)
https://http.download.nvidia.com/XFree86/Linux-x86/290.10/README/openglenvvariables.html

OpenGL Shader Disk Cache
The NVIDIA OpenGL driver utilizes a shader disk cache. This optimization benefits some applications, by reusing shader binaries instead of compiling them repeatedly. The related environment variables __GL_SHADER_DISK_CACHE and __GL_SHADER_DISK_CACHE_PATH, as well as the GLShaderDiskCache X configuration option, allow fine-grained configuration of the shader cache behavior. The shader disk cache:

  1. is always disabled for indirect rendering

  2. is always disabled for setuid and setgid binaries

  3. by default, is disabled for direct rendering when the OpenGL application is run as the root user

  4. by default, is enabled for direct rendering when the OpenGL application is run as a non-root user
The GLShaderDiskCache X configuration option forcibly enables or disables the shader disk cache, for direct rendering as a non-root user.

The following environment variables configure shader disk cache behavior, and override the GLShaderDiskCache configuration option:

Environment Variable Description
__GL_SHADER_DISK_CACHE (boolean) Enables or disables the shader cache for direct rendering.
__GL_SHADER_DISK_CACHE_PATH (string) Enables configuration of where shader caches are stored on disk.


If __GL_SHADER_DISK_CACHE_PATH is unset, caches will be stored in $XDG_CACHE_HOME/.nv/GLCache if XDG_CACHE_HOME is set, or in $HOME/.nv/GLCache if HOME is set. If none of the environment variables __GL_SHADER_DISK_CACHE_PATH, XDG_CACHE_HOME, or HOME is set, the shader cache will be disabled. Caches are persistent across runs of an application. Cached shader binaries are specific to each driver version; changing driver versions will cause binaries to be recompiled.
Click to expand...
 
Interesting, so it looks like NVIDIA drivers already cache shaders? So what matters is who's implementation is more efficient? Valves or Nvidia's?

Perhaps it's a feature that will benefit Intel/AMD users more?
 
As I understand it both Nvidia and AMD use shader cache in both windows and Linux. What Steam seems to be talking about here is taking that shader data and saving it in the cloud ? While I could see this helping out for shaders you don't use often... to me it seems like this would only really make things faster the first time you load it. Your GPU driver should save compiled shaders for later use... I guess steam is talking about skipping that initial compile and having the client download a bank of shaders for your card before you ever launch the game. (Doesn't sound revolutionary just a very cool little thing that may bump performance on initial load... pretty cool really cloud saves for shaders, the second time I type that ya that is cool)

From the Beta Update;
  • Added support for shader cache management on supported drivers (Mesa 17.1 and NVIDIA 381.26.08 or 384.59). Shader caches for games launched by Steam are placed in separate folders next to their Steam Library folder and are deleted when each game is uninstalled, or when switching drivers or graphics cards. Individual shaders are tracked and catalogued by the Steam servers in preparation for distributing pre-compiled shaders. This tracking can incur slight additional bandwidth use. The system can be disabled by setting the environment variable STEAM_ENABLE_SHADER_CACHE_MANAGEMENT=0
As naib has pointed out nvidias Linux driver does cache... as does the non-free AMD drivers.

So does the AMD open source driver;
http://www.phoronix.com/scan.php?page=article&item=mesa-171-features&num=1
"- The Mesa shader cache support was merged both for the TGSI shader cache as well as the RadeonSI binary shader cache! This should help game load times for newer titles. Currently the cache is enabled by default but we'll see if they change that prior to the Mesa 17.1.0 release. "

If your interested here is some actual benchmarks of RadeonSI's Mesa Shader Cache.
http://www.phoronix.com/scan.php?page=article&item=radeonsi-shader-cache&num=1

PS interesting note... the guy working the shader cache stuff into the AMD open source drivers is Timothy Arceri a Valve employee. It sounds to me like he has been working to get shader caching into the free driver (its been in the non free amd and nv drivers for awhile) this seems to be a continuation/expansion of his work. :)
 
Last edited:
ChadD said:
Shader caches for games launched by Steam are placed in separate folders next to their Steam Library folder and are deleted when each game is uninstalled, or when switching drivers or graphics cards
Click to expand...

Are you sure Valve's intention is to store shaders in the cloud? That sounds like they plan on storing them locally to me?
 
BulletDust said:
Are you sure Valve's intention is to store shaders in the cloud? That sounds like they plan on storing them locally to me?
Click to expand...

Locally after they have been downloaded.

If I am understanding this it works this way....

Right now NV / AMD drivers do this already, if a game compiles a shader it stores it and reuses it if need be.

What I believe this steam shader manager is doing is... downloading a list of shaders when you install a game. So if you have say a Geforce 1070 and install Dota2, the steam client will download a bunch of compiled 1070 shaders for that game and store them. This would save your card the work of compiling them in the first place... and speed up the game the first time you play it anyway. In order to get that data to start with I assume they plan to build some type of shader library for each game and different hardware. I would imagine for most people someone else will have played the game in question on the same GPU before and you wouldn't need to upload anything, but if you where the first to play a game on X hardware I assume your shader would get added to the library ? (I am guessing a bit based on what the note says)

So anyway ya stored locally for use... but if this works as I am guessing it will download prebuilt shaders when you install steam games. If you have a popular video card I would imagine not many games you install won't already have a database of precompiled shaders for you... if not steam will upload yours for the next user to share. (if it does save people from compiling a lot of shaders to start with it could make some games feel a lot smoother I'm sure... I imagine poorly coded games that have tons of shaders with small changes rather then aggressively reusing them would benefit the most from a download of precomiled before hand.)

I don't think this is going to provide major speed increases or anything... its just maturing a feature a little bit. It sounds like Valve has had at least one guy working at shader caching systems for the MESA open source drivers, it sounds to me like they took what they learned and have expanded it. It may not speed things up a ton but it does sound pretty darn elegant.
 
ChadD said:
Locally after they have been downloaded.

If I am understanding this it works this way....

Right now NV / AMD drivers do this already, if a game compiles a shader it stores it and reuses it if need be.

What I believe this steam shader manager is doing is... downloading a list of shaders when you install a game. So if you have say a Geforce 1070 and install Dota2, the steam client will download a bunch of compiled 1070 shaders for that game and store them. This would save your card the work of compiling them in the first place... and speed up the game the first time you play it anyway. In order to get that data to start with I assume they plan to build some type of shader library for each game and different hardware. I would imagine for most people someone else will have played the game in question on the same GPU before and you wouldn't need to upload anything, but if you where the first to play a game on X hardware I assume your shader would get added to the library ? (I am guessing a bit based on what the note says)

So anyway ya stored locally for use... but if this works as I am guessing it will download prebuilt shaders when you install steam games. If you have a popular video card I would imagine not many games you install won't already have a database of precompiled shaders for you... if not steam will upload yours for the next user to share.

I don't think this is going to provide major speed increases or anything... its just maturing a feature a little bit. It sounds like Valve has had at least one guy working at shader caching systems for the MESA open source drivers, it sounds to me like they took what they learned and have expanded it. It may not speed things up a ton but it does sound pretty darn elegant.
Click to expand...

It's all very interesting, furthermore it's quite obvious that Valve's push towards Linux under Steam is ongoing and they certainly aren't showing any signs of slowing down any time soon.
 
  • Like
Reactions: ChadD
like this
BulletDust said:
It's all very interesting, furthermore it's quite obvious that Valve's push towards Linux under Steam is ongoing and they certainly aren't showing any signs of slowing down any time soon.
Click to expand...

It sure does make the articles declaring steamos and Valves Linux support dead seem pretty freaking stupid.

Valve has a lot of developers doing nothing but working on Linux all day. The whole push for the improvement of the open source opengl / amd drivers is being headed by Valve employees. What is actually really cool in a way... is it sounds like this is a feature that will also benefit windows and mac users. There are plenty of things Valve has developed for Linux that has benefited ALL steam users, its more then just the obvious big picture modes and the like. :)
 
At the end of the day, no one has any figures on SteamOS sales whatsoever, literally nothing at all - Not even web counters. People are trying to claim poor SteamOS adoption due to poor sales of Steam machines, but the two are not tied to one another.

The most realistic claim is: "SteamOS adoption? NFI, nor does it really matter in the grand scheme of things".

Gaming under Linux is far from dead, no matter what the haters want to claim while following the mantra that a shameless bias towards Windows = Open and realistic.
 
BulletDust said:
Gaming under Linux is far from dead, no matter what the haters want to claim while following the mantra that a shameless bias towards Windows = Open and realistic.
Click to expand...

I've never claimed Linux was dead. But here's the reality of it. I'm looking at the Steam Store right now and the latest 25 releases dated today. Of that list of 25 games, 5 are Linux compatible, 9 are macOS compatible and all 25 are Windows compatible, 4 of them for Windows VR. And that's the way the numbers generally run each and everyday and have since Linux support came to Steam over 4 years ago.

You can preach to me about bias and whatnot but do you honestly think the average person perusing the same list is thinking "Wow, Linux gaming has come a long way!" ? What's so ironic to me about some folks on this subject is how you praise Valve for its Linux efforts but then fail to see that Valve has done more to entrench Windows gaming than anyone, including Microsoft. Not saying that Valve was trying to do that but the Steam Store makes the most compelling case for Windows gaming there is.

It is Valve, NOT Microsoft, that's tied me to Windows gaming like many millions of others. Valve is the one selling all of these Windows only games, not Microsoft. Consider that the next time you talk about "bias" and "realistic".
 
heatlesssun said:
I've never claimed Linux was dead. But here's the reality of it. I'm looking at the Steam Store right now and the latest 25 releases dated today. Of that list of 25 games, 5 are Linux compatible, 9 are macOS compatible and all 25 are Windows compatible, 4 of them for Windows VR. And that's the way the numbers generally run each and everyday and have since Linux support came to Steam over 4 years ago.

You can preach to me about bias and whatnot but do you honestly think the average person perusing the same list is thinking "Wow, Linux gaming has come a long way!" ? What's so ironic to me about some folks on this subject is how you praise Valve for its Linux efforts but then fail to see that Valve has done more to entrench Windows gaming than anyone, including Microsoft. Not saying that Valve was trying to do that but the Steam Store makes the most compelling case for Windows gaming there is.

It is Valve, NOT Microsoft, that's tied me to Windows gaming like many millions of others. Valve is the one selling all of these Windows only games, not Microsoft. Consider that the next time you talk about "bias" and "realistic".
Click to expand...

Valve pushes PC gaming heatle... your point just included pointing out that every week they release games on all 3 major PC platforms. The only thing they could do to further support "PC" gaming would be to release a few games on BSD and Hiku.

Offering a market for publishers to sell games is hardly promoting a one OS world. Valve never said they want Linux to be the ONLY PC platform. They simply don't want windows to be the only option. Ensuring quality Mac and Linux support means gamers have choice. Forget all the politics and evil MS worries. The bottom line is Valve employees a lot of Linux lovers, Valve loves Linux, Valve supports Linux.... that doesn't mean they have ever promoted the death of windows. Pointing out that MICROSOFT is on a road to remove digital store competition, and perhaps fully wall Windows is hardly something only Valve has claimed or worried about. For Valve its a huge worry clearly as a walled garden windows means Valve is out of business. Still yes Valve has been fighting for an OPEN windows platform as much as they have been fighting to advance cross os gaming. They felt the best way to ensure an OPEN windows platform was to introduce some actual options, if users and developers have options outside of windows MS can't throw their weight around. That should be pretty easy to understand.

You, yes YOU... better hope Linux (and MacOS) both continue to get a healthy amount of games launched on both. If the day ever came where releases for both dried up... ya don't kid yourself MS would wall windows up as fast as they could and kill Valve off. An actually healthy PC gaming (Windows/Mac/Linux... not just Windows as MS would like to promote) helps ensure the PC platform remains open.

Valve doesn't support Linux cause they want to kill windows, they simply want to force MS to always keep windows open for them to conduct business.
 
ChadD said:
Valve doesn't support Linux cause they want to kill windows, they simply want to force MS to always keep windows open for them to conduct business.
Click to expand...

Of course Valve isn't trying to kill 95% of it's business. I perfectly understand that Valve has made investments in Linux and macOS to lessen the dependence on the Windows users. But Windows users are the goose that laid a platinum egg for Valve. When everything on Steam is basically 100% Windows compatible, it's clear that Valve and PC game developers get that. And yes, that's of enormous value to Windows as a brand because when it comes to PC gaming, Steam proves beyond a shadow of a doubt that PC gaming for now still is Windows gaming. And Linux hasn't even caught up with macOS to really challenge that.
 
heatlesssun said:
Of course Valve isn't trying to kill 95% of it's business. I perfectly understand that Valve has made investments in Linux and macOS to lessen the dependence on the Windows users. But Windows users are the goose that laid a platinum egg for Valve. When everything on Steam is basically 100% Windows compatible, it's clear that Valve and PC game developers get that. And yes, that's of enormous value to Windows as a brand because when it comes to PC gaming, Steam proves beyond a shadow of a doubt that PC gaming for now still is Windows gaming. And Linux hasn't even caught up with macOS to really challenge that.
Click to expand...

Well that makes it official... you don't work for Microsoft.

Cause MS doesn't understand what you just explained.

MS wants "PC" to mean windows. MS is still trying to push a walled garden... and that is clear. They don't understand what you just explained. BTW its not windows that laid Valve a golden egg... Its the game industry. Windows is the operating system... the mean to operate the hardware as you have said before nothing more. Valve has made no money for MS, just as MS has made no money for valve. Unless there are MS studio games on Steam I missed somewhere.
 
ChadD said:
MS wants "PC" to mean windows.
Click to expand...

LOL! Look at Valve's own Steam Wallet Gift Cards!

https://store.steampowered.com/account/redeemwalletcode
upload_2017-7-27_16-13-50.png


Even Valve wants "PC" to mean Windows.

ChadD said:
BTW its not windows that laid Valve a golden egg... Its the game industry.
Click to expand...

I didn't say Windows, I said Windows USERS. Windows USERS made Gabe a billionaire.
 
Last edited:
Indeed Valve is not above taking marketing monies... they are out to make $ who would have thunk it. :) lol
 
Oh my God.

Once the conscious decision has been made to switch from Windows to Linux due to the issues surrounding Microsoft and Windows and an individual considers that there will be acceptable losses re: Games and is comfortable with that, titles released under Windows no longer matter as the user is not running Windows - They are running Linux.

Sure, gamers like playing games and there are all kinds of gamers. There are casual gamers, social gamers, right through to hardcore gamers and everything in between - It can safely be assumed that not one of those gamers plays 100% of the Steam catalog, not one, nada. Furthermore, the assumption that gamers will only use the platform that supports 100% of titles released and are completely devoid of morals or the ability to dabble in operating systems other than Windows is completely unfounded, frankly - It's a load of garbage not substantiated by Steam statistics re: Intel graphics.

As for your childish comments regarding 'PC', that's called marketing, that's what separates Windows from Linux - An aggressive marketing department. The fact that you notice such marketing rubbish, coupled with the fact that you actually make comment on it, speaks volumes in relation to your consumerism and natural ability to be swayed by media.

Wake up Neo, you're literally living in a dreamworld (with that thing on your head).
 
  • Like
Reactions: ChadD
like this
It also drives me nuts that for some people "hardcore" gamer has to mean a gamer with a FPS addiction. lol

By heatles definition people playing Star Craft in diapers are not hardcore enough... cause you can run that on anything after all. :)
 
ChadD said:
It also drives me nuts that for some people "hardcore" gamer has to mean a gamer with a FPS addiction. lol

By heatles definition people playing Star Craft in diapers are not hardcore enough... cause you can run that on anything after all. :)
Click to expand...

The bulk of the top ten games under Steam are games that can be played on any PC of the last six years, possibly even earlier, with modest to low specifications. There is no evidence supporting the idea that gamers are defined by the need to play 100% of the Steam library or definned by the amount of money spent on hardware.

SdJsd60h.png


Heatlesssun can deliberately ignore that fact all he likes, it's not going to change anything.
 
BulletDust said:
It can safely be assumed that not one of those gamers plays 100% of the Steam catalog, not one, nada.
Click to expand...

But it's beyond easy to come across a game that one might want to play that isn't Linux compatible on Steam since 80% of them aren't.
 
It is also interesting to see what has been happening with an Engine like Cry. Which everyone agrees looks fantastic... but is a resource/spec pig. Of all the engines developers can use they seem to pick ones that tend to not bring hardware to its knees. I know some people argue that is a bad thing... cause we want our games to look better and better. You know though I would rather play a cartoon looking game like DOTA, overwatch or something interesting like boardlands cell like art then a ultrarealistic FPS. IMO its prehaps a good thing if the push for cross platform, low spec hardware drives game development and not eye candy development.
 
BulletDust said:
The bulk of the top ten games under Steam are games that can be played on any PC of the last six years, possibly even earlier, with modest to low specifications. There is no evidence supporting the idea that gamers are defined by the need to play 100% of the Steam library or definned by the amount of money spent on hardware.

SdJsd60h.png


Heatlesssun can deliberately ignore that fact all he likes, it's not going to change anything.
Click to expand...

I've never denied that. But think about this. Valve itself reports about 32 million DAILY users these days. This list is only short oh, by 30 MILLION.
 
heatlesssun said:
But it's beyond easy to come across a game that one might want to play that isn't Linux compatible on Steam since 80% of them aren't.
Click to expand...

Well if that is what you find by all means stick to windows. Many of us find Linux to more then take care of what we want to play. For the types of games I like to play many have Linux clients... or run very well under wine. I realized long ago that buying hardware so I could play the latest greatest GPU ball buster games wasn't my thing. I was on the train for a long time then realized 90% of the games I was buying I played through once, or didn't even bother finishing... but was keeping around so I could bench them. Man that was freaking stupid. Now I play what I want to play... and I realized what I want to play doesn't require windows any more so bonus I don't have to keep a windows drive around anymore.
 
ChadD said:
It also drives me nuts that for some people "hardcore" gamer has to mean a gamer with a FPS addiction. lol

By heatles definition people playing Star Craft in diapers are not hardcore enough... cause you can run that on anything after all. :)
Click to expand...

I don't use the term "hardcore" gamer, that's a term that Linux folks often like to use to somehow mask weaknesses in Linux gaming. Be it casual or RPG or MMO or whatever, if it runs on the desktop it runs on Windows.
 
heatlesssun said:
I've never denied that. But think about this. Valve itself reports about 32 million DAILY users these days. This list is only short oh, by 30 MILLION.
Click to expand...

You mean there are double as many gamers as use steam... ok. I don't need a math degree to know that if I have a sample size of 50% of a system... I can make an extremely accurate prediction about the other half of the system.
 
ChadD said:
Well if that is what you find by all means stick to windows. Many of us find Linux to more then take care of what we want to play. For the types of games I like to play many have Linux clients... or run very well under wine. I realized long ago that buying hardware so I could play the latest greatest GPU ball buster games wasn't my thing. I was on the train for a long time then realized 90% of the games I was buying I played through once, or didn't even bother finishing... but was keeping around so I could bench them. Man that was freaking stupid. Now I play what I want to play... and I realized what I want to play doesn't require windows any more so bonus I don't have to keep a windows drive around anymore.
Click to expand...

Cool. I just don't think about the OS really when dealing with PC games. I just use Windows because of it's compatibility and that's that.
 
heatlesssun said:
I've never denied that. But think about this. Valve itself reports about 32 million DAILY users these days. This list is only short oh, by 30 MILLION.
Click to expand...

As a Linux user free from the issues surrounding Windows this doesn't mean anything to me, chances are it means nothing to most users of Steam.
 
heatlesssun said:
Cool. I just don't think about the OS really when dealing with PC games. I just use Windows because of it's compatibility and that's that.
Click to expand...

Great!

So what drives you to troll every thread relating to Linux and gaming?
 
BulletDust said:
As a Linux user free from the issues surrounding Windows this doesn't mean anything to me, chances are it means nothing to most users of Steam.
Click to expand...

You report a lot of issues with Windows. You've mentioned tons of times about how I like to "brag" about my hardware. Not that in a place like this what I have is even remotely special. But there's no way I'd put up with all of the stability and malware issues you report of Windows spending my hard earned money to buy all of this stuff. Most of which doesn't even come from Microsoft.

Just understand, I spend a lot of money on Steam and the stuff I buy isn't Linux compatible. So sure I get that Valve has done a lot for Linux gaming but they do offer a LOT of stuff for Windows users that's exclusive to Windows. Valve and Steam are a huge part of why I spend money on things that you and ChadD see no value in. Valve and it's support of VR are why a lot of people buying this stuff are doing so.

So take things in perspective. A lot of what you blast me over is supported and being developed by the very same company that you and others praise for supporting Linux gaming.
 
BulletDust said:
Great!

So what drives you to troll every thread relating to Linux and gaming?
Click to expand...

Who said anything about Linux gaming being dead? Sure I think it's crap compared to what I expect of PC gaming but as you've pointed out many times, there are those folks that don't like Windows or Microsoft and want to avoid anything they do at all cost. And there's a market there. It's not necessarily one that's anymore more practical that what you think of my expenditures on gaming. But there's a market there in both cases.

For all that Valve has done for Linux gaming, it's promoting VR pretty big these days and encouraging people to go buy powerful hardware to run that stuff. I'm just being realistic about what's going on is all.
 
heatlesssun said:
Who said anything about Linux gaming being dead? Sure I think it's crap compared to what I expect of PC gaming but as you've pointed out many times, there are those folks that don't like Windows or Microsoft and want to avoid anything they do at all cost. And there's a market there. It's not necessarily one that's anymore more practical that what you think of my expenditures on gaming. But there's a market there in both cases.

For all that Valve has done for Linux gaming, it's promoting VR pretty big these days and encouraging people to go buy powerful hardware to run that stuff. I'm just being realistic about what's going on is all.
Click to expand...

You didn't answer the question.

What drives you to troll every thread related to Linux and gaming? What drives you to make out as if Linux users exaggerate the capabilities of their prefered operating system when all they're doing is posting their honest, daily experiences? What drives you to troll threads so badly that Linux users spend a good 70% of the thread defending themselves against your shamelessly biased accusations?
 
BulletDust said:
You didn't answer the question.

What drives you to troll every thread related to Linux and gaming? What drives you to make out as if Linux users exaggerate the capabilities of their prefered operating system when all they're doing is posting their honest, daily experiences? What drives you to troll threads so badly that Linux users spend a good 70% of the thread defending themselves against your shamelessly biased accusations?
Click to expand...

You just have this unrealistic view of Valve/Steam and PC gaming. And you are ALWAYS mentioning Windows folks.
 
Status
Not open for further replies.
Back
Top