Ray Tracing Has a Noise Problem

[erek]

"Just a reminder that all games that support Ray Reconstruction had the feature turned ON for footage in this comparison (unless otherwise specified). Those games are Cyberpunk 2077, Alan Wake II and Star Wars Outlaws. Ray Reconstruction addresses noise differently to other forms of denoising but falls well short of eliminating the issue."


View: https://www.youtube.com/watch?v=K3ZHzJ_bhaI

 

[mnewxcv]

I mean noise is a real thing in raytracing. You can trade performance for quality but these are games not overnight renders. I haven't been distracted enough by in game noise to make me not enjoy it.

 

[arestavo]

I've seen it a lot. I do try ignore it, but for some games I have to actively try to.

 

[Prince Valiant]

Hardly praiseworthy stuff after six years. I wonder if people would be more demanding of improvements if RT settings were more verbose rather than "off/low/high". Are there any lossless clips of current noise performance floating around?

 

[Armenius]

I don't see what they're talking about. I definitely don't move my character or camera like they do in the video, or zoom in on my games at a factor of 300%. It seems that they went through extreme lengths to show the supposed issue.

 

[LukeTbk]

Hardly praiseworthy stuff after six years. I wonder if people would be more demanding of improvements if RT settings were more verbose rather than "off/low/high". Are there any lossless clips of current noise performance floating around?

or if noisy render of non-RT engine was not also quite popular, this is not particular to RT.

Play Control with RT off and film noise like rendering will be there anyway. In most (all ?) of DF example, it was already noisy before setting RT to on.

 

[arestavo]

of if noisy render of non-RT engine was not also quite popular, this is not particular to RT.

Play Control with RT off and film noise like rendering will be there anyway. In most (all ?) of DF example, it was already noisy before setting RT to on.

Yeah, the video talked about that. The de-noisers need a lot of work, it's just that RT makes it worse because devs are forced to lower the quality for some textures to keep things playable.

 

[Domingo]

You can also take this even more literally. Turning RT on tends to get those GPU fans going, too.

 

[Sir Beregond]

I don't know...here we are 6 going on 7 years into the Ray Tracing era of gaming graphics. I guess the question is was it worth the massive increase in price of entry? Was it worth the state of graphics today being a monopoly because the tech took AMD by surprise? Is it really that much better and noticeable an increase in quality that it was really worth it?

I don't know. On one hand I've seen a lot of good ray tracing, but on the other hand it has caused a dark side of gaming graphics. It's much more expensive. Developers are getting lazy on rasterized graphics which will only push people to keep paying more and more for actually capable RT cards. I don't know. Overall I feel very mixed. I get the idea to make something mainstream that devs would support, Nvidia had to really push RTX and in that sense they succeeded, but part of becoming a mainstream technology that will be required going forward is that the mainstream be able to use it, yet the mainstream keeps stagnating in the product stack.

 

[LukeTbk]

Feel like PS6 would be the answer to a lot of those questions.

They have no reason to care either way and will try to go with what end up being the best to have the best game on their platform we can imagine, that why they embraced ML upscaling because that seem a big winner.

Ubisoft going for it on title that do not even use marketing around it is a bit of a sign in that direction has well.

but part of becoming a mainstream technology that will be required going forward is that the mainstream be able to use it,

PS5, Xbox X are quite mainstream, without them RT required title like Indiana Jones would probably not exist like that (i.e. they would have had a code path for the main consoles made), you have all the latest consoles + getting close to 50% of steam users that have hardware accelerated RT and among people that buy new game it would be larger than that.

 

[Sycraft]

They aren't wrong, but I don't personally find it to be a big deal. Film is also noisy and yet not only is that something that most viewers are fine with, it is something that often gets added in artificially. Lots of games have a "film grain" feature. Heck even digital video has issues with it sometimes (thermal noise).

While I'd like less noisy RT rendering, and I'm hoping we see improvements as GPU power improves, it is not a huge deal to me in general. Brains seem to filter it out fairly well in most cases.

 

[Sir Beregond]

Feel like PS6 would be the answer to a lot of those questions.

They have no reason to care either way and will try to go with what end up being the best to have the best game on their platform we can imagine, that why they embraced ML upscaling because that seem a big winner.

Ubisoft going for it on title that do not even use marketing around it is a bit of a sign in that direction has well.


PS5, Xbox X are quite mainstream, without them RT required title like Indiana Jones would probably not exist like that (i.e. they would have had a code path for the main consoles made), you have all the latest consoles + getting close to 50% of steam users that have hardware accelerated RT and among people that buy new game it would be larger than that.

Fair, but still can't help but notice mainstream on the PC is kind of floundering. So either consoles are selling at a loss or a 4060 Ti having 8GB and being $400-$500 is bullshit.

 

[noko]

The #1 way to reduce noise using Raytracing is by increasing samples. Drafts
, 1000 samples per pixel, 10,000+ plus for finals roughly, Animations with motion, good lord, much more.

Second method and inferior is denoising because you are not taking actual lighting for unlit pixels like from bounce light but estimating. Noise comes from not sufficient lighting data. For example, a candle in a room, only light source with tons of objects and hallways. Photons, hundred million, no thousands billion, no trillions upon trillions will light up that room, even hallways, of course faintly. No way that type of calculation can be done in real time with a GPU. GPU may do on average 1 to 3 samples per pixel, not millions upon millions of photons per spec in the real world . Add motion and you get motion noise. Did anyone even notice how textures using RT, can be sufficiently degraded? That is coming from the denoiser.

Some people just seems to justify anything with Nvidia version of RT, noise should not be one of them. So called real time RT has some good benefits in simplifying lighting, it also has some drawbacks, performance and unfortunately noise.

 

[Flogger23m]

I don't see what they're talking about. I definitely don't move my character or camera like they do in the video, or zoom in on my games at a factor of 300%. It seems that they went through extreme lengths to show the supposed issue.

I see it quite a bit. Most if not all ray tracing games have it. Add in DLSS (which is required) and you see more visual issues. Graphics have certainly become less clear and more blurry over recent years, ray tracing is just one factor. Of course overall it looks better in most cases. Some of the examples they point out, like the Resident Evil games (think it was 2 and 3) added it later and the result was worse than the raster. But then there are other games where it does look good and is worth the downsides.

Still lots of room for improvement and part of the problem is hardware is nowhere near ready. And won't be for many years seeing how slow Nvidia (forget about AMD) is able to increase ray tracing performance.

 

[XoR_]

Graphics have certainly become less clear and more blurry over recent years, ray tracing is just one factor. Of course overall it looks better in most cases.

Graphics looks better because aliasing is a thing of the past with modern AA solutions. It is one thing I won't miss from the past. At some point MSAA stopped working (and it was only providing AA for polygon edges leaving lots of effects still aliased) and we had to rely on post-process filters like MLAA or FXAA which were not perfect and already caused blur. Later however AA moved to TAA and whole rendering started relying on TAA to look not totally broken. TAA blurs image quite a lot and especially in motion. All temporal upscalers are in one way or the other based on TAA and exhibit some issues. There are ways to mitigate some of the blurring but they are not always implemented. Nvidia does great job with DLSS2 and example of very good temporal upscaler is TSR from EPIC but only at Epic preset as even High doesn't implement tricks which de-blur moving objects.

Modern games are packed full of visual artifacts - some from temporal upscalers and some from RT and some from other tricks. It is at times artifact-fest and especially visible when frame rate is low. In fact high frame-rate helps with overall image quality a lot. If frame rate is low like below 60fps it can take longer while for image to settle-in but with 200+fps you need to move really fast to start seeing issues.

---
Solution for RT noise is AI based denoisers.
Nvidia envisioned this and advertised RTX cards to be able to do that.
It doesn't however seem like we really got high quality AI based denoisers and at most low quality AI denoisers or non-AI based denoisers. Even Quake2 RTX which was like this RT showcase didn't really use AI cores for denoising. Its denoiser was however quite good.

Where it comes to Quake2 I actually prefer looks of the game without denoiser - it is noisy as hell but it also looks in some sense quite amazing like that.
To really give good effect frame rate and resolution need to be both very high.

I plan on getting new GPU next year and if I go with most obvious choice which ends with "90" I might be able to hit 360fps at 1440p in Quake2 RTX - and especially should that happen I will definitely give Quake 2 RTX one more go
Otherwise with RT noise in other games the solution is similar: just hit crazy high frame rates and then noise stops being as visible because it denoises in the eyes. At 2-3ms frame time which roughly corresponds to 360-480Hz monitors frame time is so short it is hard to really fully capture it within vision so such noise will not be very visible and images will start to blend.

 

[noko]

Graphics looks better because aliasing is a thing of the past with modern AA solutions. It is one thing I won't miss from the past. At some point MSAA stopped working (and it was only providing AA for polygon edges leaving lots of effects still aliased) and we had to rely on post-process filters like MLAA or FXAA which were not perfect and already caused blur. Later however AA moved to TAA and whole rendering started relying on TAA to look not totally broken. TAA blurs image quite a lot and especially in motion. All temporal upscalers are in one way or the other based on TAA and exhibit some issues. There are ways to mitigate some of the blurring but they are not always implemented. Nvidia does great job with DLSS2 and example of very good temporal upscaler is TSR from EPIC but only at Epic preset as even High doesn't implement tricks which de-blur moving objects.

Modern games are packed full of visual artifacts - some from temporal upscalers and some from RT and some from other tricks. It is at times artifact-fest and especially visible when frame rate is low. In fact high frame-rate helps with overall image quality a lot. If frame rate is low like below 60fps it can take longer while for image to settle-in but with 200+fps you need to move really fast to start seeing issues.

---
Solution for RT noise is AI based denoisers.
Nvidia envisioned this and advertised RTX cards to be able to do that.
It doesn't however seem like we really got high quality AI based denoisers and at most low quality AI denoisers or non-AI based denoisers. Even Quake2 RTX which was like this RT showcase didn't really use AI cores for denoising. Its denoiser was however quite good.

Where it comes to Quake2 I actually prefer looks of the game without denoiser - it is noisy as hell but it also looks in some sense quite amazing like that.
To really give good effect frame rate and resolution need to be both very high.

I plan on getting new GPU next year and if I go with most obvious choice which ends with "90" I might be able to hit 360fps at 1440p in Quake2 RTX - and especially should that happen I will definitely give Quake 2 RTX one more go
Otherwise with RT noise in other games the solution is similar: just hit crazy high frame rates and then noise stops being as visible because it denoises in the eyes. At 2-3ms frame time which roughly corresponds to 360-480Hz monitors frame time is so short it is hard to really fully capture it within vision so such noise will not be very visible and images will start to blend.

The other solution, maybe, is using procedural textures which can use what available RT lighting data to shade the complete object.

Procedural textures take a very small foot print compared to the mega textures used today. Just more compute intensive. Add AI to mimic procedurally the very fine detail of the mega textures, the artifacts, blurring would be gone plus a 8gb card would have too much VRam. Just that today method of games does do not use them.

Now areas in a game where the pixels do have enough light samples using RT, blurring of textures and artifacting are for the most part gone.

 

[XoR_]

The other solution, maybe, is using procedural textures which can use what available RT lighting data to shade the complete object.

Procedural textures take a very small foot print compared to the mega textures used today. Just more compute intensive. Add AI to mimic procedurally the very fine detail of the mega textures, the artifacts, blurring would be gone plus a 8gb card would have too much VRam. Just that today method of games does do not use them.

Now areas in a game where the pixels do have enough light samples using RT, blurring of textures and artifacting are for the most part gone.

As usual consoles hold back progress + game devs are kinda lazy. Good tools need to be provided first with great examples why it is worth it. Today textures are good enough for even 4K games at current VRAm capacities. Examples why it is worth to put more effort in to this stuff is needed.

8GB will never be too much again IMHO
VRAM usage is high because it is easier to use more memory than bother with optimisations. Same old same old.

Worst of all all this excessive VRAM and RAM usage is not used to improve visuals that much. Say mipmaps - not many devs ever optimise them by hand and what we have stored on disk and VRAM is so generic it could be easily calculated on thefly. You get situations devs cannot fit gamefor Xbox Series S because they have no idea how engine their game users work and brute forced graphics. Lag is in some cadres off the charts - it uses more VRAM to add lag...

The only solution today is getting more memory. For GPUs 16GB is optimal for mid range cards and 24GB necessary for high end. Hopefully 32GB requirement won't happen anytime soon...

 

[noko]

As usual consoles hold back progress + game devs are kinda lazy. Good tools need to be provided first with great examples why it is worth it. Today textures are good enough for even 4K games at current VRAm capacities. Examples why it is worth to put more effort in to this stuff is needed.

8GB will never be too much again IMHO
VRAM usage is high because it is easier to use more memory than bother with optimisations. Same old same old.

Worst of all all this excessive VRAM and RAM usage is not used to improve visuals that much. Say mipmaps - not many devs ever optimise them by hand and what we have stored on disk and VRAM is so generic it could be easily calculated on thefly. You get situations devs cannot fit gamefor Xbox Series S because they have no idea how engine their game users work and brute forced graphics. Lag is in some cadres off the charts - it uses more VRAM to add lag...

The only solution today is getting more memory. For GPUs 16GB is optimal for mid range cards and 24GB necessary for high end. Hopefully 32GB requirement won't happen anytime soon...

Also who owns the big game studios now? Microsoft and with Sony’s influence, will they allow games that would run or look like crap on their consoles? Probably not.

Now nothing prevents Nvidia making a console in the future. Their arm CPU with GPU, maybe a super APU. They have game servers, GeForce now, great relationships with a number of developers, dominant in PC gaming and laptops. All their tech could be used.

 

[pendragon1]

View: https://www.youtube.com/watch?v=U8z-1Bqn4Rs

 

[pendragon1]

View: https://www.youtube.com/watch?v=KEtb0punTHk

 

[Prince Valiant]

The sad state of things .

 

[LukeTbk]

Sometime some talk as if gamedev and user are some poor victime being forced, saying that if a native 1080p minimum internal res for upscaling would help a lot of title now..... How can AMD know what betters than the gamedev themselves and the user being offered that choice...

Preferring stability-clarity-resolution to other things is completely subjective (obviously) and some talk as if this was some objective subject. The idea that IndianaJones or Alan Wake 2 look like trash on their intended target hardware (and the reason being would be some lack of quality or smearing) make you sound a bit like someone with a monofocus problem

Lot of the reasoning and talk, do not seem to hold up scrutiny when talking about a Sony exclusive on a PS5pro game for example, what would be the non that how the dev feel it give me the best result possible reason going there, there zero need to use some commercial hardware lock in anyway, it is already fully locked-down (probably a big reason why that kind of video do not spend much time thinking about those).

Some statements seem a bit empty, AI upscaling cost milliseconds (yes ? of course), better quality AI upscaling cost more (yes?), anti-aliasing with no upscaling can cost less ms (yes...), those are a start of premises (all very well known to the viewer) being said without some ending to them

 

[XoR_]

I am with Nvidia on this one.
I want RT/PT and AI and not rasterization.
Heck, I want this and I want powerful AI hardware to run fancy AI on my computer locally.

Sooner or later GPUs will process 1.5-bit weights in hardware natively and will have enough memory to take something like voxel graphics and slap details on top of it with AI.
Just look at game Teardown - it has photorealistic lighting, physics and all. Make voxels smaller, use path tracing for lighting and have pre-trained models that will add all the missing pixel-level details and we pretty much have photo-realistic game with destructible world.

And even then hardware will be by necessity fast enough to run pure rasterized games super fast and there will be people willing to buy games looking retro.
Also there will be people who complain about the way modern games are done is bad because it will still require year after year newer hardware.

In other words the more things change the more they stay the same.

 

[Toboggan]

I am with Nvidia on this one.
I want RT/PT and AI and not rasterization.
Heck, I want this and I want powerful AI hardware to run fancy AI on my computer locally.

Sooner or later GPUs will process 1.5-bit weights in hardware natively and will have enough memory to take something like voxel graphics and slap details on top of it with AI.
Just look at game Teardown - it has photorealistic lighting, physics and all. Make voxels smaller, use path tracing for lighting and have pre-trained models that will add all the missing pixel-level details and we pretty much have photo-realistic game with destructible world.

And even then hardware will be by necessity fast enough to run pure rasterized games super fast and there will be people willing to buy games looking retro.
Also there will be people who complain about the way modern games are done is bad because it will still require year after year newer hardware.

In other words the more things change the more they stay the same.

Keep dreaming