Nvidia Is Hiring Ray Tracing Talent to Revolutionize the Graphics Industry

[cageymaru]

With the steady influx of ray tracing talent that Nvidia has been hiring as of late, it is no surprise that many believe that their next generation of video cards will feature the technology. Sam Lapere's blog expresses a passionate dedication to the art of achieving photoreal rendering with GPU ray tracing. Here he documents some of the ray tracing industry veterans and scholars that Nvidia has brought in to lead their ray tracing venture.

With these recent hires and initiatives such as RTX (Nvidia's ray tracing API), it seems that Nvidia will be pushing real-time ray tracing into the mainstream really soon. I'm really excited to finally see it all come together. I'm pretty sure that ray tracing will very soon be everywhere and its quality and ease-of-use will soon displace rasterisation based technologies (it's also the reason why I started this blog exactly ten years ago).

 

[DoubleTap]

Very exciting. I remember when it took a week or longer to render a single frame of some abstract with chrome spheres and such

 

[maxz01]

Great stuff! Nvidia is preparing for the future, once chips move beyond silicon and we get significant performance jumps generation to generation again!

 

[mjz_5]

Is ray tracing really that much better?

 

[ChadD]

Is ray tracing really that much better?

Yes... but I have to say reality tells me this isn't anything anyone is going to see in a realistic fully rendered video game for years.

Game developers don't target high end gear... and AAA games still take 4-5 years to develop. I doubt highly any major studios will be targeting ray tracing hardware outside perhaps the odd 100% optional add on effect for another 3-4 years min even if Nvidia does release a high end card that can do it before the year ends. That puts the first shipping AAA titles 7-8 years out at least.

Couple that simple truth with all the $ game studios have been making on Cross platform games like fortnite... I just don't see a wave of Ray traced games any time soon.

On the other hand... just because Nvidia is spending R&D money on Ray tracing doesn't mean they are really expecting big gaming profits. There is a very good size pro market that would very much like better real time ray tracing hardware. I would think this is more likely to move Quatros then Geforces.

 

[DukenukemX]

As long as the PS4 and Xbox One don't have ray tracing capabilities, any game released will rarely have this tech unless Nvidia intervenes. Even then, it'll be a slap job that'll perform poorly.

 

[ChadD]

As long as the PS4 and Xbox One don't have ray tracing capabilities, any game released will rarely have this tech unless Nvidia intervenes. Even then, it'll be a slap job that'll perform poorly.

To make it even less likely all signs point to PS5... XboxNextOne seem to be sticking with AMD.

Having said that... rumors are MS is planning a streaming only XboxNextOne. Nvidia putting ray tracing hardware into streaming servers, actually sort of does make a ton of sense.

If Nvidia + Sony/MS manage to get a handful of big name AAA titles to go 100% ray traced and 100% streaming. That could actually be something.

If Nvidia had their choice I'm sure they would let AMD continue making all the low profit margin integrated chips for the home boxes... and eat up all the rich cheddar selling stupid high end cards to streaming farms.

Stream only ray traced games... might actually be just what MS and Sony need to get die hard gamers to jump into streaming. Fully ray traced streaming exclusives might really appeal. I could see MS going AMD integrated for the next box... and power it all with Nvidia server GPUs.

 

[euskalzabe]

I just don't see a wave of Ray traced games any time soon.

Agreed. Seems like they'll take it progressively: start with better shadows and AA, move little by little as years go and hardware gets more powerful, towards more and more raytraced elements. This could be a replay of the 90s if they can pull off that kind of progress. Considering they're using few days and denoising (making up detail), as long as that algorithm improves, may e they can get away with less and less days per render, maintaining or improving IQ while exponentially speeding it up (or, ideally, giving camera the option of slower/more detail vs faster/less accurate, just like it's been the case for resolutions!

 

[ChadD]

Agreed. Seems like they'll take it progressively: start with better shadows and AA, move little by little as years go and hardware gets more powerful, towards more and more raytraced elements. This could be a replay of the 90s if they can pull off that kind of progress. Considering they're using few days and denoising (making up detail), as long as that algorithm improves, may e they can get away with less and less days per render, maintaining or improving IQ while exponentially speeding it up (or, ideally, giving camera the option of slower/more detail vs faster/less accurate, just like it's been the case for resolutions!

Even if we never see it... its good to know R&D money is being spent on improving game IQ beyond simply turning out a few % points more FPS.

 

[Ski]

Is ray tracing really that much better?

To put it in simpler terms: Ray tracing is the holy grail of computer graphics my friend. To illustrate what this process is, think of the objects you’re seeing which are illuminated by beams of light. Now turn that around and follow the path of those beams backwards from your eye to the objects that light interacts with. That’s ray tracing. What makes ray tracing so difficult to process on a cpu is the millions of beams it's shooting out from the camera to objects which are diffused, reflected, refracted, in all different directions. Rendering engines have to calculate each and every single one of those rays, but todays computers can't process them in real time.

Rasterizing, which is what todays video games use, overcome this hurdle by displaying the three-dimensional objects on a two-dimensional screen. It’s fast. And, the results have gotten very good over time, even ray tracing is better. With rasterization, objects on the screen are created from a mesh of virtual triangles, or polygons, that create 3D models of objects. In this virtual mesh, the corners of each triangle (vertices) intersect with the vertices of other triangles of different sizes and shapes. A lot of information is associated with each vertex, including its position in space, as well as information about color, texture and its “normal,” which is used to determine the way the surface of an object is facing. Computers then convert the triangles of the 3D models into pixels, or dots, on a 2D screen. Each pixel can be assigned an initial color value from the data stored in the triangle vertices. Further pixel processing or “shading,” including changing pixel color based on how lights in the scene hit the pixel, and applying one or more textures to the pixel, combine to generate the final color applied to a pixel.

If all of this mumbo jumbo confuses you, don't worry, Disney made a brilliant video breaking down how ray tracing works so that a layman can understand. ENJOY mutha fucka!

 

[M76]

I'm using Iray and it still takes about 300 seconds to render a frame to realatively acceptable quality using two GTX1080 GPUs. So basically we'd need it to be 1000 times faster to get real time.

 

[gamerk2]

I'm using Iray and it still takes about 300 seconds to render a frame to realatively acceptable quality using two GTX1080 GPUs. So basically we'd need it to be 1000 times faster to get real time.

Well, you could use Ray Casting to cut down the processing load, and you can always design around ray tracings limitations if need be. But yeah, we're still not powerful enough to handle real time Ray Tracing at a playable speed.

 

[jnemesh]

Just another API, like physx before it, that attempts to use technology to lock gamers into Nvidia. Hard pass.

 

[Evildead666]

Just another API, like physx before it, that attempts to use technology to lock gamers into Nvidia. Hard pass.

I'm not entirely sure.
It would seem that it requires a Volta GPU, but that could just be because the Tech isn't in any of the older GPU's.
It does mention DXR (DirectX Raytracing) and I think it uses that as a base.
The examples on the Nvidia blog seem to use DXR.
So, It could be 'sort of' "open", as in it will run on all future GPU's, but knowing Nvidia, and its past, there is a very high possibility that it will be closed tight.

 

[DukenukemX]

To make it even less likely all signs point to PS5... XboxNextOne seem to be sticking with AMD.

There's a good chance that AMD is probably ahead of Nvidia when it comes to Ray Tracing, as ATI/AMD has always been ahead of graphics tech compared to Nvidia. Nvidia has a lot of nothing to show for Ray-Tracing, other than some tech demos and empty promises. More likely Nvidia is trying to tie Ray-Tracing to their hardware like they did with PhysX.

Having said that... rumors are MS is planning a streaming only XboxNextOne. Nvidia putting ray tracing hardware into streaming servers, actually sort of does make a ton of sense.

I'd really doubt that Xbox Next One is going to be cloud only. The Xbox brand would officially be dead if they went that direction.

If Nvidia + Sony/MS manage to get a handful of big name AAA titles to go 100% ray traced and 100% streaming. That could actually be something.

It could be, but it won't happen for years.

Stream only ray traced games... might actually be just what MS and Sony need to get die hard gamers to jump into streaming. Fully ray traced streaming exclusives might really appeal. I could see MS going AMD integrated for the next box... and power it all with Nvidia server GPUs.

Keep in mind that Ray-Tracing has always used the CPU instead of the GPU for many reasons. One reason is that often Ray-Tracing uses a lot more memory than what graphic cards have. Ray-Tracing often goes 30GB+ in memory usage, and all that in a graphics card isn't going to happen.

So people with 8 core 16 thread CPU's might be able to do some hybrid Ray-Tracing better than putting all this onto the GPU. GPU's already have a lot going on for it, so why put a burden like Ray-Tracing on it when modern computer CPU's are better suited for it? Modern games hardly make use of todays CPU's, which rarely go beyond 2-3 cores/threads.

For consoles to go streaming might make sense if the hardware isn't capable of it, which it might, but Sony is certainly working with AMD on a Ryzen CPU in their PS5. Maybe the whole reason why Nvidia annouced Ray-Tracing is cause AMD's version is going to use more CPU than GPU, which is something Nvidia can't do on hardware like Tegra. So basically Nvidia is locked out of the console market unless they can come up with a Ray-Tracing GPU only solution. Certainly their desktop GPU's will use quite a bit of the CPU to do Ray-Tracing.

 

[Deleted member 93354]

Great stuff! Nvidia is preparing for the future, once chips move beyond silicon and we get significant performance jumps generation to generation again!

Blah ha ha ha

 

[Deleted member 93354]

To put it in simpler terms: Ray tracing is the holy grail of computer graphics my friend. To illustrate what this process is, think of the objects you’re seeing which are illuminated by beams of light. Now turn that around and follow the path of those beams backwards from your eye to the objects that light interacts with. That’s ray tracing. What makes ray tracing so difficult to process on a cpu is the millions of beams it's shooting out from the camera to objects which are diffused, reflected, refracted, in all different directions. Rendering engines have to calculate each and every single one of those rays, but todays computers can't process them in real time.

Rasterizing, which is what todays video games use, overcome this hurdle by displaying the three-dimensional objects on a two-dimensional screen. It’s fast. And, the results have gotten very good over time, even ray tracing is better. With rasterization, objects on the screen are created from a mesh of virtual triangles, or polygons, that create 3D models of objects. In this virtual mesh, the corners of each triangle (vertices) intersect with the vertices of other triangles of different sizes and shapes. A lot of information is associated with each vertex, including its position in space, as well as information about color, texture and its “normal,” which is used to determine the way the surface of an object is facing. Computers then convert the triangles of the 3D models into pixels, or dots, on a 2D screen. Each pixel can be assigned an initial color value from the data stored in the triangle vertices. Further pixel processing or “shading,” including changing pixel color based on how lights in the scene hit the pixel, and applying one or more textures to the pixel, combine to generate the final color applied to a pixel.

If all of this mumbo jumbo confuses you, don't worry, Disney made a brilliant video breaking down how ray tracing works so that a layman can understand. ENJOY mutha fucka!

I remember the early days of povray. It was command line only. It was only a few simple objects on my first drawing but I was like "wow"

Even full on cinema releases don't use full Ray tracing. They use limited Ray tracing which can still take hours per frame sometimes. That's why they have render farms.

http://hof.povray.org/

 

[GNUse_the_force]

With that music i was expecting Nvidia 'Dawn' dancing in the club with ray-traced lights shining off her tessellated curves. But instead we have some butch dudes dancing in front of a fire naked.

Whatever floats your boat nvidia...


*tongue in cheek puns are apparently bad now.Mkay.

 

[euskalzabe]

With that music i was expecting Nvidia 'Dawn' dancing in the club with ray-traced lights shining off her tessellated curves. But instead we have some butch dudes dancing in front of a fire naked.

Whatever floats your boat nvidia...

I for one would be glad if we stopped sexualizing women in the GPU industry for absolutely no reason. You tell me why a dancing Dawn is good at marketing a piece of electronics equipment that will be used to render, say, Elder Scrolls. Maybe that music makes you think of dancing Dawn because society has, well, socialized you to think that way: thumpy music = sexy women = objects of consumption. And, if butch dudes dancing naked are Nvidia's thing, is there anything wrong with that? Not really.

Let's aim to keep sexism out of our industry... and our forum.

 

[liffie420]

Is ray tracing really that much better?

As other have said yes, ray tracing is probably the single biggest step between the visuals games have now to near cinematic/movie level video game visuals. The problem is ray tracing a scene still entails a MASSIVE performance penalty. Real time ray tracing is still probably a decade out, but we are inching closer. Think of it like this it is in the same vein as the jump from 2d to 3d games, it is that big of a deal. I would hazard to say it will be a bigger jump in video game visuals than probably any other tech in the last 20+ years.

 

[Deleted member 93354]

I for one would be glad if we stopped sexualizing women in the GPU industry for absolutely no reason. You tell me why a dancing Dawn is good at marketing a piece of electronics equipment that will be used to render, say, Elder Scrolls. Maybe that music makes you think of dancing Dawn because society has, well, socialized you to think that way: thumpy music = sexy women = objects of consumption. And, if butch dudes dancing naked are Nvidia's thing, is there anything wrong with that? Not really.

Let's aim to keep sexism out of our industry... and our forum.

Marketing addresses the segment that are more likely to buy their product. Female/homosexual players are not in a majority.

I don't make the rules. Just part of what it is. I could care less personally.

But any ideas or hints of sexuality should be totally left out of games/movies where children are present.

 

[euskalzabe]

Marketing addresses the segment that are more likely to buy their product. Female/homosexual players are not in a majority.

I don't make the rules. Just part of what it is. I could care less personally.

But any ideas or hints of sexuality should be totally left out of games/movies where children are present.

Marketing addresses the ways in which you're socialized. Change those ways, the marketing will change. So, yeah, you DO make the rules, by being part of what it is. You think you don't care, even though you're actively involved in making it what it is.

Sexuality is involved in many children's games still: every time your kid saves princess Peach from her helpless situation, the message that woman = weak = inferior is transmitted. Why can't we save our friend Toad instead? With comments like the one I mentioned earlier, the connection music = sex = woman consumption object is made, reinforcing the woman = inferior stereotype previously learned as a child and taking it further to "superior men can associate women with whatever low-level imagery because they deserve it" mentality that spreads.

Offloading your responsibility in this process to others, such as "marketing", simply shows how complicit you are in the situation. How much power you have. How many rules you perpetuate, over and over, by yourself, with friends, with your kids. Not confronting the problem, is taking a position towards that situation. It always seems an easy, exaggerated notion that affects others, until you realize you're a big component of that problem.

Anyway, I didn't mean to get into a gender tangent here. I thought there was an unnecessarily sexist comment on the thread, and addressed it - if Nvidia prefers butch robot dudes dancing, that's fantastic, not really something to criticize like the comment did VS the absence of female curves. That's all.

 

[GNUse_the_force]

 

[cptnjarhead]

This tech seems more useful for an animated film a this juncture... seeing how much computational power is required. With consoles still in a current cycle, i just dont see this in games any time soon. What are the possibilities of another tech coming along that can simulate/imitate ray tracing with lower cost and better perf? In the end... users dont care whats behind the scenes just how good it looks or imitates real life.

 

[euskalzabe]

View attachment 93217

Not offended, really just observant. Your GIF just brings you even lower. But you do you.

This tech seems more useful for an animated film a this juncture... seeing how much computational power is required. With consoles still in a current cycle, i just dont see this in games any time soon. What are the possibilities of another tech coming along that can simulate/imitate ray tracing with lower cost and better perf? In the end... users dont care whats behind the scenes just how good it looks or imitates real life.

So far, rasterization and raytracing have been the two major rendering methods. It is unlikely that we'll suddenly come up with something else that's worth the effort in the next few years. Nvidia's strategy seems to lower the cost associated with raytracing: they're already doing that with RTX - doing as few rays as possible, then denoising a very noisy unfinished image. If the end result is good, who cares if we get a noisy source? This isn't unlike rasterization, which is always an approximation, and a very inaccurate one at that (certainly less than raytracing which at least follows the laws of physics).

Now advance a few more years, say 3-5 GPU releases, and you can see a future where instead of, say, 16 rays, we can get the same result from 8 or even 4 rays (I don't really know how many they use, but if I recall correctly, I've seen 16 vs 8 in several tech demo videos). Quadruple speed bump for the same quality achieved in not that much time. If we can get to 1 ray per pixel, or even less somehow, the speed increases will keep coming. For example, we can already take this low ray count:

And denoise it to this with Nvidia's RTX:

Check this Nvidia research paper on it for more details.

 

[GNUse_the_force]

really just observant

Or just reading way too much into things.

 

[Dr. Righteous]

I seem to recall a lot of buzz about a guy that has RT RayTracing running on quakeII using a couple HP servers?

 

[euskalzabe]

Or just reading way too much into things.

Well you're the one who mentioned "lights shining off her tessellated curves" and criticized "butch dudes dancing in front of a fire naked". Never mind that they're actually robots, neither men nor women, but I guess I read too much into things, didn't I.

I seem to recall a lot of buzz about a guy that has RT RayTracing running on quakeII using a couple HP servers?

I don't remember anything about HP servers, but I do remember this from last summer:



Could really use some of that RTX denoising!

 

[GNUse_the_force]

Well you're the one who mentioned "lights shining off her tessellated curves" and criticized "butch dudes dancing in front of a fire naked". Never mind that they're actually robots, neither men nor women, but I guess I read too much into things, didn't I.

Yes.
there was no criticism involved you read way too much into things and this is way off topic.

 

[Elf_Boy]

How much, I wonder, can be cooked into existing api's?

like the mini gl wrapper doom type games used to use.

 

[euskalzabe]

Yes.
there was no criticism involved you read way too much into things and this is way off topic.

Ah. So my calling you out on it was way off topic, but your appraisal of women's curves and criticism of butch dudes dancing was on topic for a thread called "Nvidia Is Hiring Ray Tracing Talent to Revolutionize the Graphics Industry". Got it.

Either way, you're focused on that issue, I'm advancing the discussion while I respond to you. In fact, you should check other Nvidia themes in their raycasting/denoising efforts, like this post or their more recent release of Optix 5 optimized for raytracing AND denoising. This is the kind of performance we currently get:

The better denoising gets, the less rays we'll need to use to render, bringing raytracing performance closer to rasterization. It doesn't all have to be real and accurate, as long as it looks good.

 

[DukenukemX]

I for one would be glad if we stopped sexualizing women in the GPU industry for absolutely no reason. You tell me why a dancing Dawn is good at marketing a piece of electronics equipment that will be used to render, say, Elder Scrolls.

Because sexy women are pleasing to the eye? Also, Elder Scrolls with sexy mods is probably another good reason. Why you think so many people are mad about Fallout 76 mods being limited to creation club?

And, if butch dudes dancing naked are Nvidia's thing, is there anything wrong with that? Not really.

There are mods for that too.

Let's aim to keep sexism out of our industry... and our forum.

Sorry, I don't do requests. Lucky for the industry, we have Japan, where things like sexism is just a theory. I'm really surprised SJW's and feminists aren't all up in arms about Japan's media. Everything in their media is sexualized, especially the women.

 

[euskalzabe]

Because sexy women are pleasing to the eye? Also, Elder Scrolls with sexy mods is probably another good reason. Why you think so many people are mad about Fallout 76 mods being limited to creation club?

Not pleasing to the eye for everyone. I guess my GPU doesn't just scream SEX to me. So many spiky metals, that'd surely be painful. I'm much more interested in my GPU's rendering abilities and hardware specs, than connecting it to a sexualized man or woman. I guess raytraced naked avatars are going to be the best thing ever for some people.

I'll just be happy to get good reflections and shadows. Either way, even if it's light use of raytracing for some elements, it's already starting to look tough to discern from reality:

But then again, even non-raytraced characters like the ones from Uncharted 4 are beginning to look crazy good (though not at the same level as raytraced ones):

Man, even thinking that Matrix Reloaded was only 15 years ago, and what we considered amazing then now looks plasticky as hell, and they even had all the time in the world to render this!

 

[katanaD]

Lucky for the industry, we have Japan, where things like sexism is just a theory. I'm really surprised SJW's and feminists aren't all up in arms about Japan's media. Everything in their media is sexualized, especially the women.

LOL yeah, japan is an odd one. woman are highly sexualized, in more then just the media. heck.. who is leading the way in sexbots?



back to the article...

one thing i would like to see, hopefully within 10 years, is ray traced VR. now that would be pretty spiffy

 

[euskalzabe]

one thing i would like to see, hopefully within 10 years, is ray traced VR. now that would be pretty spiffy

Agreed, although we’re barely getting there with big GPUs, it’s going to be forever until we have standalone VR sets with the power to raytrace. Otherwise, if we’re just plugging the sets to the pc, then as soon as our GPUs get decent at it, so will VR sets driven by them.

 

[Dr. Righteous]

Well you're the one who mentioned "lights shining off her tessellated curves" and criticized "butch dudes dancing in front of a fire naked". Never mind that they're actually robots, neither men nor women, but I guess I read too much into things, didn't I.



I don't remember anything about HP servers, but I do remember this from last summer:



Could really use some of that RTX denoising!

Na, that wasn't it.
This was around the time frame of the Intel Larrabee and it was rumored that it had the potential for real time ray tracing. The demo looked impressive, with chrome like reflections. I recall the guy that put it together was hired by Intel (or someone) as a developer.

Found a link.

 

[euskalzabe]

Whoa, that does look good, even by today's standards. Which only confirms what a big step raytracing can be for visual quality. A fully raytraced Assassin's Creed or Tomb Raider? Yes please. Then again, a raytraced Doom may be too much for my stomach

 

[Flexion]

Because sexy women are pleasing to the eye? Also, Elder Scrolls with sexy mods is probably another good reason. Why you think so many people are mad about Fallout 76 mods being limited to creation club?

View attachment 93229


There are mods for that too.
View attachment 93230

Sorry, I don't do requests. Lucky for the industry, we have Japan, where things like sexism is just a theory. I'm really surprised SJW's and feminists aren't all up in arms about Japan's media. Everything in their media is sexualized, especially the women.

View attachment 93231

Can I ask you to post 10 more of these juicy images to trigger that dude even more? XD

 

[Deleted member 83233]

Very exciting. I remember when it took a week or longer to render a single frame of some abstract with chrome spheres and such

I've done the very thing on my Amiga 500 using Sculpt Animate. One chrome sphere on a checkerboard with blue sky took about 3 days.

 

[Stimpy88]

The idea of an affordable graphics card, being able to produce ray traced scenes comparable or better than today’s aaa titles at 4K, is a dream that is many, many years away.

And remember, we are talking about nVidia here, if they bring this to market, it certainly will not be ‘for the gamers’, unless you have very deep pockets, then where will the software be for it?

I have no doubt that high detail, high resolution, real-time 60Hz minimum, ray traced games are the holy grail, and will come to us all, but it will not be anytime soon, at least 10 years away.

And any talent that nVidia have hired over the last 12 months or so will be years away from seeing the fruits of their labour in silicon.