Current state of PhysX? Is it well and truly dead?

Discussion in 'Physics Processing' started by NoxTek, Apr 22, 2018.

  1. NoxTek

    NoxTek The Geek Redneck

    Messages:
    9,269
    Joined:
    May 27, 2002
    I've come to a point where I'm fortunate enough to have several 'back up' GPUs at my disposal should my Asus Strix GTX1080 fail at a time when crypto miners have run the prices up to astronomical levels. I've got a couple of GTX 660s, a GTX 770, a GTX 780, a GTX 980, and I think even a lowly GTX 560 floating around here now.

    So I started considering dropping one of the above extra cards in my machine as a dedicated PhysX card and I thought surely the landscape has evolved in the few years since I last took a look. Seemingly nope. The last AAA title that had PhysX support appears to be Fallout 4, and before that really it's just all of the Batman Arkham games and the Borderlands Sequels with a bunch of no-names sprinkled in.

    Why the hell has PhysX stagnated so much? When it's done right it really does add awesomeness to a game (see: Batman Arkham Asylum/City/etc). Is it that hard to code for? Does nVidia charge an exorbitant licensing fee or something?

    Meanwhile I'm still pondering throwing one of these extra cards in just to replay the few titles that do support it. Or maybe somehow offload streaming / encoding to the secondary GPU....
     
  2. cybereality

    cybereality [H]ardness Supreme

    Messages:
    4,377
    Joined:
    Mar 22, 2008
    It's still alive, in some sense, but the GPU accelerated portion is not widely supported. On the CPU side, many popular engines do use PhysX (both Unity and Unreal do).

    The basic issue is not the cost. It's free for PC, but I believe there is an extra fee for consoles (not sure if this changed).

    Mainly the problem is that the GPU acceleration only works on Nvidia cards, and big engines like Unity and Unreal are hesitant to integrate proprietary features into their code. In that same vein, developers are not eager to spend time and money on a feature only a portion of their customers can use (even if Nvidia is fairly dominate on PC). Some studios have still went with GPU PhysX, maybe due to relationship with Nvidia, or because they want to push the envelope, but it's far and few between. The situation will likely not get any better, and engines like Unreal have their own GPU particle physics now, leaving developers with less reason to integrate PhysX.

    That said, some older games can still support it, and it's awesome when it's working. I had a great time with Alice Madness Returns and Mirror's Edge, and there are probably other games that are worth going back for. I just wouldn't expect many more titles given the current situation.
     
    Aenra likes this.
  3. Brian_B

    Brian_B 2[H]4U

    Messages:
    2,720
    Joined:
    Mar 23, 2012
    It gets GPU support when nVidia decides to pay a developer to implement it.

    Similar to Gameworks.
     
  4. Aenra

    Aenra Limp Gawd

    Messages:
    193
    Joined:
    Apr 15, 2017
    Did you mean Hairworks?
    I won't have you slandering technological marvels and their unprecedented effects just for fun.

    #hairmatters

    Ironically, i'm bald, LOL
     
    Flexion and R_Type like this.
  5. Brian_B

    Brian_B 2[H]4U

    Messages:
    2,720
    Joined:
    Mar 23, 2012
    Hairworks is a subset of Gameworks. Technically, PhysX is also now under the Gameworks umbrella.

    Not to be confused with AMD's TressFX, which is AMD's version of Hairworks.
     
  6. Aenra

    Aenra Limp Gawd

    Messages:
    193
    Joined:
    Apr 15, 2017
    Now i have to wonder if your notion of irony is truly up there, or if you mistook my jesting for an actual argument.
     
  7. Brian_B

    Brian_B 2[H]4U

    Messages:
    2,720
    Joined:
    Mar 23, 2012
    I think I understand the meaning of irony but if there was a joke there I apologize, I missed it and thought you were trying to make a correction on my part.
     
    Aenra likes this.
  8. Aenra

    Aenra Limp Gawd

    Messages:
    193
    Joined:
    Apr 15, 2017
    Not in the slightest! And no worries :)
     
  9. I remember when PhysX first came out, got’s me an Ageia 128 megabyte PhysX card a few weeks before they went on sale.

    The only games that had it at that time was ghost recon and a few apps from ageia and a game, forgot the name.

    The fan died and I sent it back to bfg, this was just as they were folding, they sent the card back unfixed.

    I am sure I could find a fan or ghetto mod it but when the fan broke physx was basically dead.

    I still have the card, maybe someday it might be worth some money to some collector ?
     
    Flogger23m likes this.
  10. MyNameIsAlex

    MyNameIsAlex Limp Gawd

    Messages:
    313
    Joined:
    Mar 10, 2019
    Why sell the card when you can frame it as a reminder about early adopting technologies
     
    T4rd and Meeho like this.
  11. JVene

    JVene n00b

    Messages:
    18
    Joined:
    Apr 25, 2019
    I thought I'd chime in, because I'm a developer and know PhysX fairly well (not, by far, an expert on the product itself, but I use it and I've developed in C++ for various industries for decades).

    It's far from dead. Factually, it's about to experience quite a resurgence.

    NVidia recently released the product as open source, though that only impacts versions 3.4.2 and 4.x (now at 4.1), and the license excludes consoles (though the source is applicable to console software advancement, a title can't be released on console platforms using the open source license alone).

    The Unreal engine uses PhysX by default. Unity just upgraded their implementation in their 2019.1 release to PhysX version 4.1. Any game built on these engines most likely uses PhysX without making that clear.

    I can predict with about 85% confidence that the GPU implementation in the PhysX code will be augmented to support either OpenCL or some other GPU compute tech. NVidia never had an interest, but the "public" of developers include many (myself among them) who can and (maybe not me) will deploy GPU implementation on non-NVidia devices. I assume a number of corporations would throw resources at this, from Sony to any/all AAA publishers.

    The recent version includes new joint types (articulations), that are far more stable, promising much better automobile and robotics simulation. Performance has ramped over over the last several versions, as has stability (which has a dual meaning in a physics engine, but here I refer to the software's reliability and resource utilization, especially RAM).

    There are likely a lot more titles that used PhysX than you're familiar with, as many might not claim it specifically over the game engine used (incorporating PhysX somewhat anonymously).

    PhysX isn't simple in the sense that it is a physics engine, where complexity is just part of the deal. It is, however, well written (though it shows it's roots from many years ago compared to modern C++ code), well documented and rather comprehensible. If one uses a good game engine, the use of PhysX is almost hidden from view. Physics can be "scripted" or "modeled" in those engines. For example, in Unity, one can import a mesh, simplify a version of it for the physics simulation, set mass, friction (choose a physics material), and suddenly it's a rigid body object that reacts in the scene. One hardly knows the physics engine is actually there, except that objects are colliding, bouncing, reacting to gravity, etc.

    That said, the more complex one's design, the more work involved. Try to create a robotic arm, for example, and you're in for a winding road with any physics engine (creating child links, adjusting motor powers, setting angular limits, solving for inverse kinematics in some cases). The old joints in previous versions of PhysX were simply not stable enough to create multi-jointed robotic arms without a LOT of effort. That is not limited to robots alone - the same problem happens if you attach a trailer to a truck. Even a simple door can break its hinges if it is hit hard enough (and where that should not happen, it's nearly impossible to prevent).

    The new joints (articulations), however, are ideal for this. The same issue can impact vehicles (beyond the "standard" vehicle model provided). If you make a car that drives on a paved road, you're fine. If you want to make a "monster truck" that drives over objects, forget it. The vehicle's standard approach does not really account for a car where the tire touches a wall before the vehicle does. The system really only considers the patch of the tire that contacts the ground. Hit anything like a bump or a rock and that's like driving a shopping cart, not a car, with long legs and 1" wheels at 90 mph (can't even drive that over a door jam). The tires you see are just visual representations - they're not what the physics engine uses.

    That said, you have a good physics engine right there, so you can fashion traction aware tires that are actually round (not just a ray cast toward the ground), and simulate a "monster truck" or a tank or whatever. For enthusiasts, students and amateurs that would seem like a lot of work, but professional development teams expect it, and there are products one can add on to either the game engine (Unity) or PhysX itself to help streamline the concept. Many are "in house" though.

    Havok remains about the last of the "big ticket" engines left. PhysX was, once. Havok has a few advantages, but the price may not be worth it to many. Havok's primary advantage is determinism. Simply put, applicable to network (Internet) play, it means that all viewers/players will see the same simulation given the same inputs. PhysX is only partially deterministic in this sense. That's less of a problem than an amateur or student might think, but it does make networked game development simpler. In PhysX you can arrange for it, especially with immediate mode. This means the code runs a simulation on general objects that aren't particularly important to gameplay, then isolates key objects for simulation in a separate group under a subset of physics intended specifically for those objects/characters. This subset can be made deterministic (more than the rest of the game), allowing for synchronization on key objects.

    Unity is building their own physics engine based on partnership with Havoc. Soon they'll offer a choice of using PhysX (built in, as they have for years), Unity's own engine or the full Havok (at a price). It is part of their push for their "new" tech called "ECS", which isn't fully baked and won't be in use for at least a year or more. It's in alpha at present.

    CryEngine has their own physics engine (never used it), if I recall correctly. Amazon bought complete rights to CryEngine, released that open source, and thus allow for multiple physics engine options (if you work at it). Amazon calls their game engine Lumberyard.

    I bring these up because they are, at present, about the entire collection of competitors to PhysX outside of the open source engines, like Bullet, Newton and ODE. Newton did seem to die off, but appears to be forked or picked up by another team. ODE hasn't changed in years. Bullet, on the other hand, is still in constant development, and recently added (perhaps still beta level) a GPU compute version. In practical use, Bullet can seem on par with PhysX (for a while it was even better, before PhysX 3.x), and some large projects have used Bullet. Contributors to Bullet included Sony and IBM (limited parts). At this point, however, Bullet will have to advance to catch up to the state of PhysX 4.1.

    I'd have to guess, but I doubt anyone has seen a game with PhysX more current than about version 3.3 (I don't think the newer versions have been out long enough for a new AAA title to be released with it).

    Unity, for example, places PhysX behind a barrier. Script is written in C#, and physics is presented as a C# interface. The native C++ code of PhysX can't even be seen or accessed. It could be any physics engine inside and you wouldn't know. This also means the application is entirely dependent on Unity to get the configuration right. That's ok at release time, it likely will work well, but in a distant future (2 years), products may exist that Unity's 2 year old code can't configure, and so resources go unused unless the vendor releases an update using a newer release of Unity (or at least one that's patched with an update).

    Unreal is scripted in C++ (they have lots of options, but the point is that PhysX is not on the other side of an impenetrable barrier). Same for Lumberyard (C++).

    That said, even for Unity a developer can ignore the PhysX engine incorporated in Unity, and use the latest version in a C++ native module. I do it. NVidia is currently posting an alpha project for that, and there's a C++ library to support full scripting of Unity projects in C++, meaning that with enough work and determination one can bypass the barrier Unity puts up between code and PhysX. At that point, however, the developer is taking the responsibility of configuration. If one inquires deeply about capabilities, and connects to resources with liberal checks and options, it can be quite flexible. If they use the example code initialization, instead, you're getting a single thread PhysX implementation with a basic GPU (if any).

    Depending on the application using PhysX, the configuration can be tricky. Older products that one might toss into a new computer may not recognize the GPU, or may not configure adequately to use resources not known at the time the original product was built. For example, some might assume no more than a quad core CPU, and can't enable use for an 8 core machine. This is a key point because while you assume PhysX is running on the GPU, what you might not realize is that only SOME of the work is run on the GPU (the bulk calculation work). There is still a significant amount of work required on the CPU to connect the result of physics calculations to the visual models of the game, stream data, provide user input, etc. That work can be threaded, but if the product never envisioned an 8 or 16 core CPU, it may not even work as well as it did on the older hardware the code understands. Even drivers and GPU compute unit capabilities that don't match older versions (even though the newer one is vastly superior) may go unused because the software doesn't recognize them, and ignores them. This is less about PhysX itself and more about how PhysX was incorporated into the product.

    On another front, however, this can be due to different ABI's (the binary code for a GPU). That is unique to each model. This doesn't work like AMD/Intel CPU's, where the ABI (binary code) is nearly identical on either brand. Even within a single brand, like NVIdia, the binary generated for the GPU can differ vastly on different models/generations. This means the compiler is ON the GPU card. It isn't a great one, though. There are better "off line" compilers that aren't on the GPU card. Those, however, can't possibly be incorporated into an older product. This means only VERY compatible code can recompile on a GPU of a newer model than known during development, and then only with the ON CARD compiler, not the best optimizing compiler for that GPU.

    In other words, like I said when I opened up, it's complex in the sense that physics is complex to deploy in a game. That's just the cost of that fact.

    PhysX, on the other hand, JUST NOW got open sourced (well, ok, it was a few months ago, but no one has ever seen a released product of wide acclaim that has used this new version yet).

    Expect the new open source release to spawn a rapid evolution of PhysX, and in far more directions than anyone ever expected NVidia to support or allow.
     
  12. cybereality

    cybereality [H]ardness Supreme

    Messages:
    4,377
    Joined:
    Mar 22, 2008
    Thanks for sharing. I think that all makes sense (I'm a developer as well).

    The confusion is that Nvidia (at one point) was heavily marketing PhysX as a GPU solution, even to the point of allowing customers to add second or third cards (in an SLI system) to power PhysX.

    While the software (CPU) version of PhysX was and still is popular, it is the hardware (GPU) acceleration that most people associate with Nvidia and PhysX. On this end things are very well dying or dead.

    The problem here was two-fold. One, it was proprietary and locked to Nvidia brand hardware. Many developers did not want to exclude parts of the market, or spend time/money on features many people couldn't see (for example AMD or Intel GPU users).

    Secondly, the usefulness of hardware PhysX was vastly over-hyped. In reality, it couldn't be used to make games with more complex gameplay-affecting physics as it was too costly to read data back to the CPU.

    So you end up with generally cooler effects like capes flowing in the wind, or explosion debris, or glass breaking etc, but not anything that can enable more complex games (such as maybe a Rube Goldberg kind of game).

    As we have seen time and time again, these vendor locked proprietary features typically don't become standard, and usually only survive so long as the owner invests in the software (e.g. by sponsoring games, etc.) not because of natural market forces.

    And it is sad. Physics is games could be somewhere completely different today if things had evolved in another way.
     
    lostin3d and Brian_B like this.
  13. profiled

    profiled Limp Gawd

    Messages:
    151
    Joined:
    Feb 20, 2018
    stronger than ever!
     
    cybereality likes this.
  14. JVene

    JVene n00b

    Messages:
    18
    Joined:
    Apr 25, 2019

    These are key points. The same game on a comparable AMD GPU would be described, infamously, as running "slow as a dog" because users knew PhysX couldn't run on that GPU.

    Here's where we diverge a bit. This issue was much worse on the older bus interfaces, and made worse in older API's (older OpenGL, DX9 to DX11). During that era, too, the GPU's were less powerful, and of course the CPU's were less powerful, and RAM itself was slower (this reaches back to the era of DDR2 or before on the CPU).

    My point is that I'd have to append the caveat that the cost of moving data from GPU back to CPU was high on affordable GPU devices. NVidia was trying to use the "advantage" of GPU compute resources to sell high end (expensive) cards. It almost worked, because this "floated" NVidia from a really tough PR era (the melting GPU's of XBox 360, for example) to a point where the are a lot of enthusiasts that insist on NVidia hardware.

    Edit: I thought I'd add one more point about bus transport cost. It wasn't just the bus cost in the older API's, it was also the fact that data had to be copied before calculations could begin in the GPU. These older interfaces present the GPU as a server, with the CPU as a client. The newer API's allow the GPU to read RAM from the motherboard (with locks and permissions to be applied). This lowers the cost because the data doesn't have to be copied before calculations begin, the GPU can rip through system RAM to feed the data into the compute engine, and then write the results out to system RAM without having to copy all the data first. Even with the GPU is over a bus (a card), the bus is running upwards to 30 GBytes per second (a speed similar to system RAM itself). The bus is no longer the real issue as a result, its the copy associated with the older interfaces. With that copy removed in Metal, DX12 and Vulkan, the result of GPU calculations are available much sooner than in the older API's that require that data to be copied (even to get the result set out of the GPU memory into system memory).

    This is why it can be observed that one new GPU can be faster than two older GPU's, assuming that the software properly coordinates timing and balance of compute power between graphics and physics.

    If you look at the source of PhysX, you can see that what is put on the GPU are the large data calculation functions. They're prefixed with macros indicating they are "CUDA_CALLABLE". When building CPU versions of the engine, these functions route to a CPU implementation, but when building for GPU they route to a CUDA version. Most of the engine continues to operate on the CPU. One might say the GPU code is little more than a math library attached to the engine, but at key performance bottlenecks. When those operate in the new APIs, the GPU operates more like a vector compliment to the CPU's vector engine than the older cards could. I long ago predicted that what we now consider a separate GPU compute engine would eventually be folded into the CPU's vector processing system with much higher parallelism than CPU's made today. Perhaps I'm dreaming some, but AMD is closer to that than anyone already - just not in CUDA.

    You can measure the performance difference, and what you get is an order of magnitude higher performance on those functions in the GPU build, despite the point of data traffic over the bus. That data is packed exactly to transport quickly, not like the rest of the scene data which are more complex data structures.

    I can see at most around 20K independent, awake objects colliding and moving due to momentum on a CPU based PhysX simulation (type 4 core Intel 4Ghz device) at 30 FPS, but upwards of 100K on the GPU implementation at the same FPS.

    I do think, though, that physics modeling is not well attended in many titles. I believe, without full evidence other than my own use of various physics engines, that some developers overstuff the engine needlessly. For example, I've never found anyone using the concept of LOD in physics, only graphics. In my own work I've used multiple engine groups, so that more distant objects are simulated in a separate group run at a lower frame rate (and therefore consuming lower resources, while still keeping real time). When objects are far away you can't see and interact in such a way as to witness any difference between 60 FPS and 30 FPS (or even 20 FPS), because many of them don't move a fraction of a pixel over several frames.


    This is where we're in full agreement, and your point is exactly spot on.

    At this point, though, the PhysX code is no longer proprietary. It could be nearly trivial to implement an OpenCL or Metal based implementation.

    In the modern API's (Vulkan, Metal, DX12) there is less separation (less of a client/server model) to the GPU compute resources. With high end GPU's embedded we may well see an era, soon, where GPU compute units are no longer called GPU's - they'll be more like an ultra extension of the vector processing system. From a technical viewpoint, we really only need the last 2D (fragment) stage of the video pipeline to be aimed right at the output signal to the displays, everything else going on in the display pipeline is math, and so is all other compute work required of the GPU to this point.
     
    Last edited: Apr 27, 2019
    cybereality likes this.
  15. HiCZoK

    HiCZoK Gawd

    Messages:
    818
    Joined:
    Sep 18, 2006
    I just wish there was a way to run that ancient Cellfactor demo. Sure You can launch it with physx disabled in .ini but I want full hpysx!
    And I always wanted to run "Hangar of Doom" demo but it never launched :(
    Physx came, went and I never got a chance to play those
     
  16. cybereality

    cybereality [H]ardness Supreme

    Messages:
    4,377
    Joined:
    Mar 22, 2008
    The Cellfactor demo was awesome. Way better than any other physics demo that has come out, before or after.
     
  17. lostin3d

    lostin3d [H]ard|Gawd

    Messages:
    1,995
    Joined:
    Oct 13, 2016
    First time I had a PhysX game was one of the Sherlock Holmes games. Still got it and it even has the Ageia installer on it. Had it barely a month before Nvidia bought them out. I thought the idea was awesome and it became a feature I sought after when getting games.

    JVene & cybereality Thanks for the detailed explanations. I hope the GPU accelerated does make a comeback. It's been pretty depressing since 2015 watching that go away and CPU take over. Had great success until then with my dedicated cards but these days even when I retest my old games that supported it they don't seem to work anymore. NV PhysX indicator still shows GPU for those games but AB shows 0-1% usage and honestly zero performance gain over not using.
     
    cybereality likes this.
  18. daglesj

    daglesj [H]ardness Supreme

    Messages:
    5,050
    Joined:
    May 7, 2005
    So we've probably been using it in quite a few places but just not noticed cos this is 2019 and not 2007?
     
    lostin3d and JVene like this.
  19. bigdogchris

    bigdogchris [H]ard as it Gets

    Messages:
    17,853
    Joined:
    Feb 19, 2008
    With multicore CPU's finally seeming to be commonplace, I expect more CPU accelerated physics than I would GPU accelerated.
     
  20. JVene

    JVene n00b

    Messages:
    18
    Joined:
    Apr 25, 2019
    While I agree we'll likely see a lot of CPU based physics in lots of games, I point out that while CPU core counts grew from dual, through quad, to 16 cores or more, the GPU has also grown from a few dozen to a few thousand at a time, which means some physics simulation work just can't compare between CPU and GPU accelerated implementations.

    That said, the GPU is usually busy already, so it is up to a matter of what the simulation requires vs what the rendering requires.