NVIDIA SlowMo Even Better than Real SloMo

Discussion in '[H]ard|OCP Front Page News' started by Kyle_Bennett, Jun 18, 2018.

  1. Kyle_Bennett

    Kyle_Bennett El Chingón Staff Member

    Messages:
    52,255
    Joined:
    May 18, 1997
    NVIDIA is using its neural network to interpolate frames in video to produce smoother slow motion playback, and quite frankly the results are tremendously impressive. It even gets more impressive when NVIDIA takes slomo video and made it even slomoer. It has to be some of the most slomoest slomos ever slomoed. Thanks cageymaru.

    Check out the SloMo.

    Researchers from NVIDIA developed a deep learning-based system that can produce high-quality slow-motion videos from a 30-frame-per-second video, outperforming various state-of-the-art methods that aim to do the same.
     
    cdabc123 and SPARTAN VI like this.
  2. PantherBlitz

    PantherBlitz Limp Gawd

    Messages:
    276
    Joined:
    Apr 14, 2011
    Great, now replay review in sports will take even longer.
     
  3. DigitalGriffin

    DigitalGriffin [H]ardness Supreme

    Messages:
    4,387
    Joined:
    Oct 14, 2004
    How is this any different than TV interframe interpolation? Analyze the motion vectors and interpolate.
     
  4. Verge

    Verge [H]ardness Supreme

    Messages:
    5,757
    Joined:
    May 27, 2001
    I've seen similarly impressive stuff from an old premier plugin that did the calcs offline. Took a while, but looked great.
     
  5. Jovian

    Jovian Limp Gawd

    Messages:
    323
    Joined:
    Jun 8, 2004
    Isnt this just more FPS?
     
  6. katanaD

    katanaD [H]ard|Gawd

    Messages:
    1,243
    Joined:
    Nov 15, 2016
  7. scojer

    scojer 2[H]4U

    Messages:
    3,365
    Joined:
    Jun 13, 2009
    Ok, that's really cool.
     
  8. Nenu

    Nenu Pick your own.....you deserve it.

    Messages:
    17,908
    Joined:
    Apr 28, 2007
    You could have answered that yourself.
    Its better.
    Processors in TVs cant do much.

    I use SVP 3.1 for realtime video playback.
    Its great with a fast CPU when set to max smoothness and artifact masking to strongest.
    It will be interesting to compare if we get to use the NVidia tool ourselves.
     
  9. haste.

    haste. [H]ard|Gawd

    Messages:
    1,358
    Joined:
    Nov 11, 2011
    Pretty neat little teaser video.

    Depends on the TV. My JS9500 does fantastic. Can take horrible content and post process to make it at least acceptable if not very good. On the other hand my LG OLED which has a much better panel with a much shittier "computer" basically feeds exactly whatever content is to the panel. UHD Blu-ray it smokes the Samsung, but anything under 720p it looks awful and can't even properly stretch the image. The Samsung was a bit more expensive, but there are a number out there that can do some pretty impressive processing, just comes at a cost (wasn't willing to spend more than $3k on a secondary TV or would have went with a flagship OLED)... Much like spending money on a nice GPU.
     
  10. bugleyman

    bugleyman [H]ard|Gawd

    Messages:
    1,040
    Joined:
    Oct 27, 2010
    Seems interesting for entertainment purposes, though I wonder if the tech becoming ubiquitous could eventually lead to misapplication. I'm not sure I trust John Q. Juror to grasp the concept of interpolation.
     
    PestXs likes this.
  11. bugleyman

    bugleyman [H]ard|Gawd

    Messages:
    1,040
    Joined:
    Oct 27, 2010
    Well played. For an moment I thought you were serious. :p
     
  12. Tak Ne

    Tak Ne [H]ard|Gawd

    Messages:
    1,123
    Joined:
    Jan 28, 2008
    When I saw the first part of this I thought NV was saying "this 30fps video slowed down doesn't look as good as one we took at 240fps and slowed down even more" :D
     
  13. DigitalGriffin

    DigitalGriffin [H]ardness Supreme

    Messages:
    4,387
    Joined:
    Oct 14, 2004
    The point is NVIDIA makes it sound revolutionary when in fact they are doing the same thing TV's have been doing for you, with perhaps a slight advantage in they can analyze a couple frames ahead to perhaps create a better motion interpolation. But that isn't real time. TV's can't do that because they have to sync audio sometimes with external drivers so additional delay isn't an option.

    Sorry not impressed.
     
  14. Kyle_Bennett

    Kyle_Bennett El Chingón Staff Member

    Messages:
    52,255
    Joined:
    May 18, 1997
    And what TVs are doing is not anywhere on this level of definition or granularity.
     
  15. geok1ng

    geok1ng [H]ard|Gawd

    Messages:
    2,016
    Joined:
    Oct 28, 2007
    for some content, the results looked good, for others, much detail is lost. balloon jump was turned into a blurry mess.
     
  16. Ehren8879

    Ehren8879 Little Bitch 3

    Messages:
    4,259
    Joined:
    Sep 24, 2004
    Here's a question, can this work on video games? can it be done "on the fly" with low input lag?

    Assuming it requires less hardware resources than just rendering the effective FPS.
     
    Poseur likes this.
  17. Poseur

    Poseur Limp Gawd

    Messages:
    305
    Joined:
    Oct 7, 2009
    Someone at nVidia PR deserves a cookie. Share it with the programmers that actually did something useful though.
     
  18. Nenu

    Nenu Pick your own.....you deserve it.

    Messages:
    17,908
    Joined:
    Apr 28, 2007
    It requires major hardware to do well at 1080p. And thats not to the standard NVidia are touting.
    I cant see it being feasible when hardware is already being punished, especially at higher res.
     
  19. IcePickFreak

    IcePickFreak Gawd

    Messages:
    967
    Joined:
    Dec 1, 2010
    The tech looks good, and I know you guys didn't use it, but 2018 is already burning out the "AI" term.

    That said, I had to do a double take here - thought I was watching a man in a thong for a second.
    manthong.jpg
     
  20. TheHobbyist

    TheHobbyist Hugs Hard Johnnies [H]ard

    Messages:
    444
    Joined:
    Apr 8, 2008
    Yo Dawg, I heard you liked SloMo...
     
    hurleybird likes this.
  21. aaronspink

    aaronspink [H]ard|Gawd

    Messages:
    1,510
    Joined:
    Jun 7, 2004
    I don't know, looking at the videos, they seems to be losing a whole lot of definition. The slow mo guys examples certainly didn't shine any good light on the technology: look its slower and a shitload blurrier.
     
    atmartens and PestXs like this.
  22. aaronspink

    aaronspink [H]ard|Gawd

    Messages:
    1,510
    Joined:
    Jun 7, 2004
    doesn't do anything in video games, might as well just display the same frame 4x (like a 2x better AFR SLI)
     
  23. Kyle_Bennett

    Kyle_Bennett El Chingón Staff Member

    Messages:
    52,255
    Joined:
    May 18, 1997
    Ok
     
  24. Navilor

    Navilor Limp Gawd

    Messages:
    163
    Joined:
    Jan 12, 2009
    velusip likes this.
  25. Hielo_loco

    Hielo_loco [H]Lite

    Messages:
    80
    Joined:
    Jan 27, 2015
    Could it be that whatever program nvidia is using creates a sort of dynamic vector field off of the bitmap data in the video, and then interpolates from that instead of from the frame as a whole? I guess that would be more suited for a gpu to handle than full frame interpolation.
     
  26. hondaman

    hondaman [H]Lite

    Messages:
    64
    Joined:
    Jul 31, 2004
    Oh I think I could have produced a waaaaaaay better slo-mo demo starting with better content.
     
    Kyle_Bennett likes this.
  27. DigitalGriffin

    DigitalGriffin [H]ardness Supreme

    Messages:
    4,387
    Joined:
    Oct 14, 2004
    Women on trampolines don't count.
     
    Drewis and Kyle_Bennett like this.
  28. Stimpy88

    Stimpy88 [H]ard|Gawd

    Messages:
    1,072
    Joined:
    Feb 18, 2004
    All it’s a shame that nVidia never share tools like this. They have shown off some lovely demos of stuff over the years, and they never share, or open source it for download.

    At least we have AMD, and they share many things that benefit others, for free.
     
    brucethemoose and c3k like this.
  29. SomeoneElse

    SomeoneElse [H]ard|Gawd

    Messages:
    1,311
    Joined:
    Jan 16, 2007
    Although its cool to see it slowed down.
    Being that my degree was in digital media and design, watching this didn't really impress me that much because I could tell it was just adding frames in between which didn't exists. The tech it self is impressive to create the "extra" frames, but the results (to me) weren't that great, because the level of detail is lost due to it not being true to frame capture. This is why the slomo guy capture stuff at sometimes 100K fps.....they capture the detail frame by frame. Clearly in this video example the details are muddied versus what they had with raw capture.
     
  30. Trepidati0n

    Trepidati0n [H]ardForum Junkie

    Messages:
    12,112
    Joined:
    Oct 26, 2004
    It is the only thing that counts!
     
    hondaman likes this.
  31. brucethemoose

    brucethemoose Limp Gawd

    Messages:
    181
    Joined:
    Mar 3, 2018
    All interpolation implementations that I know of interpolate between frames. That means interpolation would introduce input lag into games, no matter how fast your algorithm is.

    However, VR does something kinda similar with frame warping, and it only works with the previous frame. It's not actually interpolating moving objects in the frame IIRC, but the idea is similar.
    https://xinreality.com/wiki/Asynchronous_Spacewarp


    There was some discussion about this idea in the ReShade forums, but nothing ever came of it: https://reshade.me/forum/suggestions/3826-frame-interpolation-shader-for-reshade

    This ^

    Speaking of AMD, they did something similar years ago with FluidMotion. Unless they recently ripped it out of the driver, you can do it in real time, on your AMD GPU, right now:



    It totally flew under the radar though, even among AMD fans. I think most people aren't really that concerned about video quality, even when major breakthroughs are happening, and the tech won't work on the big streaming video platforms since they all use crappy, closed browser renderers.
     
    Last edited: Jun 18, 2018
    Stimpy88, PestXs and Kyle_Bennett like this.
  32. deaedius

    deaedius Gawd

    Messages:
    756
    Joined:
    Jun 18, 2014
    Usually the higher the frame rate the lower the resolution especially if hitting the really high frame rates in the high hundreds or thousands anyway, nothing new here other than it is pretty sick to take slow mo and make it slow mo'er!
     
  33. Kardonxt

    Kardonxt 2[H]4U

    Messages:
    2,545
    Joined:
    Apr 13, 2009
    The Nvidia assisted slow mo looks odd to me. Like you can tell they are digitally altered images as they look a bit CGI. I wonder if it's just my brain being crazy or if I would see it in a double blind comparison.
     
  34. MaZa

    MaZa 2[H]4U

    Messages:
    2,431
    Joined:
    Sep 21, 2008
    Doesn't setting Artifact Masking to strongest almost disable the whole motion interpolation? In my eyes when I set it to strongest all fluidity is gone. I always used no artifact masking and lowest setting on motion interpolation strength. Slightly smoother but not artificially so and does not hurt the oh so precious "film look". (I'm slightly sarcastic on that one)
     
  35. StoleMyOwnCar

    StoleMyOwnCar 2[H]4U

    Messages:
    2,062
    Joined:
    Sep 30, 2013
    I mean... I think it's basically creating data. Interpolation involves adding information where there previously wasn't any. Whatever the algorithms are, it's basically artificial footage. Once it starts getting up to a certain percentage of the footage, it basically gets to the point where more of the footage is fake than real.
     
  36. NeghVar

    NeghVar 2[H]4U

    Messages:
    2,187
    Joined:
    May 1, 2003
  37. Kyle_Bennett

    Kyle_Bennett El Chingón Staff Member

    Messages:
    52,255
    Joined:
    May 18, 1997
    Well at least we now know you did not watch the video in the OP that thread is exactly about.
     
    tetris42 likes this.
  38. Nenu

    Nenu Pick your own.....you deserve it.

    Messages:
    17,908
    Joined:
    Apr 28, 2007
    It isnt quite as smooth but its still a lot better than without.
    Artefacts trailing things that move across the screen stand out too much for me.
    Masking on max gets rid of the artefacts.
     
  39. Azphira

    Azphira [H]ard|Gawd

    Messages:
    1,652
    Joined:
    Aug 18, 2003
    Now if only we could get nvidia to use a neural network to release the next gaming card 4x faster.
     
  40. tetris42

    tetris42 2[H]4U

    Messages:
    3,868
    Joined:
    Apr 29, 2014
    Why?
     
Tags: