Final Fantasy 7 Textures Upscaled With a Neural Network

Discussion in 'HardForum Tech News' started by AlphaAtlas, Jan 7, 2019.

  1. AlphaAtlas

    AlphaAtlas [H]ard|Gawd Staff Member

    Messages:
    1,713
    Joined:
    Mar 3, 2018
    An enterprising modder managed to upscale Morrowind's textures with deep learning algorithms last year, and today, PC Gamer spotted a mod that does the same thing in Final Fantasy 7. Like the Morrowind mod, this particular modder is running ESRGAN to upscale the game's textures and backgrounds 400%, with far better results than you'd get with a conventional filter. The mod is still a work in progress, but you can check out some of the shots in the project's thread.

    CaptRobau mentions using Gigapixel to upscale the images, which apparently costs about $100. But the ESRGAN algorithm itself is open source, and a pre-packaged implementation I have installed on my PC right now can run ESGRAN (or any of dozens of other neural net upscalers) on images (or videos) for free. So if you're thinking about upscaling your own images, I'd think twice before paying for Gigapixel's software.

    N4CR , your (obscene) prayers have been answered.

    As I mentioned, batch upscaling/denoising can be done for free, and it can be done with (IMO) better algorithms than ESRGAN as long as you have sufficiently powerful GPU(s). It's kind of a pain to setup, but if anyone's interested (or just wants to run it as a benchmark/stress test), I can try to make a zip file that'll run this kind of thing with minimal effort.
     
    Last edited: Jan 7, 2019
    Revdarian, N4CR, Shoganai and 8 others like this.
  2. socK

    socK 2[H]4U

    Messages:
    3,698
    Joined:
    Jan 25, 2004
    The detail it manages to extrapolate is legit bonkers.
     
    N4CR likes this.
  3. SeymourGore

    SeymourGore 2[H]4U

    Messages:
    2,800
    Joined:
    Dec 12, 2008
    Wow, screw the remake of FF VII - this ESRGAN upscale is what I want to play.
     
  4. AlphaAtlas

    AlphaAtlas [H]ard|Gawd Staff Member

    Messages:
    1,713
    Joined:
    Mar 3, 2018
    It is! In my experience, ESRGAN is kinda finicky though, as it tends to produce ringing/halo artifacts if you feed it messy images.
     
  5. deaedius

    deaedius Gawd

    Messages:
    958
    Joined:
    Jun 18, 2014
    That is pretty awesome, I would love to see some of this in video!
     
    Solhokuten likes this.
  6. AlphaAtlas

    AlphaAtlas [H]ard|Gawd Staff Member

    Messages:
    1,713
    Joined:
    Mar 3, 2018
    Send me a clip/link, and I'll show you.
     
  7. Master_shake_

    Master_shake_ [H]ardForum Junkie

    Messages:
    9,765
    Joined:
    Apr 9, 2012
    wutai looks great.

    so does Tifas seventh heaven bar.
     
  8. PeaKr

    PeaKr Gawd

    Messages:
    753
    Joined:
    Sep 6, 2004
    Interesting comments on reddit about ESRGAN.
     
  9. DF-1

    DF-1 2[H]4U

    Messages:
    2,578
    Joined:
    Jun 17, 2011
    i must be blind. i dont see anything different
     
    vegeta535 likes this.
  10. horrorshow

    horrorshow [H]ardness Supreme

    Messages:
    7,341
    Joined:
    Dec 14, 2007
    That's what I'm talking about!!

    (the original background resolution makes FF7 borderline unplayable on a modern display)
     
  11. horrorshow

    horrorshow [H]ardness Supreme

    Messages:
    7,341
    Joined:
    Dec 14, 2007
    Blind as a bat my friend....
    MmxxkAk.jpg
     
    Armenius and Peter2k like this.
  12. deaedius

    deaedius Gawd

    Messages:
    958
    Joined:
    Jun 18, 2014
    Look at distant objects. The stop light thing/pole impressive.
     
  13. AlphaAtlas

    AlphaAtlas [H]ard|Gawd Staff Member

    Messages:
    1,713
    Joined:
    Mar 3, 2018
    It really is night-and-day. As a more concrete example than low-res screenshots or a Reddit comment, I made this quick and dirty test over the weekend:
    APCJzz.jpg
    APCC65.jpg
    APCkhC.jpg
    That's actually MSRN, but it's basically the same thing.
     
  14. Rizen

    Rizen [H]ardForum Junkie

    Messages:
    9,224
    Joined:
    Jul 16, 2000
    Holy shit this is awesome.
     
    N4CR likes this.
  15. Zam15

    Zam15 [H]Lite

    Messages:
    65
    Joined:
    Jul 27, 2015
    Prepare for a flood of lazy remasters from game companies to make a quick buck with little to no work. Still impressive tech however
     
    N4CR and Flogger23m like this.
  16. alxlwson

    alxlwson You Know Where I Live

    Messages:
    6,356
    Joined:
    Aug 25, 2013
    Oh sweet baby Jesus! Wutai looks amazing! Square dumping what I'm sure is 11 buckets of cash into the FFVII REMAKE, and this is basically all we wanted in the first place.
     
    N4CR likes this.
  17. Kinsaras

    Kinsaras 2[H]4U

    Messages:
    3,574
    Joined:
    Dec 8, 2009
    That can't be vanilla FF7. My PS2 wasn't that sharp. :D
     
    alxlwson likes this.
  18. alxlwson

    alxlwson You Know Where I Live

    Messages:
    6,356
    Joined:
    Aug 25, 2013

    PS1
     
  19. Zulgrib

    Zulgrib n00b

    Messages:
    31
    Joined:
    Dec 11, 2018
    Do it please.
     
  20. Cactusj

    Cactusj n00b

    Messages:
    46
    Joined:
    Jun 4, 2018
    Find a way to replace the overworld models with the battle models and we are in business.
     
  21. chameleoneel

    chameleoneel 2[H]4U

    Messages:
    3,022
    Joined:
    Aug 15, 2005
    Finally, a real use for AI!
     
  22. sl1982

    sl1982 Limp Gawd

    Messages:
    365
    Joined:
    Jul 24, 2008
    I am an admin over on the site where this was originally posted. If you are into modding of final fantasy pc games you should check it out. Quite a bit of work has been put into bringing these games into the modern era while still being faithful to the originals.
     
    N4CR, FlawleZ and AlphaAtlas like this.
  23. bigdogchris

    bigdogchris [H]ard as it Gets

    Messages:
    17,899
    Joined:
    Feb 19, 2008
    Looks fantastic. Couple this with a model mod and you don't need the remake.
     
  24. andrewaggb

    andrewaggb Limp Gawd

    Messages:
    436
    Joined:
    Oct 6, 2004
    I'd be ok with game companies using this tech to help remake some old games. Looks like a fantastic use case for AI. There's some older movies that could benefit as well.
     
  25. velusip

    velusip [H]ard|Gawd

    Messages:
    1,577
    Joined:
    Jan 24, 2005
    The limited ability to derive detail in the background has the added (bonus) side effect of looking out of focus. You get a fake focal depth for free.
     
  26. Dodge245

    Dodge245 Limp Gawd

    Messages:
    183
    Joined:
    Oct 8, 2018
    looked fantastic when i was on my phone, looking at the pics, but on my computer ive noticed lots of small issues with the upscale version..

    That first scene posted above, the gate to the left is meant to be straight bars in the upscale some are bent others are straight.. Same with the balcony on the left, and the one right at the back. Railing at the back and the yellow and black strips all have the same wavy-ness.

    Tifas bar the fan at the back is in front of the grate in certain areas... etc..

    Dont get me wrong its amazing what the AI can do to upscale, and i imagine alot of the issue is noise from the original picture used, i would still love to see this in FF7, however it will not just be a straight AI upscale, someone is going to have to fix the upscales...
     
  27. Youn

    Youn [H]ardness Supreme

    Messages:
    5,477
    Joined:
    Jan 22, 2007
    is this how they made the super mario brother movie?
     
    alxlwson and lostinseganet like this.
  28. ff7 alright. the game that made me a gamer.
     
  29. SvenBent

    SvenBent 2[H]4U

    Messages:
    3,146
    Joined:
    Sep 13, 2008
    a benchmkar for hte collection yes please
     
  30. Youn

    Youn [H]ardness Supreme

    Messages:
    5,477
    Joined:
    Jan 22, 2007
    some great examples of it on other games/images:



    some of my favorites:

    AWd24hX.gif
    wdBo7ZH.png

    8LIWXH6.gif

    and this just looks cool:

    jKHJRTD.png
     
    Armenius, lostinseganet and USMCGrunt like this.
  31. AlphaAtlas

    AlphaAtlas [H]ard|Gawd Staff Member

    Messages:
    1,713
    Joined:
    Mar 3, 2018
    Cool, I registered earlier today under the same username. I'm not a FF fan (yet), but as I mentioned in the thread, I'm already working a package that should make running AI upscalers easy. Seems like the modders are using a combination of Avisynth and research paper demo tools now, which we can definitely improve on.
     
    zrikz and N4CR like this.
  32. GlacierNine

    GlacierNine Limp Gawd

    Messages:
    198
    Joined:
    Feb 9, 2017
    There have been mods to do this exact thing for years.
     
    lostinseganet likes this.
  33. MaZa

    MaZa 2[H]4U

    Messages:
    2,735
    Joined:
    Sep 21, 2008
    Indeed. But the effect is also quite jarring. The great majority or NPCs do not have a "high detail" models available so the replaced ones do stick out like a sore thumb.
     
  34. GlacierNine

    GlacierNine Limp Gawd

    Messages:
    198
    Joined:
    Feb 9, 2017
    The vast majority of NPCs also don't have battle models, so I contend that Cactusj's request is still very much satisfied by the mods :p
     
  35. lostin3d

    lostin3d [H]ard|Gawd

    Messages:
    2,038
    Joined:
    Oct 13, 2016
    Not into FF but no denying the results. I'd love to see someone do this to Witcher 2. I just replayed again over the holidays with the upgraded rig in 1440p with no mods but fully manually maxed AA/Anistropic and filters in NV CP, NV profile inspector and the cfg file. It looked great but something like this would take it to another level.
    71DC6A578165ACFD9041E1883D298D07DD32E2D6.jpg
     
  36. FlawleZ

    FlawleZ Gawd

    Messages:
    816
    Joined:
    Oct 20, 2010
    This is really impressive to see. It's far from perfect but certainly a step in the right direction. I spent a bit of time years ago modding FF7 and although you can improve a lot of the textures and models, the engine and mechanics are still rigid and very dated. Very nostalgic to fire up and play but I'm really looking forward to the remake of FF7.
     
  37. N4CR

    N4CR 2[H]4U

    Messages:
    3,969
    Joined:
    Oct 17, 2011
    Flat tack but dropped in to say thanks for the heads up AlphaAtlas, this is a huge chunk of the way to realising a FF7+8 up res. We don't want tomb raider FF, the play, story and feel of the originals are exactly what most of us want but with more resolution.

    Edit: I'd love to help with manual fixes, especially for FF8 which I have most time on. Next few months are crazy busy but I'll do my best.

    Edit2, I can almost imagine a Japanese version of that Hitler bunker scene playing out at squaresoft right now lel. They should have just listened, this cat ain't going back in the bag.
     
    Last edited: Jan 8, 2019
  38. sl1982

    sl1982 Limp Gawd

    Messages:
    365
    Joined:
    Jul 24, 2008
    If you think the neural network stuff is amazing you should see the work people are putting in remaking the backgrounds from scratch. Here is an example ZSwrkvS.jpg
     
    bboynitrous likes this.
  39. mbelue

    mbelue Limp Gawd

    Messages:
    216
    Joined:
    Apr 11, 2017
    Well if they make old games work with modern renderers and upscaledx textured I'll buy them a at a reasonable price.

    I prefer a closer to original style of remaster anyways.

    When you have companies rebuilding games from scratch there's too much opportunity for editorializing.

    If iD remasters Quake one and "fixes" it's physics the way they were fixed for Crash Bandicoot I'll never play it. Changing the physics would ruin that game.

    Many things, hell most things, don't need rebuilding. Just some polish and tune-up.