NVIDIA AI Inpainting Is Cool as Hell

Discussion in 'HardForum Tech News' started by FrgMstr, Apr 23, 2018.

  1. FrgMstr

    FrgMstr Just Plain Mean Staff Member

    Messages:
    48,110
    Joined:
    May 18, 1997
    No matter what you think about NVIDIA and its anti-competitive GPP, its AI powered Inpainting is damn cool to see in action. It is not perfect, but what it is getting done in real time is amazing.
    Taking away Borgnine's trademark crazy eyebrows is across the line however.


    Check out the video.

    Researchers from NVIDIA, led by Guilin Liu, introduced a state-of-the-art deep learning method that can edit images or reconstruct a corrupted image, one that has holes or is missing pixels. The method can also be used to edit images by removing content and filling in the resulting holes. Learn more about their research paper “Image Inpainting for Irregular Holes Using Partial Convolutions.”
     
  2. jardows

    jardows [H]ard|Gawd

    Messages:
    1,515
    Joined:
    Jun 10, 2015
    That's cool. Not perfect for sure. I can think of this as a nice way to make it easier to get "the shot" you want. There was a view from a hill overlooking town, with a nice mesa in the background that I always thought would be a great picture, but there was no place you could take it without having a bunch of power lines disrupting the view. This could be an easy way to get rid of those.
     
  3. Lakados

    Lakados [H]ard|Gawd

    Messages:
    1,441
    Joined:
    Feb 3, 2014
    That's all just crazy, like down right fucked up in all sorts of ways but still kinda awesome.
     
    mdburkey likes this.
  4. DangerousMan

    DangerousMan Limp Gawd

    Messages:
    251
    Joined:
    Jan 16, 2006
    I have photos that I took on my last vacation to Italy, they have random people in the background as taking the photos without them there was next to impossible. I would definitely try this out to remove the randoms and leave just me/family in the shots.
     
  5. Joust

    Joust 2[H]4U

    Messages:
    2,623
    Joined:
    Nov 30, 2017
    Interesting tech - and from that perspective I think it's very neat. However, like so many editting practices, I'm afraid this further blurs the lines between photographic art (as captured) and graphic arts (as generated).

    I'm not a photographer in the least, so my opinion is probably ill-informed.
     
  6. The Mad Atheist

    The Mad Atheist Gawd

    Messages:
    911
    Joined:
    Mar 9, 2018
    So pretty much Photoshop for noobies?
     
    SomeoneElse and nilepez like this.
  7. Joust

    Joust 2[H]4U

    Messages:
    2,623
    Joined:
    Nov 30, 2017
    More like Photoshop for Ava. (Ex Machina)
     
  8. Krenum

    Krenum [H]ardForum Junkie

    Messages:
    15,367
    Joined:
    Apr 29, 2005
    That's really cool. Great way to touch up screenshots from your games.
     
  9. WorldExclusive

    WorldExclusive [H]ardForum Junkie

    Messages:
    10,860
    Joined:
    Apr 26, 2009
    Now I can remove the cellulite from South Beach models asses with confidence.
     
    Armenius likes this.
  10. elavanis

    elavanis n00b

    Messages:
    37
    Joined:
    Feb 1, 2008
    Photoshop has had something like this since cs5 which was release in 2010.

    It won't do things like adding in back eyes but it will remove power lines from the previous posters.

    Here is video of a guy showing content aware fill from 2012.
     
    SomeoneElse, nilepez and Rahh like this.
  11. schoolslave

    schoolslave Gawd

    Messages:
    568
    Joined:
    Dec 7, 2010
    I wonder how many (and what kinds) of training/reference images they had to use for the face inpainting?
    Guess it's time to read the research paper...
     
  12. griff30

    griff30 I Lower the Boom!

    Messages:
    5,313
    Joined:
    Jul 15, 2000
    This is with desktop graphics cards or graphics cards you need a mortgage for?
     
  13. andrewaggb

    andrewaggb Limp Gawd

    Messages:
    413
    Joined:
    Oct 6, 2004
    Looked pretty impressive to me. Other than the eye replacement not really matching the face I'd say the rest of it was excellent. I'd use this for sure. Lots of photo's with something (sock, cup, kids toy, etc) that distracts from the overall scene.
     
    griff30 likes this.
  14. DangerousMan

    DangerousMan Limp Gawd

    Messages:
    251
    Joined:
    Jan 16, 2006
    Did not know this. I just tried it out and its ok; works better with smaller stuff then what I was looking to do but it does work.
    Thanks for the info.
     
  15. Rahh

    Rahh [H]ard|Gawd

    Messages:
    1,608
    Joined:
    Jan 14, 2005
    I think Photoshop does it better with the newer versions and as already stated they have been doing this since 2012. However, NVIDIA's is good too and competition is always welcomed but I don't think it can match PS yet.
     
  16. Kwaz

    Kwaz Whine & Cheezy

    Messages:
    3,594
    Joined:
    Sep 3, 2014
    Have you tried the photoshop content aware to remove them?
     
  17. DangerousMan

    DangerousMan Limp Gawd

    Messages:
    251
    Joined:
    Jan 16, 2006
    Yep, just now. Not the best outcome but it did remove some of what I wanted. There are still unwanted people in the pics but it does look better now.
     
  18. mr_zen256

    mr_zen256 2[H]4U

    Messages:
    2,246
    Joined:
    Dec 29, 2005
    That's cool. Pretty much a slightly more advanced Photoshop clone stamp / healing brush tool.
     
    Mav451 likes this.
  19. oROEchimaru

    oROEchimaru [H]ardness Supreme

    Messages:
    4,658
    Joined:
    Jun 1, 2004
    anyone else tormented by the white out eyes?
     
  20. Merc1138

    Merc1138 2[H]4U

    Messages:
    2,086
    Joined:
    Sep 25, 2010
    No, go back go doodling in your coloring book.
     
  21. SpeedyVV

    SpeedyVV [H]ardness Supreme

    Messages:
    4,203
    Joined:
    Sep 14, 2007
    “To the best of our knowledge, we are the first to demonstrate the efficacy of deep learning image inpainting models on irregularly shaped holes,” the NVIDIA researchers mentioned.

    Is it just me that finds that intriguing?

    Mind you, the last thing I think of doing is inpainting models on their irregular shaped holes.
     
  22. rgMekanic

    rgMekanic [H]ard|News Staff Member

    Messages:
    3,749
    Joined:
    May 13, 2013
    Pfft, fake.. FAKE

    They already had everyones pictures when they logged into Facebook through Geforce Experience, they just put it back together, no AI needed
     
    lostin3d and WhoMe like this.
  23. WhoMe

    WhoMe Gawd

    Messages:
    827
    Joined:
    Jan 3, 2018
    I'm waiting for when I don't need to ever take a photo, the AI will do it for me...and it better stay out of my bedroom. Heck the way things are going the AI will need one image of you then seamlessly put your imagine into any compromising situation it wants and make it look real to forensics...who needs killer bots?
     
    _l_ likes this.
  24. _l_

    _l_ I Am A Cock

    Messages:
    1,151
    Joined:
    Nov 27, 2016
    So, will AI eventually be used to make imbeciles smart so they can challenge the naturally non-imbecilic for their high tech job positions?
     
    WhoMe likes this.
  25. WhoMe

    WhoMe Gawd

    Messages:
    827
    Joined:
    Jan 3, 2018
    What high tech job positions? And seriously while I think AI is going to change a lot of things we are starting to see it, the timeframe I'm not sure of...I remember back in 90's when a lot of people predicted computers were about to take away every computer programmers job...still waiting...but maybe it's in sight now?
     
  26. Gottfried Leibnizzle

    Gottfried Leibnizzle Limp Gawd

    Messages:
    258
    Joined:
    Apr 29, 2015
    I forgot about the former. Ahhh back in my day....
     
    SomeoneElse and griff30 like this.
  27. rsaotome

    rsaotome Limp Gawd

    Messages:
    167
    Joined:
    Jul 28, 2005
    How well does it do with removing a mustache? Just asking for a few people at Warner Bros.
     
    TheOne&OnlyZeke likes this.
  28. aaron33

    aaron33 n00b

    Messages:
    22
    Joined:
    Jan 5, 2010
    It'll put back a more feminine mustache.
     
    griff30 likes this.
  29. SomeoneElse

    SomeoneElse [H]ard|Gawd

    Messages:
    1,734
    Joined:
    Jan 16, 2007
    I was just about to post this. Adobe came out with it quite awhile ago, the nvidia design seems to do it quicker which means they are leveraging the AI platform to create/fill the content. Though a lot of it was obvious it was altered it was quite interesting to see it add in eyes from a photo on its own, although it looked like the same set of eyes that were put on the supermodels face as the older actor's face( can't remember his name).
     
  30. lostin3d

    lostin3d [H]ard|Gawd

    Messages:
    1,968
    Joined:
    Oct 13, 2016
    Local t.v. news had a story a week or two ago with Peele showing off some tech using himself with live rendering tech to create a fake Obama announcement. It looked a little odd but to someone not really used to digital quirks they might not catch all the signs. The anchor then went to a company in Cali that specializes in that stuff and experienced it with live rendering of himself as other people it was impressive and scary. Pics done thru your card seem pretty easy after seeing that.

    In other NV news. OCD mentioned some other new feature involving voxel lighting techniques: https://overclock3d.net/news/softwa...soon_with_increased_performance_and_quality/1

    Kind of neat but ironically they state that the compute hardware needs are pretty demanding and that's why they still haven't fully rolled it out. Even more ironic since if they hadn't effectively disabled hardware based PhysX many of us with decent older cards could probably easily benefit from this.
     
    SomeoneElse likes this.
  31. TheOne&OnlyZeke

    TheOne&OnlyZeke 100% Irish

    Messages:
    10,215
    Joined:
    Jul 21, 2000
    Man they need to re-release that bluray and fix that shit proper
    He looks like a mutant
     
  32. Kwaz

    Kwaz Whine & Cheezy

    Messages:
    3,594
    Joined:
    Sep 3, 2014
    OCD, lol
     
    lostin3d likes this.
  33. Armenius

    Armenius I Drive Myself to the [H]ospital

    Messages:
    17,171
    Joined:
    Jan 28, 2014
    Finally something useful for AI to do. Now I can churn out memes 10 times as fast.
     
    lostin3d likes this.
  34. exiled350

    exiled350 Gawd

    Messages:
    994
    Joined:
    Jun 26, 2013
    The real question people should be asking is 'what happens when you apply said tech to censored boobs?' do you get eyes instead of nipples???
     
    Armenius and griff30 like this.
  35. bman212121

    bman212121 [H]ard|Gawd

    Messages:
    1,528
    Joined:
    Aug 18, 2011
    OverClocking Disorder? Pretty sure there's some people around here with that!

    I'm not sure if that's funnier or the (un)intentional mention of nVNews.
     
  36. Kwaz

    Kwaz Whine & Cheezy

    Messages:
    3,594
    Joined:
    Sep 3, 2014
    The medical diagnosis would more appropriately describe OC3D videos.
     
    lostin3d and bman212121 like this.
  37. griff30

    griff30 I Lower the Boom!

    Messages:
    5,313
    Joined:
    Jul 15, 2000
    Neither, it removes them and you get a pair of kneecaps in the upper chest.
     
    Armenius likes this.
  38. Joust

    Joust 2[H]4U

    Messages:
    2,623
    Joined:
    Nov 30, 2017
    That was said with a disconcerting amount of confidence and certainty. Prob- probably your friend researched this?
     
    griff30 and Armenius like this.
  39. ianken

    ianken [H]ard|Gawd

    Messages:
    1,953
    Joined:
    Feb 21, 2006
    Codename: Project Stalin.

    He would of LOVED this. Best way to erase people from the party. LOL.
     
    griff30 likes this.
  40. lostin3d

    lostin3d [H]ard|Gawd

    Messages:
    1,968
    Joined:
    Oct 13, 2016
    When good AI pic software goes bad:

    pqacn09yoxcz.png