Google Brain Super-Resolution Image Tech Makes “Zoom, Enhance!” Real

Discussion in 'HardForum Tech News' started by Megalith, Feb 7, 2017.

  1. Megalith

    Megalith 24-bit/48kHz Staff Member

    Messages:
    13,004
    Joined:
    Aug 20, 2006
    Google has published a paper titled “Pixel Recursive Super Resolution” that demonstrates how it is now possible to turn a tiny, pixelated mess into a more detailed, usable image. Wow, now you can watch your favorite make-believe investigator hitting an “enhance” button to get the face of a suspect without laughing. But in light of all of these fancy advancements in neural network technology, how is it that we are still stuck with bicubic resizing for enlarging photos? I feel like Adobe would have introduced something superior by now—I mean, they did manage to create pure magic like content-aware fill. There is waifu2x, I suppose…

    …it's impossible to create more detail than there is in the source image—so how does Google Brain do it? With a clever combination of two neural networks. The first part, the conditioning network, tries to map the the 8×8 source image against other high resolution images. It downsizes other high-res images to 8×8 and tries to make a match. The second part, the prior network, uses an implementation of PixelCNN to try and add realistic high-resolution details to the 8×8 source image. Basically, the prior network ingests a large number of high-res real images—of celebrities and bedrooms in this case. Then, when the source image is upscaled, it tries to add new pixels that match what it "knows" about that class of image. For example, if there's a brown pixel towards the top of the image, the prior network might identify that as an eyebrow: so, when the image is scaled up, it might fill in the gaps with an eyebrow-shaped collection of brown pixels.
     
    Last edited: Feb 7, 2017
  2. Zarathustra[H]

    Zarathustra[H] Official Forum Curmudgeon

    Messages:
    27,638
    Joined:
    Oct 29, 2000
    I can't imagine that it would be particularly accurate though.

    The system would have to use some sort of heuristics to make educated guesses in order to fill in the missing information in that pixelated mess. They could make it prettier, and possibly even look believable, but accurate? Or accurate in a "stand up in a court of law" kind of way? I doubt it.
     
  3. scojer

    scojer 2[H]4U

    Messages:
    3,840
    Joined:
    Jun 13, 2009
    Regardless, it is awesome to see fiction turn into non-fiction.
     
    Zarathustra[H] likes this.
  4. DeathFromBelow

    DeathFromBelow [H]ardness Supreme

    Messages:
    7,105
    Joined:
    Jul 15, 2005
    Exactly. In this case they used a bunch of pictures of celebrities to 'reconstruct' an image from the blurry mess. If they used pictures of different people the results would change.

    Sounds like they've basically taught the computer to hallucinate realistic imagery to replace blurry data, sort of like how our brains hallucinate to mask the blind spots in our vision.
     
    Last edited: Feb 7, 2017
    DocSavage and BlueFireIce like this.
  5. rgMekanic

    rgMekanic [H]ard|News Staff Member

    Messages:
    3,721
    Joined:
    May 13, 2013
  6. Bounty

    Bounty Limp Gawd

    Messages:
    237
    Joined:
    Jun 10, 2016
    So what you're saying is if we take an 8x8 pixel blob picture of me and run it through Google's algorithms, it'll spit out a blurry picture of Chris Hemsworth. Also, if we take an 8x8 pixel blob picture of my testicles (in such a way that the shadows happen to line up with eyebrows etc.) and run it through Google's algorithms, it'll spit out a blurry picture of Liam Hemsworth?
     
  7. Retronym

    Retronym Something big is coming.

    Messages:
    11,069
    Joined:
    Mar 5, 2007
    burn the witch.
     
    rgMekanic likes this.
  8. Koolthulu

    Koolthulu Gawd

    Messages:
    773
    Joined:
    Mar 24, 2011
    "reasonably successful in real-world testing"
    "human observers ... were fooled 10 percent of the time"

    Wow, a 10% success rate is considered successful.
     
  9. sfsuphysics

    sfsuphysics I don't get it

    Messages:
    13,661
    Joined:
    Jan 14, 2007
    So where's the information in the 8x8 image that has teeth in it? They kind of pop out of no where.
     
  10. NeoNemesis

    NeoNemesis 2[H]4U

    Messages:
    2,385
    Joined:
    Mar 10, 2004
    They have to start somewhere.
     
  11. robble

    robble [H]ardness Supreme

    Messages:
    6,501
    Joined:
    Jun 6, 2004
    That third one should have morphed into Michael Jackson.
     
  12. Exavior

    Exavior [H]ardForum Junkie

    Messages:
    9,646
    Joined:
    Dec 13, 2005
    unless I just didn't gather something here correctly it has to have a higher res imagine so that it can turn it into a 8x8 version to compare to the first picture. I would hope that if I took a normal picture and turned it into a 8x8 version a computer could do the same and come up with a good idea of the original picture.

    For this to be even slightly impressive to me they would have to be able to take any random picture of anything at any size and make a more detailed version of it. Until then they aren't doing anything great
     
  13. gulguran

    gulguran Limp Gawd

    Messages:
    263
    Joined:
    Jul 30, 2015
    But it isn't, it uses sampling of better quality images to arrive at a match if I read it right which means it's just smoke and mirrors. If you need an original that is somewhat similar then it's not science fiction.
     
  14. Tawnos

    Tawnos 2[H]4U

    Messages:
    3,805
    Joined:
    Sep 9, 2001
    They probably have a pass in the secondary stage that rewards a close proximity score to a re-pixellated image.
     
  15. Tawnos

    Tawnos 2[H]4U

    Messages:
    3,805
    Joined:
    Sep 9, 2001
    What? In sci-fi there are plenty of sequences where existing pieces are used for enhancement, creating a composite of a suspect, etc.
     
  16. jardows

    jardows [H]ard|Gawd

    Messages:
    1,477
    Joined:
    Jun 10, 2015
    Looking at the sample pictures, especially of the first and last person, the middle image looks like a completely different person than the final image. I can't imagine this tech being used any time soon for any real results.
     
  17. Jagger100

    Jagger100 [H]ardness Supreme

    Messages:
    7,419
    Joined:
    Oct 31, 2004
    Alex Jones cream dream. All the 'enhanced' historical pictures with Time Travelling Elvis will be legion. Break the Conditioning!!!!!!
     
  18. Anarchist4000

    Anarchist4000 [H]ard|Gawd

    Messages:
    1,659
    Joined:
    Jun 10, 2001
    Is this how Fallout managed the latest HD texture pack?
     
  19. DocSavage

    DocSavage 2[H]4U

    Messages:
    2,409
    Joined:
    Dec 18, 2002
    This will be very handy for some of those foreign videos I come across.