Google Brain Super-Resolution Image Tech Makes “Zoom, Enhance!” Real

Megalith

24-bit/48kHz
Staff member
Joined
Aug 20, 2006
Messages
13,004
Google has published a paper titled “Pixel Recursive Super Resolution” that demonstrates how it is now possible to turn a tiny, pixelated mess into a more detailed, usable image. Wow, now you can watch your favorite make-believe investigator hitting an “enhance” button to get the face of a suspect without laughing. But in light of all of these fancy advancements in neural network technology, how is it that we are still stuck with bicubic resizing for enlarging photos? I feel like Adobe would have introduced something superior by now—I mean, they did manage to create pure magic like content-aware fill. There is waifu2x, I suppose…

…it's impossible to create more detail than there is in the source image—so how does Google Brain do it? With a clever combination of two neural networks. The first part, the conditioning network, tries to map the the 8×8 source image against other high resolution images. It downsizes other high-res images to 8×8 and tries to make a match. The second part, the prior network, uses an implementation of PixelCNN to try and add realistic high-resolution details to the 8×8 source image. Basically, the prior network ingests a large number of high-res real images—of celebrities and bedrooms in this case. Then, when the source image is upscaled, it tries to add new pixels that match what it "knows" about that class of image. For example, if there's a brown pixel towards the top of the image, the prior network might identify that as an eyebrow: so, when the image is scaled up, it might fill in the gaps with an eyebrow-shaped collection of brown pixels.
 
Last edited:

Zarathustra[H]

Official Forum Curmudgeon
Joined
Oct 29, 2000
Messages
29,630
I can't imagine that it would be particularly accurate though.

The system would have to use some sort of heuristics to make educated guesses in order to fill in the missing information in that pixelated mess. They could make it prettier, and possibly even look believable, but accurate? Or accurate in a "stand up in a court of law" kind of way? I doubt it.
 

DeathFromBelow

Supreme [H]ardness
Joined
Jul 15, 2005
Messages
7,316
I can't imagine that it would be particularly accurate though.
Exactly. In this case they used a bunch of pictures of celebrities to 'reconstruct' an image from the blurry mess. If they used pictures of different people the results would change.

Sounds like they've basically taught the computer to hallucinate realistic imagery to replace blurry data, sort of like how our brains hallucinate to mask the blind spots in our vision.
 
Last edited:

Bounty

Gawd
Joined
Jun 10, 2016
Messages
547
Exactly. In this case they used a bunch of pictures of celebrities to 'reconstruct' an image from the blurry mess. If they used pictures of different people the results would

Sounds like they've basically taught the computer to hallucinate realistic imagery to replace blurry data, sort of like how our brains hallucinate to mask the blind spots in our vision.
So what you're saying is if we take an 8x8 pixel blob picture of me and run it through Google's algorithms, it'll spit out a blurry picture of Chris Hemsworth. Also, if we take an 8x8 pixel blob picture of my testicles (in such a way that the shadows happen to line up with eyebrows etc.) and run it through Google's algorithms, it'll spit out a blurry picture of Liam Hemsworth?
 

Koolthulu

Gawd
Joined
Mar 24, 2011
Messages
773
"reasonably successful in real-world testing"
"human observers ... were fooled 10 percent of the time"

Wow, a 10% success rate is considered successful.
 

sfsuphysics

I don't get it
Joined
Jan 14, 2007
Messages
13,976
So where's the information in the 8x8 image that has teeth in it? They kind of pop out of no where.
 

robble

Supreme [H]ardness
Joined
Jun 6, 2004
Messages
6,499
That third one should have morphed into Michael Jackson.
 

Exavior

[H]F Junkie
Joined
Dec 13, 2005
Messages
9,677
unless I just didn't gather something here correctly it has to have a higher res imagine so that it can turn it into a 8x8 version to compare to the first picture. I would hope that if I took a normal picture and turned it into a 8x8 version a computer could do the same and come up with a good idea of the original picture.

For this to be even slightly impressive to me they would have to be able to take any random picture of anything at any size and make a more detailed version of it. Until then they aren't doing anything great
 

gulguran

Limp Gawd
Joined
Jul 30, 2015
Messages
264
Regardless, it is awesome to see fiction turn into non-fiction.
But it isn't, it uses sampling of better quality images to arrive at a match if I read it right which means it's just smoke and mirrors. If you need an original that is somewhat similar then it's not science fiction.
 

Tawnos

2[H]4U
Joined
Sep 9, 2001
Messages
3,807
So where's the information in the 8x8 image that has teeth in it? They kind of pop out of no where.
They probably have a pass in the secondary stage that rewards a close proximity score to a re-pixellated image.
 

Tawnos

2[H]4U
Joined
Sep 9, 2001
Messages
3,807
But it isn't, it uses sampling of better quality images to arrive at a match if I read it right which means it's just smoke and mirrors. If you need an original that is somewhat similar then it's not science fiction.
What? In sci-fi there are plenty of sequences where existing pieces are used for enhancement, creating a composite of a suspect, etc.
 

jardows

[H]ard|Gawd
Joined
Jun 10, 2015
Messages
1,802
Looking at the sample pictures, especially of the first and last person, the middle image looks like a completely different person than the final image. I can't imagine this tech being used any time soon for any real results.
 

Jagger100

Supreme [H]ardness
Joined
Oct 31, 2004
Messages
7,580
Alex Jones cream dream. All the 'enhanced' historical pictures with Time Travelling Elvis will be legion. Break the Conditioning!!!!!!
 
Top