AI Creates Programs from Simple Screenshots

Megalith

24-bit/48kHz
Joined
Aug 20, 2006
Messages
13,000
Interesting news for the lazier, armchair programmers out there: the appropriately named “Pix2Code” is an application created by a neural network that can generate code automatically when it’s fed with screenshots of a GUI. The article suggests that it’s superior to humans because it can create code for Android, iOS, and “web-based technologies”: apparently, most coders only know how to craft for one platform. This may already exist, but I would like to see a version for web design: just create a layout in Photoshop and have an application translate it to the proper HTML and CSS.

Artificial intelligence is threatening worker’s jobs in dozens of different professions, from manufacturing and legal to banking and driving, and many more. Now, it’s time to add programming to that list. A new neural network being built by a Danish startup called UIzard Technologies IVS has created an application that can transform raw designs of graphical user interfaces into actual source code that can be used to build them. It uses cutting-edge machine learning technologies to create a neural network that can generate code automatically when it’s fed with screenshots of a GUI.
 
Creating a GUI from a screenshot isn't that difficult of a task. I know a few places that already had tools to do something similar, albeit not using a neural net. But, it won't surprise me when in the near future, you can just speak commands such as "Add Click Routine for Button X called XYZ". Right now, the most annoying thing about GUI development is that every platform seems to prefer their own vocabulary and approach, and the majority of my time seems to be trying to figure out how to just do basic stuff.
 
Well I don't think anybody is going to feel threatened by that.

As a software developer I look forward to being able to describe problems or algorithms and just tell the computer what I'm trying to do and have it code for me. This demo doesn't really do much though. I realize it's just a starting point, but It's not like they showed it a screenshot of facebook's web page and had it make something similar. It was fed the most basic ios app and an extremely simple bootstrap page and was able to duplicate it. But it would have no understanding of what sort of code lies behind the interface.

It would be so much more impressive to show something videos of the application or let it crawl and use facebook (or hardocp or whatever) and discover the logic and relationships on it's own and build an equivalent. Build it's own tech news aggregator :)

I feel like we are many years away from that.
 
this seems like a cool tool for programmers more than anything?

for instance...
a. program with the language you are comfortable with
b. now you need to upgrade it to version xyz but the producer doesnt have an easy upgrade path... have AI help "upgrade" from .net 3 to .net 4.5 etc

or...
a. program is very old (batch, aix etc)...
b. upgrade it to program language c# ...

while also having it create the gui. seems handy!
 
This is just an extension of the Prototyping GUI editor like Visual Basic. Just instead of drag-and-drop GUI objects, and then click-to-add basic event handler code to open-up another window, it does all that for you.

Would save some more time for building new projects skeleton for you, but still doesn't handle the heavy lifting. This couldn't begin to write a complex application like Microsoft Excel, for-instance.

But for simpler apps, this could cut down development time significantly.
 
I don't claim to be a programming guru, I have only worked in software dev 15 years or so. Geez, that long?

Back on track, this is like the previous posts said. A neat tool for developers to use. Hardly a replacement. If all you can do is write code, sure it's scary. A developer does so much more.

Unless I made my living writing brochureware websites I have no fear. Ha, actually if that's what I did I'd probably be happy for something to do it for me so I can compete against some third world company that has a legion of programmers working for $3 an hour.

Yeah, that's and exaggeration, but you get the point.
 
Wasn't the target a web page? So it has HTML code, and the thing is just picking up on that and fiddling with it?
 
Back
Top