Algorithms and Hardware for Deep Learning

The computer industry has been busy in recent years trying to figure out how to speed up the calculations needed for artificial neural networks...much effort has gone into designing special-purpose hardware to run such computations. Google, for example, developed its Tensor Processing Unit, or TPU, first described publicly in 2016. More recently, Nvidia introduced its V100 Graphics Processing Unit, which the company says was designed for AI training and inference, along with other high-performance computing needs.

Perhaps they’re all making a giant mistake.

In a paper posted last week, Beidi Chen, Tharun Medini, and
Anshumali Shrivastava of Rice University, argue that the specialized hardware that’s been developed for neural networks may be optimized for the wrong algorithm...

Click to expand...