Scaling Logistic Regression Via Multi-GPU/TPU Training

[erek]

"Logistic regression is a simple, but powerful, classification algorithm. In this blog post we’ll see that we can view logistic regression as a type of neural network.

Framing it as a neural network allows us to use libraries like PyTorch and PyTorch Lightning to train on hardware accelerators (like GPUs/TPUs). This enables distributed implementations that scale to massive datasets.

In this blog post I’ll illustrate this link by connecting a NumPy implementation to PyTorch.

I’ve added this highly scalable logistic regression implementation to the PyTorch Lightning Bolts library which you can easily use to train on your own dataset."

https://towardsdatascience.com/scaling-logistic-regression-for-multi-gpu-tpu-training-b4898d5049ff

 

[Iratus]

I scanned it, seems a bit meh, just using a library to make things a bit less laborious rather than anything new. Don’t get me wrong, it’s smart and it’s getting super quick traction but it’s an ease thing primarily.

There’s some interesting stuff coming out of Facebook labs, Princeton CV Lab and Columbia (Shuran Song is a fricking genius) and PyTorch is getting crazy with possibilities but it’s always a first order problem of algorithm, second order of running it and then third making it scaleable to greater levels of complexity or number of applications. The hardware and scaling thereof just contributes to making newer algorithms viable.

Pretty much everything in this space (and I’m being grossly reductive) is just discrete math at large scale. You can easily get a PhD for the ease of use and the lightning author is gonna, but the real interesting thing is what that enables people to do. Lots of things are theoretically possible but impractical for ‘normals’ without good frameworks and libraries.

(Obligatory why JavaScript is everywhere despite being shit.)