Understanding Neural Networks

Wed, 01 Nov 2023 19:54:22 +0100

Understanding of Neural nets from first-principles: Brain dump

So I was reading my company’s IT newsletter the other day where one of the topics was sparse modeling (discussing this Forbes article) and it got me thinking again about some things I was reading the past months, about trying to understand how and why (mainly) Deep Learning works.

First-principles

In a way, sure we know how it works on the microscopic level of each individual neuron (activation functions, matrix multiplications, gradient descent and all that), and we also often describe it at a high level (where we tend to greatly anthropomorphize it: “the model learned to do X because in all its examples it saw this object from the same angle, ..”). But there are many questions in between where it seems we never connected the dots, instead relying on empiricism, often crude observations and post-hoc justifications for choices here:

Machine Learning on Joris Bukala | Math & ML

Understanding Neural Networks

Understanding of Neural nets from first-principles: Brain dump

First-principles