Socials

Post 22: What is the Basis of neural networks success? 🧐

Published:

 

Mathematical functions allow us to model the behavior of many things. Everything can be described in terms of functions, but to do so, we need to know how many variables to include and how they behave. In other words, creating good functions is not easy, it requires a lot of research and creativity.

Now that we have vast amounts of data, neural networks are widely used because they can learn a wide variety of patterns from the data. In this interaction, the “Universal Approximation Theorem” assures us that a sufficiently complex neural network can approximate any type of function.

How do we make a neural network more complex? One way is by increasing the number of neurons, which allows it to learn more complicated patterns. For example, if we use 20 neurons to model a parabola function, the result won’t be very accurate. But if we increase it to 50 neurons, the network can now approximate the parabola much more precisely.

The parabola is a simple function, but neural networks have been used to approximate much more complex functions. For instance, they have been used to approximate the Schrödinger equation and model molecules with up to 30 electrons, when in its classical form, modeling just one electron is already quite complicated. Another example is predicting “canonical events” that occur in our lives: birth, marriage, death, etc. By breaking life down into a sequence of events, neural networks can learn functions that, in a very general way, “predict our future”. Even chaotic systems, which are among the most difficult to model due to the sensitivity of their variables, can be modeled to a certain extent with neural networks.

Now, whether everything can really be described by functions (which obviously is not the case), how good these functions are, the quality of the data the neural networks are learning from, and a myriad of other questions are other debates.

img

Refs:

And the articles where they approximate