Kolmogorov Arnold Network (KAN)

A new neural network just dropped from MIT aka Kolmogorov-Arnold Network (KAN). Refer to this page for the superposition theorem.

This is a pure Kolmogorov–Arnold (KAN) Convolutional Network for image classification! Following the VGG16 architecture, we can simply replace the convolution and linear layers with their KAN equivalent! Using convolutional and linear KAN layer implementation from this repo: https://github.com/Indoxer/LKAN/tree/main

Automatic pruning

Automatic pruning with Kolmogorov-Arnold Networks (KANs) to make the network sparser, more efficient, and more interpretable.

The automatic pruning process in KANs works as follows:

For each node, the maximum L1 norm of its incoming and outgoing activations is calculated.
If both the maximum incoming and outgoing L1 norms are below a certain threshold, the node is considered inactive and pruned away.
However, automatic edge pruning is not performed to avoid accidentally removing important edges with small L1 norms.
Manual pruning of nodes and edges is still possible if desired.

Kolmogorov Arnold Network (KAN)

The coding steps

A KAN model is created using the `KAN` constructor. The model has 2D inputs, 1D output, and 5 hidden neurons. It uses cubic splines (k=3) with 5 grid intervals (grid=5). The `seed` parameter is set to 0 for reproducibility.
A dataset is created using a lambda function `f(x,y) = exp(sin(pi*x)+y^2)`.
The KAN model is trained using the `train` method with optimization algorithm (LBFGS), and regularization parameters (`lamb` and `lamb_entropy`).

After training, automatic pruning is applied to the model using the `prune` method. This removes inactive neurons based on the L1 norm of their incoming and outgoing activations.

By pruning the KAN, the network becomes sparser, reducing computational complexity and improving interpretability. The pruned network retains the most important nodes and edges while removing redundant or less significant ones. This can help in understanding the underlying structure and relationships learned by the model.

Kolmogorov Arnold Network (KAN)

A life dedicated to the pursuit of infinity

Andrey Nikolaevic Kolmogorov was one of the leading figures in mathematics of the 20^th century and one of the most important mathematicians of all time. The list below presents some important facts of Kolmogorov's life and work:

1903 - Kolmogorov was born in Tambov on 25 April. His mother dies in his birth.
1908 - Little Andrey starts to show a literacy in mathematics.
1910 - He is being adopted by his aunt and both are moving to Moscow.
1920 - Kolmogorov graduates from high-school. He begins to study at the Moscow State University and at the same time at the Mendeleev Institute of Chemistry and Technology.
1921-22 - He attends seminars in history and publishes his first research paper on history. At the same period Kolmogorov worked out and proved several results in set theory and in the theory of Fourier series.
1922 - Kolmogorov's research work gained international recognition for construction a Fourier series that diverges almost everywhere.
1925 - He graduates from the Moscow State University and begins to study under the supervision of Nikolai Luzin and Alexandr Khinchin. He publishes a study in intuitionistic logic.
1929 - He earns his Doctor of Philosophy degree.
1930 - Long trip with Alexandrov to Göttingen, Munich and Paris where he meets other famous mathematicians and develops a huge body of work.
1931 - He publishes a pioneering work titled About the Analytical Methods of Probability. He becomes Professor at the Moscow State University.
1933 - Kolmogorov published his book Foundations of the Theory of Probability, laying the formalization of the theory in an axiomatic basis.
1935 - He becomes chairman of probability theory at the Moscow State University.
1936 - Kolmogorov contributed to the field of ecology and generalized the Lotka-Volterra model of predator-prey systems. Along with Alexandrov were involved in the political persecution of their common teacher Nikolai Luzin.
1938 - He publishes an important study on the stochastic processes.
1941 - His research starts to focus on the phenomenon of turbulence and chaotic processes producing many important results in the classical mechanics.
1942 - Kolmogorov is getting married with Anna Dmitrieva Egorova.
1954 - Presentation of Kolmogorov-Arnold-Moser theorem at the International Congress of Mathematicians.
1957 - Jointly with his student V.I. Arnold completely solved Hilbert's Thirteenth problem.
1987 - Kolmogorov died in Moscow on 20 October.

You can read more information about the work of Kolmogorov in an online article as part of the project MacTutor History of Mathematics archive hosted by the University of St. Andrews.