The Real Tropical Geometry of Neural Networks for Binary Classification

We consider a binary classifier defined as the sign of a tropical rational function, that is, as the difference of two convex piecewise linear functions. In particular, the set of functions represented by a ReLU neural network can be regarded as a subset in the parameter space of tropical rational functions, specifically, it is contained as a semialgebraic set. We initiate the study of two different subdivisions of the parameter space of tropical rational functions with fixed number of terms in the numerator and denominator: a subdivision into semialgebraic sets, on which the combinatorial type of the decision boundary is fixed, and a subdivision into a polyhedral fan, capturing the combinatorics of the partitions of the dataset. Our findings extend and refine the connection between neural networks and tropical geometry by observing structures established in real tropical geometry, such as positive tropicalizations of hypersurfaces and tropical semialgebraic sets. This is work with Marie-Charlotte Brandenburg and Georg Loho.

Bio
https://www.math.ucla.edu/~montufar/