Poster Presentation

Poster A in Poster Session A: Tuesday, August 4, 9:30 – 11:15 am, Kimmel Center, Shorin & Rosenthal Rooms

Parallel synapses and divisive normalization enhance classification capacity

Caitlin Lienkaemper¹, Marissa Masden², Gabriel Koch Ocker³, Ila R Fiete¹; ¹Massachusetts Institute of Technology, ²University of Puget Sound, ³Boston University

Presenter: Caitlin Lienkaemper

Classic results on the computational power of neural networks modeled neurons as perceptrons, ‘point neurons’ that sum inputs linearly. However, real neurons can exploit nonlinearities at synapses or dendrites. In recent work, Song and Benna (2025) explored a model with parallel synapses, multiple contacts between a pre- and postsynaptic neuron, each applying a sigmoidal transformation with learnable parameters in the absence of other dendritic nonlinearities. Under these assumptions, they observed that the effective transmission function from pre- to postsynaptic neuron is constrained to be monotonic, but otherwise flexible. They found in simulations that such neurons achieve a higher classification capacity than perceptrons: the critical capacity -- the threshold ratio between number of input patterns to input dimension where a random labeling is at least 50% likely to be separable -- appears to grow with the logarithm of the input dimension. For a perceptron, the critical capacity is constant. We extend this work with a geometric characterization of separable and inseparable patterns, giving an analogue to the ``XOR problem" for this model. We use our characterization to show that the classification capacity is higher for divisively normalized input. We prove that when normalization is exact and the number of parallel synapses grows with the number of input points, every labeling of the input is separable. In numerical simulations, we observe that this increased capacity remains when normalization is approximate. Finally, we reduce the classification capacity problem for this neuron model to a standard perceptron capacity problem with correlated input. Based on this result, we find a lower bound on the classification capacity for this neuron model using classical statistical mechanics techniques (Gardner and Derida, 1988). Overall, this work gives us insight into the computational role of both parallel synapses and divisive normalization.

Topic Area: Methods, Tools, Theory & Neural Coding