Limits on fast, high-dimensional information processing in recurrent circuits

V Ruetten, and G Hennequin
COSYNE, 2017  

Abstract


Fast and reliable processing of information in the brain is critical for decision-making.Local cortical circuits receive high-dimensional inputs, and produce high-dimensional outputs [Gao & Ganguli (2015)]. What are the fundamental limits on rapid multiplexing in recurrent neuronal networks in the presence of either strong noise or weak signals, and what are the main tradeoffs? Previous work has characterized the information capacity of model neuronal networks in the context of short-term memory, and only for one-dimensional input signals [Ganguli et al. (2008); Lim and Goldman (2012)]. Here, we give a complete characterization of the Shannon information capacity for the broad class of normal linear networks. These networks are equivalent to a set of decoupled information channels that fade and/or oscillate with varying time constants. We calculate the optimal time constants for each channel as a function of the fraction of the total input signal variance that goes through it, and the amount of time the stimulus must be optimally represented. We then explore the behaviour of nonnormal networks, which include the family of excitation-inhibition networks that more naturally model the dynamics of cortical circuits. These networks can outperform normal networks on short timescales, by transiently increasing the signal-to-noise ratio (SNR) through fast, selective amplification of the input. This is not possible in normal networks, in which we could show that the SNR must decay with time. Finally, directly optimizing network connectivity for information transmission yields nonnormal networks very similar to those we have previously proposed to explain movement- related activity in primary motor cortex. We therefore suggest that the cortex might employ nonnormal dynamics to process fast-varying signals that are either weak or mixed with task-unrelated activity.