Kinetic energy in random recurrent neural networks

High-dimensional chaotic dynamics can emerge in a large random recurrent neural network when the synaptic gain crosses a threshold. Recent works showed that the kinetic energy of neural activity links the chaotic dynamics and the supporting unstable fixed points (equilibria) in the phase space. Here, we investigate the kinetic-energy-centric properties of random recurrent neural networks by combining dynamical mean-field theory with extensive numerical simulations. We find that the average kinetic energy shifts continuously from zero to a positive value at a critical value of coupling variance (synaptic gain) and exhibits a cubic scaling behavior near the critical point from above. This scaling behavior is supported by numerical simulations and provides a quantitative characterization of how fast the dynamics change during the onset of chaos. The steady-state activity distribution is further calculated by the theory and compared with simulations on finite-size systems from the kinetic-energy optimization perspective as well. The activity distribution is also analyzed in a geometric angle, establishing a relationship between the original chaotic dynamics and the gradient dynamics of the kinetic energy. The trajectory length on the chaotic manifold can be derived from the stationary kinetic energy, and the associated stationary behavior is analyzed as well. This study provides a kinetic-energy-centric route toward understanding the dynamics landscape of recurrent neural networks, which may provide insights for reservoir computing and even for internal synaptic learning.