{"ID":6024424,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-08T22:07:47.709502596Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.05457","arxiv_id":"2607.05457","title":"Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests","abstract":"Deep neural networks often contain substantial hidden-state redundancy, but most compression methods operate directly on weights, neurons, or quantised representations without explicitly characterising the dynamical role of internal states. This paper proposes a controllability-observability framework for empirical state-order reduction of deep neural networks. By viewing a trained network as a depth-indexed nonlinear dynamical system, we construct data-driven reachability, observability, and balanced Gramians from hidden-state snapshots and output Jacobians. The resulting A/B/C tests estimate layer-wise reachable, observable, and jointly reachable--observable ranks. These ranks are then used not only as diagnostic measures of hidden-state redundancy, but also as actual compressed layer widths for realised reduced networks. Experiments on MNIST and CIFAR-10 compare the proposed balanced realisation against projection-based reduction, unstructured pruning, structured pruning, low-rank SVD, dynamic INT8 quantisation, and linear baselines. On MNIST, a four-layer SiLU DNN is reduced from state order 1024 to 277, giving 72.95% state compression and 73.48% parameter compression, while maintaining 95.45% accuracy compared with 96.60% for the full model. On CIFAR-10, a larger SiLU DNN is reduced from state order 4608 to 1339, giving 70.94% state compression and 83.09% parameter compression, while preserving accuracy from 54.45% to 54.44% and reducing CUDA inference latency by approximately 3X. The results show that balanced reachable-observable ranks provide a principled empirical minimal-realisation criterion for designing compact neural architectures with little or no loss in accuracy.","short_abstract":"Deep neural networks often contain substantial hidden-state redundancy, but most compression methods operate directly on weights, neurons, or quantised representations without explicitly characterising the dynamical role of internal states. This paper proposes a controllability-observability framework for empirical sta...","url_abs":"https://arxiv.org/abs/2607.05457","url_pdf":"https://arxiv.org/pdf/2607.05457v1","authors":"[\"Anis Hamadouche\",\"Amir Hussain\"]","published":"2026-07-05T22:07:22Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"math.DS\"]","methods":"[]","has_code":false}