{"ID":2832358,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.05342","arxiv_id":"2512.05342","title":"First Demonstration of Second-order Training of Deep Neural Networks with In-memory Analog Matrix Computing","abstract":"Second-order optimization methods, which leverage curvature information, offer faster and more stable convergence than first-order methods such as stochastic gradient descent (SGD) and Adam. However, their practical adoption is hindered by the prohibitively high cost of inverting the second-order information matrix, particularly in large-scale neural network training. Here, we present the first demonstration of a second-order optimizer powered by in-memory analog matrix computing (AMC) using resistive random-access memory (RRAM), which performs matrix inversion (INV) in a single step. We validate the optimizer by training a two-layer convolutional neural network (CNN) for handwritten letter classification, achieving 26% and 61% fewer training epochs than SGD with momentum and Adam, respectively. On a larger task using the same second-order method, our system delivers a 5.88x improvement in throughput and a 6.9x gain in energy efficiency compared to state-of-the-art digital processors. These results demonstrate the feasibility and effectiveness of AMC circuits for second-order neural network training, opening a new path toward energy-efficient AI acceleration.","short_abstract":"Second-order optimization methods, which leverage curvature information, offer faster and more stable convergence than first-order methods such as stochastic gradient descent (SGD) and Adam. However, their practical adoption is hindered by the prohibitively high cost of inverting the second-order information matrix, pa...","url_abs":"https://arxiv.org/abs/2512.05342","url_pdf":"https://arxiv.org/pdf/2512.05342v1","authors":"[\"Saitao Zhang\",\"Yubiao Luo\",\"Shiqing Wang\",\"Pushen Zuo\",\"Yongxiang Li\",\"Lunshuai Pan\",\"Zheng Miao\",\"Zhong Sun\"]","published":"2025-12-05T00:52:46Z","proceeding":"cs.ET","tasks":"[\"cs.ET\",\"cs.AR\",\"cs.NE\"]","methods":"[\"Convolutional Neural Network\"]","has_code":false}