{"ID":2840715,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.13602","arxiv_id":"2511.13602","title":"Nonparametric Estimation of Joint Entropy via Partitioned Sample-Spacing","abstract":"We propose a nonparametric estimator of multivariate joint entropy based on partitioned sample spacing (PSS). The method extends univariate spacing ideas to $\\mathbb{R}^{d}$ by partitioning into localized cells and aggregating within-cell statistics, with strong consistency guarantees under mild conditions. In benchmarks across diverse distributions, PSS consistently outperforms $k$-nearest neighbor estimators and achieves accuracy competitive with recent normalizing flow-based methods, while requiring no training or auxiliary density modeling. The estimator scales favorably in moderately high dimensions ($d = 10$--$40$) and shows particular robustness to correlated or skewed distributions. These properties position PSS as a practical and reliable alternative to both $k$NN and NF-based entropy estimators, with broad utility in information-theoretic machine learning tasks such as total-correlation estimation, representation learning, and feature selection.","short_abstract":"We propose a nonparametric estimator of multivariate joint entropy based on partitioned sample spacing (PSS). The method extends univariate spacing ideas to $\\mathbb{R}^{d}$ by partitioning into localized cells and aggregating within-cell statistics, with strong consistency guarantees under mild conditions. In benchmar...","url_abs":"https://arxiv.org/abs/2511.13602","url_pdf":"https://arxiv.org/pdf/2511.13602v2","authors":"[\"Jungwoo Ho\",\"Sangun Park\",\"Soyeong Oh\"]","published":"2025-11-17T17:05:34Z","proceeding":"math.ST","tasks":"[\"math.ST\",\"stat.ML\"]","methods":"[]","has_code":false}