{"ID":6267743,"CreatedAt":"2026-07-10T01:11:38.759438437Z","UpdatedAt":"2026-07-11T18:20:13.703712842Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.07855","arxiv_id":"2607.07855","title":"NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RL","abstract":"Hierarchical Implicit Q-Learning (HIQL), an offline goal-conditioned RL method, selects subgoals by value-function advantages alone. This rule has two coupled failure modes. Optimistic bias treats lucky stochastic outcomes as skillful choices, and mode collapse reduces a multi-modal subgoal distribution to a single Gaussian mean that often falls in unreachable regions. We propose NFTR (Normalizing Flows subgoal policies with Triangle-slack Reweighting). A conditional Normalizing Flow replaces the Gaussian policy, and a closed-form mode-averaging result identifies NFs as the minimal generative class for AWR-based subgoal selection. A triangle slack score, built on the architectural triangle inequality without relying on distance accuracy, multiplicatively corrects the AWR weight to downweight subgoals whose detour cost exceeds average reachability. Triangle-slack vanishes on geodesics in deterministic MDPs and remains a conservative upper bound on composability violation under stochastic dynamics. The RWDR objective preserves AWR's population-level monotonic improvement and admits a three-term suboptimality decomposition. Together, these two ingredients yield subgoal selection that provably avoids the Gaussian collapse described above and remains stable under stochastic dynamics. GitHub page: https://github.com/erdemtbao/NFTR","short_abstract":"Hierarchical Implicit Q-Learning (HIQL), an offline goal-conditioned RL method, selects subgoals by value-function advantages alone. This rule has two coupled failure modes. Optimistic bias treats lucky stochastic outcomes as skillful choices, and mode collapse reduces a multi-modal subgoal distribution to a single Gau...","url_abs":"https://arxiv.org/abs/2607.07855","url_pdf":"https://arxiv.org/pdf/2607.07855v1","authors":"[\"Erdemt Bao\",\"Xing Lei\",\"Jun Chen\"]","published":"2026-07-08T18:33:48Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[]","has_code":false,"code_links":[{"ID":614095,"CreatedAt":"2026-07-10T01:11:38.759438437Z","UpdatedAt":"2026-07-10T01:11:38.759438437Z","DeletedAt":null,"paper_id":6267743,"paper_url":"https://arxiv.org/abs/2607.07855","paper_title":"NFTR: From Provable Mode-Averaging to Geodesic Subgoal Selection in Offline Goal-Conditioned RL","repo_url":"https://github.com/erdemtbao/NFTR","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}