hiw3c 日报精选
arXiv cs.AI/CL/LG 网页快照

经典能量学习推理速度比循环模型快1000倍

原文标题 · Classical Energies Learn to Reason 1,000x Faster than Looped Models

作者: Benjamin Hoover, Hugh Blayney, James C. R. Whittington | 循环模型已成为推理的重要范式,在如数独极端版和迷宫唯一版等简单推理谜题上取得了成功。然而,这些模型可能不稳定、训练速度慢,且解决简单谜题时需使用大量参数和FLOPs。基于能量下降方法不仅能缓解这些问题,还能带来优势:内置早期停止机制、适应问题需求的计算

来源:arXiv cs.AI/CL/LG · arxiv.org

正文为英文,可一键机器翻译(仅首次需要等待)

Computer Science > Artificial Intelligence

Title: Classical Energies Learn to Reason 1,000x Faster than Looped Models

Abstract: Looped models have emerged as a prominent paradigm for reasoning, with successes on simple reasoning puzzles such as Sudoku-Extreme and Maze-Unique. However, these models can be unstable, slow to train, and use many parameters and FLOPs to solve simple puzzles. Descent on an energy not only mitigates these issues but also offers advantages: built-in early stopping, compute that adapts to problem difficulty, fast training without differentiating through a solver, and interpretable weights that learn the constraints of the puzzle. Here we demonstrate this with a learned version of Hopfield & Tank's energy from the 80s, which we term CRAM, solves 99.996% of Sudoku-Extreme and 100% of Maze-Unique test puzzles after 42 ms and 132 ms of training on one GPU, at least 1,000x faster than looped models on these tasks. CRAM also matches baseline looped models at their highest accuracy on Sudoku-Extreme using at least 282x fewer inference FLOPs. Our work suggests two things. First, that energies are an extremely promising direction for training reasoning from data. Second, a caution against using trivially solvable benchmarks like Sudoku and Maze to test the scalability of looped reasoning techniques.

Submission history

Access Paper:

Additional Features

Current browse context:

References & Citations

BibTeX formatted citation

Bookmark

Bibliographic and Citation Tools

Code, Data and Media Associated with this Article

Demos

Recommenders and Search Tools

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .