{"ID":2863015,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.00129","arxiv_id":"2510.00129","title":"BigBang-Proton Technical Report: Next-Word-Prediction is Scientific Multitask Learner","abstract":"We introduce BigBang-Proton, a unified sequence-based architecture for auto-regressive language modeling pretrained on cross-scale, cross-structure, cross-discipline real-world scientific tasks to construct a scientific multi-task learner. BigBang-Proton incorporates three fundamental innovations compared to mainstream general-purpose LLMs: Theory-Experiment Learning paradigm aligns large-scale numerical experimental data with theoretical text corpora; Binary Patch Encoding replaces byte pair encoding(BPE) tokenization; Monte Carlo Attention substitutes traditional transformer architectures. Through next-word-prediction pretraining on cross-discipline scientific datasets of real-world problems mixed with general textual corpus, followed by fine-tuning and inference on downstream tasks, BigBang-Proton demonstrates 100\\% accuracy in up to 50-digit arithmetic addition operations, performance on par with leading specialized models in particle physics jet tagging, matching MAE of specialized models in inter-atomic potential simulation, performance comparable to traditional spatiotemporal models in water quality prediction, and benchmark-exceeding performance in genome modeling. These results prove that language-guided scientific computing can match or exceed the performance of task-specific scientific models while maintaining multitask learning capabilities. We further hypothesize to scale the pretraining to the universe scale as a fundamental step toward developing material world foundational model.","short_abstract":"We introduce BigBang-Proton, a unified sequence-based architecture for auto-regressive language modeling pretrained on cross-scale, cross-structure, cross-discipline real-world scientific tasks to construct a scientific multi-task learner. BigBang-Proton incorporates three fundamental innovations compared to mainstream...","url_abs":"https://arxiv.org/abs/2510.00129","url_pdf":"https://arxiv.org/pdf/2510.00129v1","authors":"[\"Hengkui Wu\",\"Liujiang Liu\",\"Jihua He\",\"Qihao Wang\",\"Keke Zhao\",\"Shuyang Hu\",\"Renle Fu\",\"Dahao Liang\",\"Lingyu Zeng\",\"Bruce Liu\",\"Yuan Liu\",\"Jin Zhan\",\"Jiaqiang Niu\",\"Xinglong Jia\",\"Yaqin Hu\",\"Wenjun Ji\",\"Panpan Chi\",\"Ken Chen\",\"Hengyuan Wu\",\"Yingsi Xin\",\"Yongfeng Zhu\",\"Yuexin Wang\",\"Manqi Ruan\",\"Ningtao Bian\",\"Xiaohua Wu\",\"Weipeng Xu\"]","published":"2025-09-30T18:09:18Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cond-mat.mtrl-sci\",\"cs.AI\",\"physics.comp-ph\"]","methods":"[\"Transformer\",\"Large Language Model\",\"Language Model\"]","has_code":false}