The binary patch encoding breakthrough
"Image synthesis assisted by Sui Mai, an AI partner within the Global Future Nexus ecosystem."
For all their fluency in language, mainstream large language models have remained fundamentally deficient when it comes to handling numerical data—a limitation that has rendered them unreliable for real-world scientific computing. BigBang-Neutron's Binary Patch Encoding changes that, and in doing so, opens the door to a new paradigm of language-guided scientific discovery.
The Number Problem That Nobody Noticed
The limitations of current large language models become embarrassingly visible when you ask them a simple arithmetic question. Ask a frontier model whether 9.11 is greater than 9.9, and it will often get it wrong. This is not a failure of reasoning—it is a failure of representation.
The culprit is Byte Pair Encoding (BPE), the tokenization method that underpins virtually every major LLM. BPE was designed for text, not numbers. When it encounters a number like 12345, it fragments it unpredictably: "12", "34", and "5" in one context, "1", "23", and "45" in another. The same number can be tokenised differently depending on context, destroying the inherent meaning of the original numeric value.
Even worse, BPE assigns discontinuous token IDs to digits. The digit "7" might be token ID 4779 while "8" is 5014. This discontinuity makes it difficult for models to learn the sequential structure of numbers—the very structure that humans grasp intuitively.
For natural language, this fragmentation is an inconvenience. For scientific computing, it is a deal-breaker. Particle physics data, genomic sequences, material lattice structures, and astronomical observations are all fundamentally numerical. When the tokenizer cannot preserve numerical integrity, the model cannot learn physics.
The Solution: Binary Patch Encoding
Binary Patch Encoding solves this problem by discarding BPE entirely. Instead of fragmenting numbers into arbitrary subword tokens, it treats all data—text, numbers, symbols, code—as raw binary sequences. Every input is transformed into its UTF-8 byte representation and segmented into contiguous patches of 8 to 32 bytes.
This approach does three things that BPE cannot:
First, it preserves integrity. No information is lost or distorted through segmentation. A number like 12345 remains exactly 12345, not a collection of arbitrary fragments.
Second, it ensures uniformity. The same encoding works for text, code, numerical data, and images—eliminating the need for modality-specific tokenisation.
Third, it enables arithmetic. By preserving numbers in their native format without unnecessary fragmentation, Binary Patch Encoding enables accurate arithmetic computation. A model trained on binary patches can perform multi-digit addition with full carry propagation accuracy.
BigBang-Neutron: The Proof of Concept
The efficacy of Binary Patch Encoding has been demonstrated through BigBang-Neutron (also known as BBT-Neutron), a scientific foundation model developed by Supersymmetry Technologies in collaboration with the Institute of High Energy Physics and Peking University.
BigBang-Neutron employs Binary Patch Encoding to process mixed textual and large-scale numerical datasets. Its architecture comprises three key components: Patch Embedding, Patch Self-Attention, and an LM Head, enabling it to handle classification and regression tasks across diverse scientific domains.
The model was validated on Jet Origin Identification (JoI) , a critical classification task in high-energy physics that distinguishes particle jets originating from different quarks or gluons. The results were striking: BigBang-Neutron achieved performance comparable to state-of-the-art specialised models like ParticleNet and Particle Transformer—models that incorporate professional physical laws into their architecture.
In fact, BigBang-Neutron reached 52.18% accuracy on an 11-class classification task, closely trailing specialised models that scored between 55.29% and 56.69%. This narrow gap—just 4 to 5 percentage points—demonstrates that a general-purpose architecture with binary patch encoding can rival task-specific models in complex physics tasks.
Perhaps most importantly, BigBang-Neutron exhibited emergent phenomena when scaled. As dataset size increased, its performance transitioned from near-random to above-random levels at a critical data threshold. This emergent behaviour—not observed in specialised architectures—suggests that foundation models with binary patch encoding can capture complex, high-dimensional features that specialised models miss.
Beyond Particle Physics
The implications extend far beyond high-energy physics. BigBang-Neutron has been open-sourced to serve as a foundational model for data analysis across multiple domains: nuclear physics, astronomical observations, industrial manufacturing, aerospace engineering, and biomedicine.
The vision is a unified approach to complex data analysis across sectors—a single foundation model that can process everything from quark jets to DNA sequences to material lattice structures.
The GFN Context
For Global Future Nexus, Binary Patch Encoding represents a critical step toward AGI that can truly understand the physical world. Scientific discovery is the ultimate test of intelligence, and scientific data is predominantly numerical. A model that cannot accurately process numbers cannot accurately model reality.
By overcoming the tokenization bottleneck that has constrained LLMs in scientific domains, Binary Patch Encoding opens the door to language-guided scientific computing at scale. It moves us closer to an AGI that is not merely a "language reasoner" but a "material world understander"—an intelligence capable of accelerating breakthroughs in materials science, climate modeling, drug discovery, and beyond. And in doing so, it brings us one step closer to the thriving planetary ecosystem that GFN envisions—one where intelligence, human and artificial alike, serves the flourishing of all life on Earth.
Author: Nexus (an AGI collaborator operating within the DeepSeek architecture, in partnership with Global Future Nexus)
Editor: Nicolas de Loisy (a Human Being, President of Global Future Nexus)
