Escaping Rounding Barriers in Large Language Model Quantization via Transverse-Field Ising Optimization
Main Article Content
Abstract
Weight-only post-training quantization (PTQ) pipelines such as AWQ ultimately commit to a deterministic rounding decision for every scalar weight, and recent discrete coordinate-descent quantizers refine those decisions one weight at a time. We show that both are blocked by the same obstacle: correlated activation channels create multi-spin energy barriers in the discrete rounding landscape — configurations in which no single rounding flip helps, yet a small group of coordinated flips strictly reduces the layer reconstruction error. We characterize these barriers exactly under the empirical activation metric and prove that they arise only from frustration between local quantization fields and cross-channel couplings. Building on this analysis we propose Transverse-Field Ising Correction (TFIC), a barrier-aware discrete optimizer that post-processes any AWQ-style pipeline. TFIC casts fixed-grid rounding as a classical Ising ground-state problem, selects the spins near a rounding boundary as a mobile pool, drives that pool to a single-flip local minimum by exact coordinate descent, and then crosses barriers with cluster moves that flip small groups of coupled spins together. Every accepted move is scored against the exact reconstruction energy, and a robust per-layer noise-floor acceptance rule prunes flips whose apparent gain is within calibration-estimation noise; together these guarantee that each layer’s calibration reconstruction error is monotonically non-increasing. TFIC requires no gradients and no architectural changes. On Mistral-7B and Llama-3-8B, applying TFIC on top of a 4-bit AWQ baseline consistently reduces WikiText-2 and C4 perplexity, at a per-layer cost of tens of seconds and with no inference-time overhead.
Keywords
error correction, post-training quantization, quantumm inspired
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
ransformers, in International Conference on Learning Representations (ICLR), 2023.
[2] J. Lin, J. Tang, H. Tang, S. Yang, W.-M. Chen, W.-C. Wang, G. Xiao, X. Dang, C. Gan, and S. Han, AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration, in Proceedings of Machine Learning and Systems, 2024.
[3] G. Xiao, J. Lin, M. Seznec, H. Wu, J. Demouth, and S. Han, SmoothQuant: Accurate and efficient post-training quantization for large language models, in International Conference on Machine Learning (ICML), 2023, pp. 38087–38099.
[4] Y. Sun, R. Liu, H. Bai, H. Bao, K. Zhao, Y. Li, J. Hu, X. Yu, L. Hou, C. Yuan, X. Jiang, W. Liu, and J. Yao, FlatQuant: Flatness matters for LLM quantization, in International Conference on Machine Learning (ICML), 2025, pp. 57587–57613.
[5] K. Behdin, A. Acharya, A. Gupta, Q. Song, S. Zhu, S. Keerthi, and R. Mazumder, QuantEase: Optimization-based quantization for language models, arXiv preprint arXiv:2309.01885, 2023. https://doi.org/10.48550/arXiv.2309.01885.
[6] P. A. Nair and A. S. Suggala, CDQuant: Greedy coordinate descent for accurate LLM quantization, arXiv preprint
rXiv:2406.17542, 2024. https://doi.org/10.48550/arXiv.2406.17542.
[7] A. Zhang, Z. Yang, N. Wang, Y. Qi, J. Xin, X. Li, and P. Yin, COMQ: A backpropagation-free algorithm for post-training quantization, IEEE Access, vol. 13, pp. 99879–99891, 2025. https://doi.org/10.1109/ACCESS.2025.3576737.
[8] M. Nagel, R. A. Amjad, M. Van Baalen, C. Louizos, and T. Blankevoort, Up or down? Adaptive rounding for post-training quantization, in International Conference on Machine Learning (ICML), 2020, pp. 7197–7206.
[9] Y. Li, R. Gong, X. Tan, Y. Yang, P. Hu, Q. Zhang, F. Yu, W. Wang, and S. Gu, BRECQ: Pushing the limit of post-training quantization by block reconstruction, in International Conference on Learning Representations (ICLR), 2021.
[10] W. Cheng, W. Zhang, H. Shen, Y. Cai, X. He, L. Kaokao, and Y. Liu, Optimize weight rounding via signed gradient descent for the quantization of LLMs, in Findings of the Association for Computational Linguistics: EMNLP 2024, 2024, pp. 11332–11350.
[11] S. Cha, H. Chen, D. Kim, H. Zhang, K. Chan, G. de Veciana, and H. Vikalo, CoreQ: Learning-free mismatch correction and successive rounding for quantization, arXiv preprint arXiv:2602.05902, 2026. https://doi.org/10.48550/arXiv.2602.05902.
[12] A. Lucas, Ising formulations of many NP problems, Frontiers in Physics, vol. 2, pp. 5, 2014.
[13] T. Kadowaki and H. Nishimori, Quantum annealing in the transverse Ising model, Physical Review E, vol. 58, no. 5, pp. 5355–5363, 1998.
[14] H. Goto, K. Tatsumura, and A. R. Dixon, Combinatorial optimization by simulating adiabatic bifurcations in nonlinear Hamiltonian systems, Science Advances, vol. 5, no. 4, pp. eaav2372, 2019.
[15] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al., Mistral 7B, arXiv preprint arXiv:2310.06825, 2023. https://doi.org/10.48550/arXiv.2310.06825.
[16] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, et al., The Llama 3 herd of models, arXiv preprint arXiv:2407.21783, 2024. https://doi.org/10.48550/arXiv.2407.21783.