Hugging Face has disclosed a bug in its Transformers library that let the decoder of the UMT5 model attend to future tokens during training, so the model could see the answer it was meant to predict. The change came on July 30, 2026, when Transformers enabled PyTorch's scaled dot-product attention for T5. The model still ran and no tests crashed, so the bug remained in the main branch for almost two months until it was fixed on September 28. When Hugging Face later replayed those commits, a small UMT5 test model running on CPU flagged the regression on every one of them. The company noted that a standard forward-pass check would have missed it. To catch problems like this earlier, Hugging Face is proposing a standard called Model Integration Contracts, packaged as transformers-mic. Each contract has three parts: a fixture, which is a shrunken model built from a real checkpoint's configuration with fewer and narrower layers but every layer type kept; a usage path, which is a runnable example such as text generation or a training step; and baselines, which are outputs recorded from that example and reviewed by a person. The experimental UMT5 fixture uses 66 MB of weights, compared with 1,227 MB for the source checkpoint, so it can run on a cheap CPU runner. The company said the approach complements its full test suite rather than replacing it, which includes more than 88,000 CPU tests on main and more than 200,000 GPU tests in a daily run. The contracts are pinned to frozen model revisions and recorded in a fleet dataset on the Hub, so each CI run can be reproduced later. For Australian businesses and professionals, the direct impact is small. The bug affected only builds from the main branch between July 30 and September 28, 2026, so teams that trained or deployed UMT5 during that window may want to check whether affected code was involved. The proposal itself is open and free to use, and it does not change obligations under the Privacy Act. The practical lesson for teams that fine-tune or serve open-source models is to pin library versions and run a basic check that generation and loss stay stable after each upgrade.