An open-source project called Laya makes structured decisions, classifications, scores or yes-or-no answers, over text in a single forward pass instead of generating text token by token, according to its GitHub repository. The project has drawn more than 26,000 stars since its release this month.
Laya is built to replace two common approaches: slow autoregressive generation that carries parsing risk, or lighter classifiers that don't support many languages, the repository says. It works across more than 100 languages by automatically routing a request to an English or multilingual model checkpoint, with script detection happening in under a millisecond.
The system reports single-question latency of 33 milliseconds, or 7.2 milliseconds per question when run in a batch on an Nvidia T4 GPU, which its documentation says is six to seven times faster than comparable systems. It uses an encoder-only architecture across three model checkpoints and trains with reinforcement learning against strictly proper scoring rules to keep its confidence scores calibrated.
On the project's own typed-decisions benchmark, its fine-tuned checkpoint scored 0.766 accuracy against a reported 0.727 for a competing system, with roughly three times better calibration, according to the repository. The project is Apache 2.0 licensed.
A lot of agent pipelines call a full LLM just to make a yes-or-no or pick-one-of-five decision, then pay for both the generation and the parsing. A model built to skip the generation step entirely is a bet that most of those calls never needed an LLM's language ability in the first place.