An open-source project called TypeLLM gives large language models a way to guarantee their output matches a predefined schema, an enum, boolean or number, without changing the model's architecture, according to its GitHub repository.
The framework restricts a model to a single token when it needs to pick from a fixed set of categories, keeping computational cost low while ruling out responses outside the allowed schema, the repository says. It supports dependency-aware execution across schema fields with key-value cache reuse, an optional reasoning mode with a token budget per field, and permutation averaging to cut down on bias from the order options are presented in.
Built on the SGLang inference engine, TypeLLM works with open-source models such as Qwen3.8-27B and also supports vision-language models with image inputs, according to the repository. On the project's own typed-decisions benchmark, it scored 195 of 231 without a reasoning step and 228 of 231 with reasoning enabled.
The project is Apache-2.0 licensed and has drawn 783 stars and 52 forks on GitHub since it was created this month.
Structured output is one of the more brittle parts of building on an LLM: models drift outside the schema, and validation-and-retry loops burn tokens and add latency. A constraint enforced at the token level, rather than after the fact, is a more direct fix, if the accuracy numbers hold up outside one benchmark.