~ / serving-runtimes / serve-any-open-model / vllm-structured-decisions
vLLM Structured Decisions
Upstream vLLM serves /v1/systemone directly: decision models with a supported read strategy run as pooling models and answer choice / noul / score from label log-prob mass (--logprobs-mode raw_logprobs), so the shared state rides prefix caching like any chat prefix. What Clef-Flash and H2O's shim build on, as an official endpoint rather than a per-model hack; distinct from the vLLM Semantic Router listed under Applications.
- section
- Serving & runtimes
- group
- Serve any open model
- source
- docs.vllm.ai
- added to list
- oct 24