▌system-one

~ / serving-runtimes / serve-any-open-model / vllm-structured-decisions

vLLM Structured Decisions

Upstream vLLM serves /v1/systemone directly: decision models with a supported read strategy run as pooling models and answer choice / noul / score from label log-prob mass (--logprobs-mode raw_logprobs), so the shared state rides prefix caching like any chat prefix. What Clef-Flash and H2O's shim build on, as an official endpoint rather than a per-model hack; distinct from the vLLM Semantic Router listed under Applications.

section
Serving & runtimes
group
Serve any open model
source
docs.vllm.ai
added to list
oct 24

also in Serve any open model