Using Docker - vLLM
vLLM offers an official Docker image for deployment. The image can be used to run OpenAI compatible server and is available on Docker Hub as vllm/vllm-openai.
#vllm
Public notes from activescott tagged with #vllm
Public notes from activescott tagged with #vllm
vLLM offers an official Docker image for deployment. The image can be used to run OpenAI compatible server and is available on Docker Hub as vllm/vllm-openai.