Using Docker - vLLM
vLLM offers an official Docker image for deployment. The image can be used to run OpenAI compatible server and is available on Docker Hub as vllm/vllm-openai.
#vllm + #llm
Public notes from activescott tagged with both #vllm and #llm