#js + #llama.cpp

You can use the OpenAI Compatible Provider package to use language model providers that implement the OpenAI API.

Below we focus on the general setup and provider instance creation. You can also write a custom provider package leveraging the OpenAI Compatible package.

We provide detailed documentation for the following OpenAI compatible providers:

LM Studio
NIM
Heroku
Clarifai

The general setup and provider instance creation is the same for all of these providers.